Reliability of a Tablet Computer-Based Dyslexia Screening Application Using an Eye-Tracking System

Article information

Ann Child Neurol. 2024;32(2):99-104
Publication date (electronic) : 2024 March 21
doi : https://doi.org/10.26815/acn.2023.00360
1Sky Pediatric Clinic, Seoul, Korea
2Yonsei Gomdori Pediatric Clinic, Seoul, Korea
3Department of Pediatrics, College of Medicine, The Catholic University of Korea, Seoul, Korea
Corresponding author: Tae-Hoon Eom, MD Department of Pediatrics, College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Korea Tel: +82-2-2258-6184, Fax: +82-2-537-4544 E-mail: good1976@hanmail.net
Received 2023 November 29; Revised 2024 January 16; Accepted 2024 January 18.

Abstract

Purpose

The early detection and management of dyslexia are crucial for preventing irreversible educational gaps and various negative consequences for affected students. However, diagnosing dyslexia is challenging because it requires a comprehensive assessment. Dyslexia screening tests that utilize fast, automated, computer-based technology can be useful for early identification and management. In this paper, we introduce a tablet computer-based dyslexia screening application that uses an eye-tracking system and verify its reliability.

Methods

The study included 200 participants between 8 and 13 years of age from an elementary school, all of whom underwent dyslexia screening tests twice. The screening was conducted using the VisualCamp SeeSo eye-tracking Android Software Development Kit v3.0.0, implemented on Samsung Galaxy Tab S5e tablets. The eye-tracking system measured reading speed by gaze, mean gaze fixation time, gaze fixation frequency, saccadic length, and regression ratio. To assess the reliability of the two sets of measurements, the intraclass correlation coefficient (ICC) was employed.

Results

Excellent reliability was found for measurements of gaze fixation frequency (ICC=0.83), gaze fixation mean time (ICC=0.82), and reading speed by gaze (ICC=0.76), and good reliability for measurements of regression ratio (ICC=0.75) and saccadic length (ICC=0.72).

Conclusion

This study demonstrated that the tablet computer-based dyslexia screening application reliably measured eye movements in subjects with dyslexia. Furthermore, the application proved to be highly reliable and potentially suitable for use in clinical or school settings, eliminating the need for a laboratory environment and extensive equipment.

Introduction

Dyslexia originates from the Greek words dys, which means “difficulty,” and lexia, meaning “words” [1,2]. The term dyslexia was first used in 1883 by German ophthalmologist Berlin [3], who reported six patients with reading difficulties in his research. Even earlier, in 1877, another German physician, Kussmaul [4], reported a patient who had normal intelligence, speech, and vision but had reading difficulty, which he called “word blindness.” Over the past 100 years, the term “dyslexia” has become increasingly common; nonetheless, the existence and definition of dyslexia have been extensively debated. Currently, through the contributions of many researchers and recent advances in neuroimaging technology, dyslexia is considered a disorder caused by neurobiological defects associated with reading skills [5,6].

In the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), a specific learning disorder is defined as a neurodevelopmental disorder characterized by difficulty in learning and using academic skills such as reading, writing, and arithmetic. The DSM-5 diagnostic criteria for a specific learning disorder are summarized as follows. (1) Difficulties learning and using academic skills, as indicated by the presence of at least one of the following symptoms that have persisted for at least 6 months, despite the provision of interventions that target those difficulties. (2) The affected academic areas are substantially and quantifiably below those expected for the individual’s chronological age, and cause significant interference with academic or occupational performance, or with activities of daily living. (3) The learning difficulties begin during school-age years but may not become fully manifest until the demands for those affected academic skills exceed the individual’s limited capacities. (4) The learning difficulties are not better accounted for by intellectual disabilities, uncorrected visual or auditory acuity, other mental or neurological disorders, psychosocial adversity, lack of proficiency in the language of academic instruction, or inadequate educational instruction.

In addition, the term dyslexia is used as an alternative to specify a pattern of learning difficulties characterized by problems with accurate or fluent word recognition, poor decoding, and poor spelling abilities [7]. Furthermore, dyslexia is included in the umbrella category of Specific Learning Disability under the U.S. Individuals with Disabilities Education Act (IDEA), and students with dyslexia can receive special education and services [8].

The early detection and management of dyslexia are crucial for preventing irreversible educational gaps and various negative consequences for affected students. However, diagnosing dyslexia is challenging because it requires a comprehensive assessment of word recognition, decoding, spelling, reading comprehension, and comprehension [9-12]. Currently, screening for dyslexia is not routinely performed in clinical settings. Furthermore, there are limitations for teachers in identifying dyslexic students in the classroom, which is also a significant burden for teachers.

Computerized screening methods with objective measures can help address these issues. In this context, several studies using eye movements to screen for dyslexia have been performed. These studies attempted to determine and apply measurable parameters of eye movement observed in dyslexia. Eye movements in dyslexic children differ from those in normal children [13-17]. Dyslexia is not an eye movement disorder; however, the differences in eye movement reflect problems of accuracy and fluency in the reading process of dyslexia. Recently, smartphones and tablet computers equipped with high-performance sensors and cameras have become widely available. Software on these smartphones and tablet computers could be a promising solution to screen for dyslexia.

Therefore, we developed a tablet computer-based dyslexia screening application software. The present study introduces this application and presents a validation of its reliability, as a preliminary step in the process of identifying subjects with dyslexia.

Materials and Methods

1. Reading materials in the dyslexia screening application

The Department of Korean Language Education at Chuncheon National University of Education collaborated with elementary school teachers to develop reading materials for each grade in the dyslexia screening application. The reading materials for each grade included two articles, and the articles were written with vocabulary, sentences, content, and grammar appropriate for the grade. Each article was written to be as short as possible to minimize the influence of memory on comprehension. Furthermore, the content of the articles was designed to be accessible and understandable without requiring specific background knowledge.

2. Devices and technology used in the dyslexia screening application

The dyslexia screening application was run on Samsung Galaxy Tab S5e tablets (Samsung Electronics, Suwon, Korea). The tablet has a resolution of 1,600×2,560 pixels, a 10.5-inch display, a Qualcomm Snapdragon 670 chipset (SDM 670, Qualcomm, San Diego, CA, USA), and a front-facing selfie camera (8 MP, f/2.0, 26 mm, 1/4", and 1.12 µm). The dyslexia screening application used the VisualCamp SeeSo eye-tracking Android Software Development Kit (SDK) v3.0.0 (VisualCamp, Seoul, Korea, https://seeso.io).

3. Participants and study protocol

This study included 200 participants between 8 and 13 years of age attending an elementary school in Changwon, Gyeongsangnam Province. This study was approved by the Institutional Review Board of Seoul St. Mary's Hospital (KC23RISI0647). Written informed consent was obtained from all pariticipants. The grade distribution of the participants is summarized in Table 1. All participants underwent the dyslexia screening test twice using the procedure described below. Before reading the presented materials, the eye-tracking system was first calibrated for each participant. Fig. 1 shows the tablet computer screen, and the eye-tracking system was calibrated by looking at a total of five points in the center and four corners. Then, nine words were presented on the screen to guide the reading process, and the generated gaze data and word position values were used to ensure proper calibration (Fig. 2). After calibration and verification, the reading materials (including two articles) were presented to each participant via the tablet computer screen, and eye-tracking was performed (Fig. 3).

Grade distribution of the 200 children between 8 and 13 years of age attending an elementary school

Fig. 1.

The eye-tracking system was first calibrated by having the user look at a total of five points in the center and four corners on the tablet computer screen.

Fig. 2.

Nine words were presented on the screen, and the generated gaze data and word position values were used to ensure proper calibration.

Fig. 3.

After calibration and verification, the reading materials (including two articles) were presented to each participant via the tablet computer screen, and eye-tracking was performed.

4. Statistical analysis

Reading speed by gaze (words/minute), gaze fixation mean time (ms), gaze fixation frequency (fixation count/100 words), saccadic length (letters), and regression ratio (%) were measured by the eye-tracking system in the dyslexia screening application. Excel (Microsoft, Redmond, WA, USA), R version 4.3.0 (https://www.r-project.org/), and RStudio (2023.03.0+386, https://www.posit.co/) were used to process and analyze the data. The mean values with standard deviations of eye movement measurements using the eye-tracking system were recorded. The intraclass correlation coefficient (ICC) was used to assess the reliability between two data sets with repeated measurements. The ICC values were calculated using two-way mixed effects and a single measurement model with absolute agreement, and classified according to the following guidelines: values >0.75 indicated excellent reliability; 0.60–0.75, good reliability; 0.40–0.60, fair reliability; and <0.4, poor reliability [18].

Results

The measurements of gaze fixation frequency (fixation count/100 words; first 80.19±30.41, second 84.40±33.44, ICC=0.83), gaze fixation mean time (ms; first 378.09±81.14, second 373.27±79.38, ICC=0.82), and reading speed by gaze (words/minute; first 160.59±51.58, second 154.99±52.17, ICC=0.76) showed excellent reliability. The measurements of regression ratio (%; first 9.65±5.45, second 9.59±5.73, ICC=0.75) and saccadic length (letter; first 6.31±1.44, second 6.42±1.44, ICC=0.72) showed good reliability (Table 2).

The reliability of eye movement measurements using an eye-tracking system in a tablet computer-based dyslexia screening application

Discussion

The results of the present study showed that the eye-tracking system of the tablet computer-based dyslexia screening application could reliably track eye movements during the reading process. In particular, the ICC values of gaze fixation frequency and gaze fixation mean time showed excellent reliability, and all measurements showed good or excellent reliability. In the process of reading, eye movements consist of saccades and fixations. Saccades are rapid jumping eye movements that occur simultaneously in both eyes between two or more fixation phases [19]. During these fixation phases, the text's visual information is processed by the fovea at intervals between saccades [20,21]. Saccades can be directed both forward and backward during reading. Backward saccades, or regressions, are primarily used to revisit text content when reading difficulties arise [19,20,22]. In individuals with dyslexia, the patterns of eye movement during reading are well documented. These patterns include an increase in fixation time and frequency, a reduction in saccade length, a higher rate of backward saccades (regressions), and, as a result, a slower reading speed [13-17]. This study successfully measured these characteristic patterns of the reading process in dyslexia using a tablet-based application.

Dyslexia screening tests using fast, automated technology based on a computer can be useful for the early identification and management of individuals at risk of dyslexia. As mentioned above, the early detection and management of dyslexia are crucial for improving the prognosis and preventing irreversible consequences. In addition, the results of the present study showed the possibility of applying highly reliable dyslexia screening using fast, automated applications through readily available smartphones or tablets, unlike in a laboratory environment requiring a significant amount of equipment. The algorithm of the eye-tracking system (VisualCamp SeeSo v3.0.0) in this dyslexia screening application calculates the user's gaze data using the front camera image (red, green, and blue [RGB]) of a smartphone or tablet and precisely classifies the calculated gaze data into gaze fixations and saccades through an internal eye movement classification operation. This algorithm is expected to improve the accuracy of eye movement tracking, even when using smartphones or tablets with limited equipment and specifications. Notably, in this study, the ICC values of gaze fixation frequency, gaze fixation mean time, and reading speed by gaze showed excellent reliability for each calculated value. The ICC values for the regression ratio and saccadic length were also good. Moreover, as more gaze data on saccadic movements are collected and the algorithms are refined, the reliability is expected to increase further. With the recent advancements in eye movement tracking technology and the development of sophisticated computational algorithms, this method is also being explored for its potential application in diagnosing and studying a range of disorders, including autism spectrum disorder, schizophrenia, and Parkinson's disease [23-25].

Although eye movement abnormalities in dyslexia are well established, it is important to recognize that dyslexia is not a disorder of eye movements. Dyslexia is a neurodevelopmental disorder characterized by impairments in accurate or fluent word recognition, poor decoding, and poor spelling abilities. The characteristic eye movements are attributed to impairments with accurate and fluent word recognition due to poor decoding during the reading process [2,13,26]. Nonetheless, assessing eye movements in dyslexia provides objective, quantifiable insights into various reading processes and their associated impairments, without relying on the notion that dyslexia is primarily an eye movement disorder. The key benefit of eye movement assessments lies in their ability to directly evaluate the reading process and its individual components [27,28]. In contrast, current dyslexia screening tests, such as the Korean Language Base Reading Assessment and the Comprehensive Learning Test-Reading, do not assess the reading process itself. Instead, they measure only the comprehension issues that arise from underlying reading difficulties [13]. This limitation can lead to suboptimal screening and assessment, as it may be influenced by various factors unrelated to dyslexia that can impact reading comprehension. Thus, assessments may not accurately identify dyslexia, and the results should be interpreted with caution.

In summary, the results of the present study showed that the tablet computer-based dyslexia screening application, which we recently developed, reliably measured eye movements that are assessed in dyslexia screening. Furthermore, the tablet-based dyslexia screening application, which does not require a laboratory environment with a significant amount of equipment, was shown to be highly reliable and potentially applicable in a clinical or school setting. The tablet computer-based dyslexia screening application has distinct advantages over current screening tests for the assessment of eye movements in dyslexia and has practical implications associated with the reading process [13,27,28].

However, the present study also had several limitations. Dyslexia can coexist with conditions such as speech/language disorders, attention deficit-hyperactivity disorder, and executive function disorder [29,30]. In addition, intellectual disability or the presence of comorbid conditions may result in symptoms of dyslexia, and in these situations, abnormalities in eye movements may be observed during the reading process. However, in developing a dyslexia screening application, we focused more on evaluating impairments in the reading process than on specific diagnoses. For a screening test, a problem-focused approach is considered more reasonable and practical. In addition, this pilot study was designed to verify the reliability of this dyslexia screening application. Therefore, further studies are necessary to validate our dyslexia screening application and determine acceptable sensitivity and specificity values for use in clinical practice.

In conclusion, the tablet computer-based dyslexia screening application we recently developed showed high reliability in this study. Furthermore, by incorporating eye movement measurements, the computer-based dyslexia screening test has the potential to be a valuable tool for assessing the actual reading process. This study served as a preliminary investigation, confirming the reliability of the screening test. Moving forward, we plan to conduct a case-control study to further establish its validity. Ultimately, the clinical applicability of this or any other screening test will depend on achieving acceptable levels of sensitivity and specificity, which will require a large-scale study with a diverse cohort. Consequently, there is a need for continued efforts to develop superior early screening tools that can improve the prognosis for dyslexic patients and prevent irreversible outcomes.

Notes

Young-Hoon Kim is an editorial board member of the journal, but he was not involved in the peer reviewer selection, evaluation, or decision process of this article. No other potential conflicts.

Author contribution

Conceptualization: SP, THE, and YHK. Data curation: SP and JS. Formal analysis: SP and THE. Methodology: SP, JS, and THE. Project administration: SP, THE, and YHK. Visualization: SP and THE. Writing-original draft: SP and THE. Writing-review & editing: SP, JS, THE, and YHK.

References

1. Kuerten AB, Mota MB, Segaert K. Developmental dyslexia: a condensed review of literature. Ilha Desterro 2020;72:249–70.
2. Payne T, Turner E. Dyslexia: a parents' and teachers' guide. Vol. 3 Philadelphia: Multilingual Matters; 1999.
3. Berlin R. Über dyslexie. In : Arnold DDB, Kostlin O, Teuffel J, eds. Medicinisches Correspondenzblatt des Württembergischen Ärztlichen Landesvereins. Vol. 53 Stuttgart: E. schweizerbart’sche; 1883. p. 209–10.
4. Kussmaul A. Diseases of the nervous system and disturbances of speech. In : von Ziemssen H, ed. Cyclopaedia of the practice of medicine London: Sampson Row, Maston, Searle & Rivingston; 1877. p. 770–8.
5. Kirby P. Dyslexia debated, then and now: a historical perspective on the dyslexia debate. Oxf Rev Educ 2020;46:472–86.
6. Mahmoodi-Shahrebabaki M. Dyslexia: past, present, and future. Paper presented at 6th Annual MTSU Literacy Research Conference; 2018 Feb 21; Murfreesboro, TN. Available from: https://ssrn.com/abstract=3118677.
7. American Psychiatric Association. Neurodevelopmental disorders: DSM-5® selections Washington DC: American Psychiatric Pub; 2015.
8. Yell ML, Shriner JG, Katsiyannis A. Individuals with disabilities education improvement act of 2004 and IDEA regulations of 2006: implications for educators, administrators, and teacher trainers. Focus Except Child 2006;39:1–24.
9. Sanfilippo J, Ness M, Petscher Y, Rappaport L, Zuckerman B, Gaab N. Reintroducing dyslexia: early identification and implications for pediatric practice. Pediatrics 2020;146e20193046.
10. Schatschneider C, Torgesen JK. Using our current understanding of dyslexia to support early identification and intervention. J Child Neurol 2004;19:759–65.
11. Fletcher JM, Francis DJ, Foorman BR, Schatschneider C. Early detection of dyslexia risk: development of brief, teacher-administered screens. Learn Disabil Q 2021;44:145–57.
12. Colenbrander D, Ricketts J, Breadmore HL. Early identification of dyslexia: understanding the issues. Lang Speech Hear Serv Sch 2018;49:817–28.
13. Nilsson Benfatto M, Oqvist Seimyr G, Ygge J, Pansell T, Rydberg A, Jacobson C. Screening for dyslexia using eye tracking during reading. PLoS One 2016;11e0165508.
14. Vajs I, Papic T, Kovic V, Savic AM, Jankovic MM. Accessible dyslexia detection with real-time reading feedback through robust interpretable eye-tracking features. Brain Sci 2023;13:405.
15. Nerusil B, Polec J, Skunda J, Kacur J. Eye tracking based dyslexia detection using a holistic approach. Sci Rep 2021;11:15687.
16. Smyrnakis I, Andreadakis V, Selimis V, Kalaitzakis M, Bachourou T, Kaloutsakis G, et al. RADAR: a novel fast-screening method for reading difficulties with special focus on dyslexia. PLoS One 2017;12e0182597.
17. Jothi Prabha A, Bhargavi R. Predictive model for dyslexia from fixations and saccadic eye movement events. Comput Methods Programs Biomed 2020;195:105538.
18. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 1994;6:284–90.
19. Handler SM, Fierson WM, ; Section on Ophthalmology; Council on Children with Disabilities; American Academy of Ophthalmology, ; American Association for Pediatric Ophthalmology and Strabismus, et al. Learning disabilities, dyslexia, and vision. Pediatrics 2011;127:e818–56.
20. McConkie GW, Zola D. Some characteristics of readers' eye movements. In: von Euler C, Lundberg I, Lennerstrand G, editors. Brain and reading: structural and functional anomalies in developmental dyslexia with special reference to hemispheric interactions, memory functions, linguistic processes, and visual analysis in reading. Proceedings of the 7th International Rodin Remediation Conference at the Wenner-Gren Center, Stockholm and Uppsala University, June 19-22, 1988. New York: Stockton Press; 1989. p. 369-81.
21. Rayner K. Eye movements in reading and information processing. Psychol Bull 1978;85:618–60.
22. Olitsky SE, Nelson LB. Reading disorders in children. Pediatr Clin North Am 2003;50:213–24.
23. Tseng PH, Cameron IG, Pari G, Reynolds JN, Munoz DP, Itti L. High-throughput classification of clinical populations from natural viewing eye movements. J Neurol 2013;260:275–84.
24. Benson PJ, Beedie SA, Shephard E, Giegling I, Rujescu D, St Clair D. Simple viewing tests can detect eye movement abnormalities that distinguish schizophrenia cases from controls with exceptional accuracy. Biol Psychiatry 2012;72:716–24.
25. Wang S, Jiang M, Duchesne XM, Laugeson EA, Kennedy DP, Adolphs R, et al. Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron 2015;88:604–16.
26. Rayner K, Pollatsek A, Ashby J, Clifton C Jr. Psychology of reading 2nd edth ed. New York: Psychology Press; 2012.
27. Vagge A, Cavanna M, Traverso CE, Iester M. Evaluation of ocular movements in patients with dyslexia. Ann Dyslexia 2015;65:24–32.
28. Starr MS, Rayner K. Eye movements during reading: some current controversies. Trends Cogn Sci 2001;5:156–163.
29. Willcutt EG, Pennington BF. Psychiatric comorbidity in children and adolescents with reading disability. J Child Psychol Psychiatry 2000;41:1039–48.
30. Pennington BF, Bishop DV. Relations among speech, language, and reading disorders. Annu Rev Psychol 2009;60:283–306.

Article information Continued

Fig. 1.

The eye-tracking system was first calibrated by having the user look at a total of five points in the center and four corners on the tablet computer screen.

Fig. 2.

Nine words were presented on the screen, and the generated gaze data and word position values were used to ensure proper calibration.

Fig. 3.

After calibration and verification, the reading materials (including two articles) were presented to each participant via the tablet computer screen, and eye-tracking was performed.

Table 1.

Grade distribution of the 200 children between 8 and 13 years of age attending an elementary school

Grade Total (n=200) Male Female
1 15 (7.5) 7 (3.5) 8 (4.0)
2 20 (10.0) 11 (5.5) 9 (4.5)
3 30 (15.0) 10 (5.0) 20 (10.0)
4 36 (18.0) 20 (10.0) 16 (8.0)
5 51 (25.5) 26 (13.0) 25 (12.5)
6 48 (24.0) 28 (14.0) 20 (10.0)

Values are presented as number (%).

Table 2.

The reliability of eye movement measurements using an eye-tracking system in a tablet computer-based dyslexia screening application

Variable First Second ICC
Reading speed by gaze (words/min) 160.59±51.58 154.99±52.17 0.76
Gaze fixation mean time (ms) 378.09±81.14 373.27±79.38 0.82
Gaze fixation frequency (fixation count/100 words) 80.19±30.41 84.40±33.44 0.83
Saccadic length (letters) 6.31±1.44 6.42±1.44 0.72
Regression ratio (%) 9.65±5.45 9.59±5.73 0.75

Values are presented as mean±standard deviation.

ICC, intraclass correlation coefficient.