Translate this page into:
A comparative study between an indigenously developed psychometric test battery “pSuMEDhA” and “CogScreen-AE”
*Corresponding author: Anitha Thambidurai, Indian Air Force, India. durai.anitha@gmail.com
-
Received: ,
Accepted: ,
How to cite this article: Thambidurai A, Sharma V, Sowgandhi C, Biswal P. A comparative study between an indigenously developed psychometric test battery “pSuMEDhA” and “CogScreen-AE.” Indian J Aerosp Med 2024;68:2-9. doi: 10.25259/IJASM_2_2024
Abstract
Objectives:
In today’s combat environment, a pilot’s duty necessitates a higher level of precision, agility, speed, attention, memory, situational awareness, risk assessment, and planning. As a result, the ability to evaluate critical aspects of cognition is essential. The present study was conducted with an aim to compare performance measures in two different cognitive test batteries, namely, “pSuMEDhA” and CogScreen – Aeromedical Edition (CogScreen AE).
Material and Methods:
Fifty selected healthy adult volunteers were made to undergo two cognitive test batteries; CogScreen AE and pSuMEDhA. The comparable variables pertaining to speed, accuracy, and process measures were identified between the two tests and analyzed. The variables included speed measures pertaining to the dual-task test (DTT - PSU), digit symbol substitution test (DSST), and Mackworth clock test of pSuMEDhA with the dual task test (DTT-COG), symbol digit coding (SDC), and continuous performance test of (CPT) CogScreen AE. Similarly, the accuracy measures of aircraft orientation test (AOT) and DSST of pSuMEDhA with matching to sample and SDC of CogScreen AE process measures of card sorting test of pSuMEDhA with shifting attention (MTS) test of CogScreen AE, respectively, were compared.
Results:
The mean age of the sample was 31.78 ± 0.74 years with gender distribution of 64% males and 36% females. Using Bland–Altman plot analysis, the speed and accuracy measures of DSST and SDC were found to be congruent, implying both the test batteries, that is, pSuMEDhA and Cogscreen AE, would assess working memory optimally. Other comparable tests showed insignificant correlations. Further, the exclusive variables of each test battery demonstrated uniqueness in the design of the tests, fulfilling the purpose for which they were developed.
Conclusion:
Certain evaluation tasks by pSuMEDhA and CogScreen AE assess the same cognitive capacity, although they are administered differently. Both cognitive test batteries, however, accurately assess individual variances of cognition. One of pSuMEDhA’s tasks is to evaluate an aviator’s threat perception and appreciation through threat index assessment. On the other hand the purpose of CogScreen AE is to screen pilots for neurocognitive impairment and Taylor’s aviation factor score.
Keywords
Cognitive test battery
pSuMEDhA
CogScreen aeromedical edition
Cognitive performance
INTRODUCTION
A pilot’s role in modern day combat duties requires a higher degree of declarative and procedural knowledge. It becomes essential to analyze the learnt information (declarative) at crucial flight scenarios for their judicial application (procedural).[1] The diagnostic and statistical manual of mental disorders-5 defines the domains of cognition as complex attention, social cognition, learning and memory, language, perceptual motor function, and executive function.[2] The pilot must possess and train to acquire better accuracy, efficiency, speed, attention, memory, situational awareness, risk assessment, and planning. Thus, it is crucial to be able to assess important aspects of cognition efficiently and precisely.[3]
Various tests of cognitive competency have been identified as crucial to piloting ability. The physical demands of flying a modern aircraft have decreased, but the cognitive demands have increased, indicating that age-related changes in cognitive function are becoming increasingly important. Pilots are affected by a variety of psychomotor performance variables and conditions in an operational flying environment.[3] Reaction time, divided attention, selective attention, alertness, vigilance, and fatigue are the factors to be considered.
An individual’s cognition can be assessed by standard cognitive tests, which can be paper-pencil type or computer-assisted psychometric tests. Computerized cognitive tests extensively used in the world are MindStreams, cognitive drug research battery, CANTAB, ImPACT, ANAM, and CogStat. A few cognitive test batteries, such as CogScreenAeromedical Edition (CogScreen-AE), multidimensional aptitude battery, MICROPAT, WOMBAT, PABT, and MicroCog, are used globally by Air Forces for evaluating the pilot’s cognitive ability.
In Indian Air Force (IAF), the major test batteries used for assessing the aircrew’s cognitive abilities and ruling out dysfunctions with Raven’s Progressive Matrices, Bhatia Battery of Performance Test of Intelligence, Wechsler Memory Scale, and CogScreen-AE. Institute of Aerospace Medicine (IAM), – Psychomotor Evaluation Designed for Aviators is an indigenously designed psychomotor cognitive test battery at and has potential applications in aerospace environment. However, “pSuMEDhA” is yet to be validated. The present study aimed to compare the selected tests of pSuMEDhA with CogScreen AE the same cognitive domain with corresponding brain areas and correlate them on comparable variables.
MATERIAL AND METHODS
Study sample
A total of 50 non-aircrew healthy adult volunteers (Male – 32, Female – 18, mean age of 31.78 ± 0.74 years) participated in the study. The selection of the participants was based on the following inclusion criteria: healthy adult volunteers, age group between 20 and 50 years, educational qualification of minimum 12th standard, functional knowledge of computer operations, individuals with good sleep hygiene with optimum sleep hours, and no dependence on stimulants such as coffee, tea, and tobacco. Participants with any comorbidity or any associated illness and on treatment for the same, previous history of head injury, known cases of neurological or psychiatric illness, and on treatment for the same were excluded from the study. To prevent any confounding effect on the performance of the test, the prerequisites followed were as follows: (a) Abstinence from alcohol for at least 24 hours before the test (b) Taking any stimulants such as coffee and tea was denied (c) Any kind of medication for any ailments, and (d) minimum 8 hours of restful sleep.
Materials
The details of the cognitive test batteries used in this study are as follows:-
Psychomotor evaluation designed for aviators – “pSuMEDhA”
It is a cognitive test battery developed at IAM indigenously and uses standardized software for psychomotor cognitive evaluation of the aircrew. “pSuMEDhA” is designed as a self-administered test battery comprising of eight separate subsets to assess various domains of cognitive functions such as response inhibition, selective attention, sustained attention, vigilance, reaction time, mental speed tracking, coordination, visual search and working memory, deductive reasoning, threat assessment response, and attribute identification. The eight subset tests in pSuMEDhA[4] are as follows: simple reaction test (SRT), Mackworth clock test (MCT), dual-task test (DTT-PSU), Stroop test (ST), digit symbol substitution test (DSST), aircraft orientation test (AOT), threat perception and estimation test (TPET), and card sorting test (CST). pSuMEDhA test battery software suite has been tested for use on any Microsoft Windows 7.0 to Windows 10.0 desktops or laptops.
CogScreen -AE
This cognitive screening tool is computer-administered and scored to quickly identify deficits or changes in attention, short- and immediate-term memory, visual perception, simultaneous information processing, sequencing, logical analysis, calculation, reaction time, and executive functioning. CogScreen is a battery of thirteen tests that majorly fall under the cognitive faculties of speed, accuracy, process, and throughput. In addition, Taylor’s aviation factor scores of CogScreen AE are attribute identification, motor coordination, visual association memory, speed/working memory, and tracking accuracy.[5]
The 13 subsets of CogScreen AE are as follows; backward digit span, Math, visual sequencing comparison (VSC), symbol digit coding (SDC) – immediate recall and delayed recall (SDC), matching to sample (MTS), Manikin (MAN), divided attention test (DAT), VSC + DAT, auditory sequence comparison, Path Finder – Numeric sequencing and Letter sequencing (PF), shifting attention test (SAT), dual-task tracking (DTT-COG), and continuous performance test (CPT). The test station components are Accutouch LCD Touch screen monitor, Surface Pro or Mini PC Computer, Windows XP, 7, 10 Operating System, USB Gaming Mouse and USB Keyboard, Gel wrist pad and Mouse pad, Headphones and Stylus.
Protocol
Informed consent was obtained from the participants after explaining the procedure in detail. The study was approved by the Institute Ethics Committee. On day 1 of the study, the participants were made to undergo CogScreen AE at the Department of Aviation Psychology, IAM. CogScreen AE being a sophisticated computerized test, specific instructions regarding the equipment components, their placement and operating procedure were given. Before the commencement of each test, a practice session was conducted by the test battery to ensure that the participant had understood the instructions of each specific test properly.
After completion of the CogScreen AE test, participants were made to undergo the pSuMEDhA test after a gap of 24 h (day 2). At the beginning of the pSuMEDhA test battery, participants have the option to undergo practice session, which has all the tests except ST and CST for a total duration of 5 min. This session was skipped to obtain robust baseline data of the individuals.
Comparable variables
The tests of pSuMEDhA with CogScreen AE assessing the same cognitive domain with corresponding brain areas were compared. The outcome variables of these comparable tests are speed measures, accuracy measures, and process measures, and the details are enlisted in Table 1.
| Cognitive ability | pSuMEDhA | CogScreen AE |
|---|---|---|
| Speed measures | ||
| Psychomotor function, tracking and pursuit | Dual task test (DTT-PSU) and lag error | Dual task tracking (DTT-COG) and absolute error in dual tracking test (DTTAABS) |
| Sustained attention and psychomotor function | Mackworth clock test (MCT) | Continuous performance test (CPT) |
| Visual working memory, visual search and deductive reasoning | Digit symbol substitution test (DSST) | Symbol digit coding test (SDC) |
| Accuracy measures | ||
| Perceptual motor function and spatial orientation | Aircraft orientation test (AOT) | Matching to sample test (MTS) |
| Visual working memory, visual search and deductive reasoning | Digit symbol substitution test (DSST) | Symbol digit coding test (SDC) |
| Process measures | ||
| Conceptualization, executive function and shifting attention | Card sorting test (CST) | Shifting attention test (SAT) |
Statistical analysis
The collected data were analyzed under five categories: (a) Outcomes of tests of pSuMEDhA and CogScreen AE, which assessed the same cognitive domains with corresponding brain areas were analyzed by the following methods: (i) Descriptive analysis, (ii) Bland–Altman plot Analysis, (b) correlation for validity of the comparable variables of similar tests of pSuMEDhA and CogScreen AE, (c) descriptive analysis of variables which are exclusive of “pSuMEDhA” cognitive test battery, (d) descriptive analysis of variables which are exclusive of CogScreen – AE, and (e) Assessing individual’s pattern of response to a simulated digital cognitive test – CPT.
RESULTS
The descriptive analysis of the comparable outcome variables is listed in Table 2. The analysis revealed that DSST and SDC tests had negligible differences in reaction time (0.25 s) and accuracy (0.6%) as per Bland–Altman plot [Figures 1 and 2]. Pearson’s Correlation also revealed a significant correlation (r = 0.6, P = 0.000) between the two. This infers that DSST and SDC assess the same cognitive ability and deliver near identical outcomes. Similarly, the difference in reaction time (0.16 s) and lag error (0.39 pixels) of DTT-PSU and DTTCOG was minimal, which may be considered insignificant in real world terms [Figures 3 and 4]. This makes the DTT of both tests useful for assessing an individual’s tracking and pursuit ability.
| Speed measures | ||
| Test name | Median (IQR) in ms | Std dev |
| DTT - PSU | 577.5 (101.5) | 63.1 |
| DTT - COG | 706 (293.7) | 262.8 |
| MCT | 572 (91.5) | 36.3 |
| CPT | 491 (64.7) | 36.4 |
| DSST | 1651 (326.2) | 210.1 |
| SDC | 1449.5 (532.7) | 352.4 |
| Accuracy measures | ||
| Test name | Mean in % | Std dev |
| AOT | 49.6 | 21.3 |
| MTS | 88.9 | 15.6 |
| DSST | 98.3 | 1.7 |
| SDC | 99.3 | 1.1 |
| Process measures | ||
| Test name | Mean | Std dev |
| CST | 0.0400 | 0.19795 |
| SAT | 2.1800 | 1.94506 |
IQR: interquartile range, Std dev: Standard deviation, DTT: Dual task test, MCT: Mackworth clock test, CPT: Continuous performance test, DSST: Digit symbol substitution test, SDC: Symbol digit coding test, AOT: Aircraft orientation test, MTS: Matching to sample test, SDC: Symbol digit coding test, CST: Card sorting test, SAT: Shifting attention test
- Bland–Altman plot for Reaction Time (RT) in digit symbol substitution test versus symbol digit coding. SD: Standard deviation.
- Bland–Altman plot for accuracy of digit symbol substitution test versus symbol digit coding.
- Bland–Altman plot for RT (reaction time) in DTT-PSU versus DTT-COG. SD: Standard deviation.
- Bland Altman lag error DTT-PSU versus DTT-COG.
Speed measures pertaining to MCT and CPT showed marginal differences in the Bland–Altman plot [Figure 5] and were found to be insignificantly correlated. The accuracy scores of AOT and MTS, along with process measures of CST and SAT, also yielded insignificant test outcomes. The mean reaction time of SRT was 448.74 ± 40.12 ms, and the mean risk index of TPET was 85.12 ± 29.19%. Table 3 illustrates the descriptive analysis of exclusive variables of CogScreen AE. The median reaction time of each of the 2-minute blocks of CPT showed the trends in the reaction time with the test duration [Figure 6].
- Bland–Altman plot for Reaction Time (RT) in Mackworth clock test versus continuous performance test.
| Speed Measures | ||
| Test name | Median (IQR) in ms | Std dev |
| MATH | 28.62 (0-56) | 9.155 |
| VSC | 2.00 (1-3) | 0.468 |
| MANIKIN | 2.00 (1-4) | 0.565 |
| DAT | 1.90 (1-4) | 0.513 |
| ASC | 0.67 (0-1) | 0.171 |
| PFC | 1.26 (1-3) | 0.428 |
| SAT | 1.03 (1-2) | 0.323 |
| Accuracy Measures | ||
| Test name | Mean (%) | Std dev |
| MATH | 75.20 | 21.59 |
| VSC | 99.10 | 1.94 |
| MANIKIN | 82.16 | 19.19 |
| ASC | 89.80 | 9.14 |
| PFC | 97.58 | 5.26 |
| SAT | 61.37 | 14.48 |
| BDS | 88.16 | 17.98 |
IQR: Interquartile range, Std dev: Standard deviation, VSC: Visual sequencing comparison, DAT: Divided attention test, SAT: Shifting attention test
- Trend chart of Reaction Time (RT) in continuous performance test. CPT: Continuous performance test, RT: Reaction Time.
DISCUSSION
Aircrews are a unique subset of professionals who operate in an environment intolerant to human errors and, hence, are expected to have superior cognitive abilities to meet the challenging demands of flying. In the present study, selected tests having the same cognitive domain with corresponding brain areas of two cognitive test batteries (pSuMEDhA and CogScreen AE) are compared to get an insight into the cognitive abilities assessed by the specific tests and to establish their psychometric properties as well as correlation. The analysis of the test outputs of these tests has been discussed below.
Dual-task test (pSuMEDhA) versus dual-task tracking (CogScreen AE)
The dual-task method is the most prevalent way to measure divided attention. Two tasks are being measured at the same time. Researchers have employed dual-task methods to understand memory functioning, such as doing a speeded visual tracing test while remembering a series of numbers.[6] In the context of aviation, flying is a closed loop tracking task. A pilot needs to have better attention and tracking ability to sense the changes during flying and deliver correct input for a safe flight. The tracking/tracing ability of an aircrew is important during target tracking in a mission and approach landing. Taylor et al. have stated that dual task and divided attention scores of CogScreen AE predicted training, compliance with procedure, and crew resource management.[7]
The sample’s median reaction time of DTT of CogScreen AE was found to be more than the DTT of pSuMEDhA by 159 ms, which is considered negligible in practical applications. This difference could be attributed to the additional cognitive functions assessed by DTT of CogScreen AE, such as working memory. The standard deviation (SD) of Lag error of pSuMEDhA was much smaller than the SD of Lag error of CogScreen AE. This could be due to the simpler and direct assessment of tracking in pSuMEDhA, where the individual uses the mouse to track the target. Whereas in DTT of CogScreen AE, the individual utilizes the two arrow mark keys on the keyboard to track the target. The complex procedure of tracking in CogScreen AE would have resulted in a larger SD in comparison with that of pSuMEDhA. On analysis, DTT of pSuMEDhA did not indicate a significant correlation with that of CogScreen AE. This might be possibly due to the measurement of cognitive ability like working memory by DTT of CogScreen AE, but in pSuMEDhA, the test has been designed to exclusively measure tracking and psychomotor ability.
Both tests are unique in their own ways and hold their own strengths and weaknesses. When required to measure an individual’s tracking and psychomotor ability alone, DTT of pSuMEDhA is a better choice for researchers. However, when an individual’s higher ability for procedure compliance, multitasking, and management of complex tasks need to be assessed; DTT of CogScreen AE may be preferred.
MCT (pSuMEDhA) versus CPT (CogScreen AE)
In the aviation environment, vigilance is an important aspect that requires an individual to recognize infrequent stimuli and distinguish them from “noise” or distracter stimuli over long periods of flight time. The aircrew is frequently required to make or inhibit a reaction to the target in such tasks. Human functional neuroimaging studies have linked the execution of vigilance and sustained attention operations, including those guided by internal representations, to activation of frontal (dorsolateral prefrontal and anterior cingulate) and parietal cortical areas, mainly in the right hemisphere.[8] Other aviation stressors, such as hypoxia, mental overload, and fatigue, have been shown to affect sustained attention.[9] Hence, assessment of sustained attention plays a key part during aircrew selection and evaluation. The MCT in pSuMEDhA and CPT in CogScreen AE have been designed to measure this cognitive domain.
In the present study, the median reaction time in MCT was found to be 572 ms with an interquartile range (IQR) of 91.5 ms, and CPT was 491 ms with an IQR of 64.75 ms, and the difference was statistically significant. The probable reason could be the difference in the test administration method. The test duration in both the test batteries is not more than 15 min; however, the number of presentations in MCT is 15 and in CPT is 54. Hence, it may be ascertained that the method of administration may have possibly influenced the output. Lichstein et al. found that the psychological resources were more strongly taxed during the second half an hour of the MCT exam. When using the MCT to differentiate between groups, the second half hour might be more sensitive.[10]
The graphical representation of change in reaction time in CPT test duration when measured in blocks of 2 min showed a consistent increase in reaction time as the test progressed. The trend chart computed confirmed that sustained monotonous activity caused increased reaction time. In the last block of the test, participants showed a mild recovery in reaction time. This could be perhaps attributed to the awareness of test duration.
DSST (pSuMEDhA) versus SDC (CogScreen AE)
The speed of information processing, being a key component in higher piloting abilities, was reported in a study by Taylor et al.[11] Yesavage et al. confirmed this finding, indicating that aviators with the greatest processing speed scores had the least rate of deterioration in flight simulator performance. [12] Similarly, the speed of processing new information, which could be in the form of a change in the surroundings, would be the next logical step in a closed-loop flying task (after vigilance). The sooner this shift is detected and processed, the sooner corrective action can be taken to prevent errors. This suggests that processing speed is an important factor to consider while performing a flying activity.[12]
In the present study, a statistically significant positive correlation (r = 0.6, P = 0.000) was established in the sample’s median reaction time between DSST and SDC tests. To perform well in DSST, it necessitates good motor speed, attention, and visuo-perceptual capabilities, such as scanning and the capacity to write or draw (i.e., basic manual dexterity). Associative learning may also have an impact on performance. Fronto-parietal activation linked to DSST results has been regarded as representing both onboard working memory processing and low-level visual search. Due to these characteristics, the DSST is used in so many different applications; it provides unique chances for comparison.[13]
AOT (pSuMEDhA) versus MTS (CogScreen AE)
AOT is adapted from Arthur Benton’s Judgment of Line Orientation Test and assesses visual working memory and visuo-perceptual speed, visuo-construction, and spatial processing, while MTS assess visual-perceptual speed, spatial processing, and visual working memory. Verde et al. found that pilots outperformed non-pilots in terms of accuracy in directional judgments.[14] The factors contributing to better directional judgment by the pilots could be greater flight experience of pilots. This experience could have increased the pilot’s capacity to complete spatial tasks by allowing them to become more comfortable with the three-dimensional navigation system. Military pilots are immune to negative spatial bias and can employ a different technique, although one that is more difficult in terms of cognitive load and comes at a cost in terms of time when compared to the general population.
Similarly, in the present study, the accuracy measure was observed to be 49.6% on AOT and 88.9% on MTS. This could be due to the fact that the participants in the study were not pilots. AOT is a performance test, where the participant is required to respond by manipulating the given test material. MTS is a multiple-objective test where the participant has to recall, interpret, and solve the task presented to them. This could be the probable reason why the participants could have felt MTS was easier to solve than AOT. This implies that MTS can be used for general population assessment, whereas AOT is more suitable for aircrew assessment. No significant correlation could be established between these accuracy measures. The possible reason behind this could be the additional abilities like visuo-spatial orientation assessed by AOT along with working memory, whereas MTS exclusively measures working memory.
Failure to maintain set in CST versus SAT discovery failure to maintain set (SATDIFAI) in SAT
Higher order skills in the following domains are referred to as executive functioning: abstract reasoning, problem-solving, attention, mental flexibility, initiation, planning, and inhibition. This group of skills is most frequently connected to the brain’s frontal lobe area.[15] Multiple executive functions, in combination with other cognitive talents, are linked to measures of flying, navigating, communicating and can even predict performance. On Pearson’s correlation, both the tests were found to be weakly correlated (r = 0.246 and P = 0.08). The process measure of CST and SAT assesses the same cognitive ability and has shown varied outcomes due to the different methods of administration. This implies that CST or SAT may be used in aircrew selection and evaluation as both yields the required results.
Future scope and limitations
Taylor et al.’s factors of CogScreen AE were derived by correlating the scores of CogScreen AE with the flight simulator performance and pilot age.[7] Similar cumulative scores can be derived from individual tests of pSuMEDhA in future by correlating the scores with the flight simulator performance of aviators. In addition, the logistic regression probability value coefficient of CogScreen-AE estimates the probability of brain dysfunction from a logistic regression algorithm inbuilt in the CogScreen AE equipment. On the other hand, pSuMEDhA is designed to assess the aviator’s cognitive abilities by psychometric testing under various aviation stressors robustly.
CONCLUSION
In the present study, the speed and accuracy measures of DSST and SDC were found to be congruent, implying that both the test batteries, that is, IAM pSuMEDhA and CogScreen AE, assess working memory optimally. The outcome variables of the other comparable tests had marginal differences with insignificant outcomes. It may be inferred that the insignificant results may be due to the difference in the methods of administration in each cognitive test battery. This implies that the comparable cognitive tests of both the test batteries measure the same parameter of specific cognitive abilities and corresponding brain regions, which plays a vital role in an aircrew’s training and performance. Both pSuMEDhA and CogScreen AE measure individual differences with respect to cognitive abilities. However, pSuMEDhA needs to be validated so that in future it can be effectively utilized in the evaluation of aircrew in IAF.
Acknowledgments
The authors would like to express their sincere gratitude to Dr. A Agarwal, Indian Air Force, for his support and guidance during the conceptualization of this research and to Dr. NK Tripathy, Dr. MS Indian Air Force for his valuable feedback and constructive remarks, which contributed to enhancing the quality of this research.
Ethical approval
The research/study complied with the Helsinki Declaration of 1964.
Declaration of patient consent
The authors certify that they have obtained all appropriate patient consent.
Conflicts of interest
Vipin Sharma is on the editorial board of the journal.
Use of artificial intelligence (AI)-assisted technology for manuscript preparation
The authors confirm that there was no use of artificial intelligence (AI)-assisted technology for assisting in the writing or editing of the manuscript and no images were manipulated using AI.
Financial support and sponsorship
Nil.
References
- Cognitive science and aviation training: Foundations for effective learning and operational application In: SAE Technical paper series. SAE international world aviation congress and exposition. Montreal, Canada, USA: SAE International; 2003. p. :4.
- [CrossRef] [Google Scholar]
- Neurocognitive disorders In: Diagnostic and statistical manual of mental disorders (5th ed). Sec. 2. Washington, D.C: American Psychiatric Publishing; 2013. p. :593-5.
- [Google Scholar]
- Computer-assisted cognitive function assessment of pilots. ADF Health. 2001;2:29-36.
- [Google Scholar]
- CogScreen AE: Professional manual Florida: Psychological Assessment Resources; 1995.
- [Google Scholar]
- Divided attention In: Encyclopedia of clinical neuropsychology. New York: Springer Science, Business Media; 2011. p. :879-80.
- [CrossRef] [Google Scholar]
- Relationship of CogScreen-AE to flight simulator performance and pilot age. Aviat Space Environ Med. 2000;71:373-80.
- [Google Scholar]
- Sustained attention In: Encyclopedia of clinical neuropsychology. New York: Springer Science, Business Media; 2011. p. :2440-3.
- [CrossRef] [Google Scholar]
- The sleep, subjective fatigue, and sustained attention of commercial airline pilots during an international pattern. Chronobiol Int. 2006;23:1357-62.
- [CrossRef] [PubMed] [Google Scholar]
- The Mackworth clock test: A computerized version. J Psychol. 2000;134:153-61.
- [CrossRef] [PubMed] [Google Scholar]
- The effects of information load and speech rate on younger and older aircraft pilots' ability to execute simulated air-traffic controller instructions. J Gerontol. 1994;49:191-200.
- [CrossRef] [PubMed] [Google Scholar]
- Initial cognitive performance predicts longitudinal aviator performance. J Gerontol B Psychol Sci Soc Sci. 2011;66:444-53.
- [CrossRef] [PubMed] [Google Scholar]
- Digit symbol substitution test: The case for sensitivity over specificity in neuropsychological testing. J Clin Psychopharmacol. 2018;38:513-9.
- [CrossRef] [PubMed] [Google Scholar]
- Spatial orientation and directional judgments in pilots VS. nonpilots. Aerosp Med Hum Perform. 2018;89:857-62.
- [CrossRef] [PubMed] [Google Scholar]
- Executive functioning In: Encyclopedia of clinical neuropsychology. New York: Springer Science, Business Media; 2011. p. :991.
- [CrossRef] [Google Scholar]
