Translate this page into:
Effect of different types of irrelevant speech on performance in Simulated Flight Task and Heart Rate Variability
Abstract
Task irrelevant speech has been found to produce changes in task performance by an individual. However, the effect of different types of irrelevant speech remains a less investigated area. In this study, the effect of three different types of irrelevant speech have been analysed in terms of decrement in Flight Oriented Task Performance (FOPT) and their effect on indices of Heart Rate Variability (IIRV). In this experiment, 30 non-aircrew subjects were presented serially with three different types of irrelevant speech i.e. Non-intelligible speech, intelligible speech and conversation while they executed a Flight Oriented Performance Task (FOPT). The decrements in task performance and the ECG data for IIRV indices were obtained under different test conditions. There was a significant overall effect of irrelevant speech on FOPT performance (F=9.00, p=0.0293). Intelligible speech led to greater errors in task performance as compared to conversation (p=0.0027). Out of all the IIRV indices, irrelevant speech produced significant effect on low frequency (LF) bands (F=5.561, p=(M)014). Non-intelligible speech produced significant differences as compared to the baseline in LF power (p= 0.004). 11 was also observed that there were significant differences between non-intelligible speech and conversation in the LF power (p= 0.0209). Irrelevant speech produces a significant effect on task performance and physiologic measure of Heart Rate Variability. Intelligible irrelevant speech produces the highest decrement in performance. The physiological cost of non intelligible irrelevant speech is high even when the effect is not reflected in their task performance.
Keywords
Irrelevant speech effect
Flight Oriented Performance Task (FOPT)
Heart Rate Variability (IIRV)
Introduction
In aviation, where efficient management of flow of information to the brain is required, attentional selectivity is of paramount importance. But such selectivity is not merely the voluntary inclusion of relevant information and the complete neglect of irrelevant information. The irrelevant speech concept has shown that unattended auditory information is both registered and organised by the brain. This obligatory process of organising the irrelevant speech affects activities that require continuous monitoring to derive the information on the trends of rapidly occurring changes. Aviation related activities in air or on the ground require the continuous monitoring of displays to derive the current status information vis-a-vis the immediate past and thus determine the error and take corrective action. The impact of such pre-attentive processing is greater in the auditory modality because we cannot easily prevent the sensory information entering the brain i.e. we cannot easily shut or re-direct the ears to avoid sound in the same way as we can shut or move the eyes to avoid registering the reflection of light from a given object [1, 2, 3].
The effect of irrelevant speech on task performance during different task conditions or phases of flight has been studied [4,5,6,7]. However, this irrelevant speech has been variously reported to increase or decrease performance based on the phase or type of task in which it is given[8,9,10]. Moreover, most of the literature on the irrelevant speech effect focuses on the disruption of serial recall tasks (repeating numbers or alphabets in serial order) [1,3,11,12,13,14, 15]. Therefore, research on applicability of the irrelevant speech effect on a more naturalistic task, that does not strictly comprise of serial recall, needs further scientific scrutiny.
There is scarce literature on the effect of different types of irrelevant speech on the workload of an operator. To study the effect of different types of irrelevant speech it is essential to keep the primary task uniform across test conditions and subjects. Intelligible speech has been the most commonly studied irrelevant speech [2,3,14,15]. Hughes and Jones found that the effect of irrelevant speech appears to be independent of its intensity [3]. The effect of intelligible words has been found to increase the effect of irrelevant speech only by 12% as compared to non-intelligible words [14]. The “Changing State” hypothesis states that any irrelevant speech yields “order information” which is liable to corrupt any serial recall task being performed simultaneously [3]. Irrelevant conversation has been hypothesised to act in two ways. Firstly it would ensure obligatory processing of the irrelevant speech. Secondly the process of articulation itself is known to add to the irrelevant speech effect [15]. Thus it was hypothesised that the mental workload of an operator will be modified in the presence of task irrelevant speech. This study aimed at differentiating between three different types of irrelevant speech (Intelligible, Non Intelligible, Conversation) in terms of the decrement in performance and the physiologic cost as reflected by HRV.
Material and Methods
In this study the mental workload of an operator in presence of different types of irrelevant speech while performing a uniform Flight Oriented Performance Task on a PC based simulator has been evaluated.
A PC based simulator developed by Axxonnet Solutions Ltd has been used as the primary task. The user interface presents an instrument panel with the basic flight and navigational display. The task is to maintain the target heading and altitude for the duration of the trial. The roll and pitch of the tracking task are controlled by a joystick. The behaviour of the four flight instruments is in turn directed by these two variables. It should be noted that this is not an accurate flight simulation. The relationships linking flight dynamics have been simplified to enable flight-inexperienced participants to take part, as well as to allow greater and more flexible experimental control. In view of these limitations, it is called a Flight Oriented Performance Task (FOPT) rather than a flight simulator [16]. The deviation from the ideal flight conditions leads to an error score, thus a larger error score depicts poorer performance.
For acquiring heart rate data, physiologic monitoring system (BESS-AMS 2000) has been used. The leads were attached to the transducer of the equipment and the signals were amplified, band passed (1-40 Hz), digitized with a sampling frequency of 2048 Hz with 12 bit resolution AD (analog-digital) card and were stored on the harddisk of the PC for offline analysis [17,18],
For further analysis of HRV data, the Kubios HRV, Heart Rate Variability (HRV) analysis software developed by the Biosignal Analysis and Medical Imaging Group (BSAMIG) at the Department of Physics, University of Kuopio, Kuopio, Finland has been used [19],
Protocol
Adult male non aircrew volunteer subjects (n=30) with minimum educational qualification of MBBS participated in this study. Prior to conduct of the study, basic medical examination of the subjects were performed to ensure adequate visual and hearing standards that are important in the study.
Thereafter, the subjects were demonstrated the Flight Oriented Performance Task (FOPT) and provided with practice sessions at “flying” it till they achieved satisfactory levels. The subjects were instructed to observe total abstinence from alcohol for 12 hours and avoid smoking for 4 hrs before the experiment as these could potentially contaminate Heart Rate Variability (HRV) readings. The study was designed to be in the form of a randomised control experiment.
On reporting for the experiment, the subject was instrumented for heart rate recordings. The ECG electrodes were placed in standard CM-5 configuration after adequate skin preparation. Each test condition was for a duration of 5 min during which the subject “flew” the FOPT on an IBM laptop with the joystick in the dominant hand and was given a fixed profile on the FOPT. The subjects were instructed to concentrate on the primary task wherein he had to maintain a given heading, airspeed and altitude as displayed on his screen and ensure minimum error.
The experiments were conducted in a sound protected environment. A pre-recorded commercially available audio-book was used to provide the irrelevant intelligible speech (IS) test condition. The non-intelligible speech (NIS) was prepared by digitally reversing the speech of the audio-book. The auditory stimulus for Conversation (CN) was in the form of direct speech from the experimenter as a series of questions which had to be answered by the subject. The speech conditions of loudness and rate of words per minute were maintained at similar levels across the three speech conditions.
Statistics
The descriptive statistics of both data sets of error scores on FOPT and the HRV indices were obtained. Analysis of variance (ANOVA) with repeated measures was carried out for FOPT scores and HRV data. Thereafter post-hoc analysis was carried out by Bonferroni’s Multiple Comparison Test to evaluate differences between individual pair of experimental conditions such as between baseline and intelligible speech or intelligible speech and non-intelligible speech. 95% confidence limits were calculated and p < 0.05 was set as the level of significance.
Results
Task Performance
The error scores of FOPT performance under different test conditions of baseline (Base), intelligible speech (IS), non-intelligible speech (NIS) and conversation (CN) is presented in Table 1 and Figure 1.
There is a significant overall effect of irrelevant speech on FOPT performance (F=9.00, p=0.0293).
On post hoc analysis of the effect exerted by the different kinds of speech, intelligible speech produced significantly greater decrements in FOPT scores as compared to that during conversation (p=0.0027).
| Base FOPT | IS FOPT | NIS FOPT | CN FOPT | |
|---|---|---|---|---|
| Error score | Error score | Error score | Error score | |
| Mean | 175113 | 208773 | 164389 | 185968 |
| Std. Deviation | 130450 | 161264 | 94533 | 207882 |
- Error scores of Flight Oriented Performance Task (FOPT) during different test conditions of baseline (Base), intelligible speech (IS), non-intelligible speech (NIS) and conversation (CN)
| Base LF/HF Ratio | IS LF/HF Ratio | NIS LF/HF · Ratio | CN LF/HF Ratio | |
|---|---|---|---|---|
| Mean | 1.0 | 1.7 | 23 | 1.6 |
| Std. Deviation | 0.58 | 1.3 | 1.7 | 1.7 |
Heart Rate Variability
Table 2 shows the LF/HF ratio values under the different test conditions. Table 3 and Figure 3 shows the normalised power for Low Frequency (LF), while Table 4 shows the normalised power in High Frequency (HF).
There is a significant overall effect of irrelevant speech on normalised powers of LF spectrum (F=5.561, p=Q.Q014), HF component (F=5.559, p=0.0014) and LF/HF ratio (F=16.0, p=0.0014).
Post-hoc analysis of the HRV indices in frequency domain revealed that Non-Intelligible speech produced significant differences as compared to the baseline in LF power (p= 0.004), HF power (p= 0.0011) and LF/HF ratio (p=0.0013).
| BaseLF Power | ISLF Power | NISLF Power | CNLF Power | |
|---|---|---|---|---|
| Mean | 47 | 57 | 64 | 53 |
| StcL Deviation | 14 | 16 | 14 | 17 |
- Normalised power of Low Frequency (LF) components of spectral analysis of HRV during different test conditions of Baseline (Base), Intelligible speech (IS), Non-Intelligible speech (MS) and Conversation (CN)
| BaseHF Power | ISHF Power | NISHF Power | CNHF Power | |
|---|---|---|---|---|
| Mean | 53 | 43 | 36 | 47 |
| Stdo Deviation | 14 | 16 | 14 | 17 |
It was also observed that there were significant differences between Non-Intelligible speech and Conversation in all the three indices of HRV.
Discussion
This study was conducted to evaluate the effect of irrelevant speech/conversation on the operator while performing a predefined Flight Oriented Performance Task. To this end, two different parameters i.e. performance evaluation at a Flight Oriented Performance Task in form of error scores and Heart Rate Variability which is a physiological metric had been used.
Overall effect of irrelevant speech
FOPT error scores
A sum of the error scores for the three parameters of heading, altitude and airspeed was used to evaluate the performance at the primary task wherein a lower error score depicted better performance.
The results of the study show a significant overall effect of irrelevant speech on FOPT performance (F=9.00, p=0.0293). In this study, the finding shows that irrelevant speech effect is not restricted to serial recall tasks but also effects performance in a continuous monitoring task like the FOPT. This finding is in accordance with the performance decrements seen in ground based tasks as well as in-flight or simulator based studies [4, 20, 21].
HRV indices
Irrelevant speech had a significant effect on the normalised powers of LF component of the spectral indices of HRV (F=5.561, p=0.0014). The LF power spectrum is evaluated in the range from 0.04 to 0.15Hz. This band especially in its normalised units, is known to reflect mental stress [22]. In controlled experiments of this study lasting for 40 mins and assessing only the overall change and not baseline conditions, this stress directly reflects the change in mental workload. Thus the effort required to maintain performance in the presence of irrelevant speech causes a sympathetic activation which is reflected in changes in the LF component of HRV.
Comparison between different types of irrelevant speech
FOPT error scores
The comparison between task performances under intelligible speech and Conversation showed significant difference from each other (p=0.0027). However the other conditions of irrelevant speech did not produce any statistically significant difference in task performance relative to each other. The difference between means shows the highest error scores or performance decrement occurred during Intelligible speech conditions. The trend shows the next maximum decrement to be during Conversation. The test condition with Non-Intelligible speech showed the lowest error score values.
The level to which intrusion by the distracter takes place is determined by the allocation of processing resources [23]. The highest level of decrement in performance during Intelligible speech is in concurrence with previous research [14]. In case of Non-Intelligible speech, it is possible that the entire episode is categorised as irrelevant and no further resources are allocated to it consciously other than the obligatory pre-attentive organisation that is taking place. On the other hand, on exposure to Intelligible speech the semanticity of the speech prevents the categorisation of the whole episode as irrelevant speech. This speech due to its intelligible component is sampled continuously and checked for irrelevance throughout the period it is available thereby adding to the mental workload.
A significant finding in this study has been the low disruption in FOPT scores caused by Conversation with the experimenter. In the serial recall tasks, articulatory suppression was found to produce a higher level of disruption. This was attributed to the articulatory objects interfering with encoding of the To be Remembered’ items in short term memory [1,24]. In this experiment, the Conversation required accessing the long term memory to retrieve simple stored information. Hence there were enough mental resources available to prevent any task decrement. Since the degree of difficulty of the questions were very low, the subjects who were medical graduates gave more or less automatic responses requiring minimal cognitive resources.
Heart rate variability
The highest scores for the LF power (mean=64, SD=14) were observed during conditions of Non-Intelligible speech. Moreover it was found that Non-Intelligible speech produced significant changes in the LF power when compared to that of the baseline (p=0.0040). This finding suggests that there could be a significant sympathetic activation during the Non-Intelligible speech test condition [22, 25, 26]. The Non-Intelligible speech could be a source of continuous arousal/ irritation/ nuisance value for the subject as evidenced by this autonomic response.
Conversation with the experimenter produced the lowest values of LF power (mean=53, SD=17). It was also observed that there were significant differences in HRV indices between Non-Intelligible speech and Conversation in LF power (p= 0.0209). Thus Conversation utilising long term memory to reply to simple questions does not require high levels of effort on part of the subject.
Intelligible speech seems to produce lower changes in the LF power (mean=57, SD=16). It is probable that the ease of classifying intelligible as relevant or irrelevant is naturalistic to the operator and hence does not extract high physiological costs in terms of effort invested.
Analysis of the interplay of results shown by different parameters
The task performance shows the highest workload to be present during the condition of Intelligible speech. This is however not reflected in the low HRV indices during Intelligible speech. This finding suggests that while semanticity contributes to decrements in task performance, the ease of classifying it as relevant or irrelevant is naturalistic to the operator and hence does not extract high physiological costs in terms of effort invested.
The physiologic metric of HRV reveals greatest sympathetic activation during the presence of Non-Intelligible speech. This trend is not reflected in large decrements in task performance. It may be summarised that though the effect of Non-Intelligible speech was not reflected in task performance, the physiological cost of this distraction is high. Such an effect which is transparent to the conscious percept of the subject is more dangerous as its effects may not be perceived till a threshold is crossed, where it would only be evident as a sudden decrement in performance.
It is therefore evident from the performance as well as the HRV indices that the subjects cope better with irrelevant conversations than with irrelevant intelligible or non-intelligible speech.
Conclusion
This study suggests that irrelevant speech of any kind produces a significant overall effect on the performance of the primary task as well as the physiological state of the individual. This study also suggests that such irrelevant intelligible speech produces the maximum task decrements. This decrement on task performance may become all the more exaggerated in the critical phases of flight or under known conditions of high mental workload such as inflight emergencies or IMC flying. A known effect of Irrelevant speech is its disruption of serial recall tasks [12, 13, 15]. Therefore tasks such as checklists and briefing between pilots before landing are more likely to get disrupted. Moreover the probability of encountering irrelevant speech increases as the aircraft nears the destination and the air traffic increases. Therefore standardisation of phraseology and terminology to be used over the open line is of paramount importance for regulatory authorities.
In this study the effect of Non-intelligible speech has been found to be the least on task performance even though the physiological cost as evidenced by HRV is high. English-only in ATC communications is not mandated on a global scale. This lack of standardization and regulation has led to much variation of english proficiency in the air transport industiy.
In this study, Irrelevant Conversation is seen to produce lower levels of task decrement as compared to Intelligible speech. The Conversation used in the experiments were of the simplest degree of difficulty and primarily accessed the long term memory. This characteristic of irrelevant Conversation supports the use of chat frequencies on long duration sorties. In-flight Conversation between pilots in multicrew cockpit has also been advocated to improve Crew Resource Management (CRM). The very nature of Conversation is that being voluntary it can be terminated at an)’ signs of increasing mental workload. However the effect of such irrelevant Conversation on vigilance needs to be studied.
Conflict of Interest : None.
References
- Locus of the irrelevant speech effect in short-term memory. Journal of Experimental Psychology: Learning, Memory and Cognition. 1991;17:578-84.
- [Google Scholar]
- Explaining the irrelevant sound effect: Temporal distinctiveness or changing state? Journal of Experimental Psychology: Learning, Memory and Cognition. 1999;25:810-814.
- [Google Scholar]
- The intrusiveness of sound: Laboratory findings and their implications for noise abatement. Noise Health. 2001;4:51-70.
- [Google Scholar]
- Applying the Cognitive Streaming Theory to Air Traffic Management: A Preliminary Study. Eurocontrol 2001 Report No. QinetiQ/CHS/ P&D/CR010575/1.0
- [Google Scholar]
- Review of Workload Measurement, Analysis and Interpretation Methods. European Organisation for the Safety of Air Navigation 2003 CARE-Integra-TRS-130-02-WP2
- [Google Scholar]
- Assessing pilot workload: why measure heart rate. HRV and respiration? Biological Psychology. 1992;34:259-88.
- [Google Scholar]
- Integrated Modelling of Cognition and the Information Environment: A Multilevel Investigation (Process and Product Modelling) of Attention Allocation to Visual Displays. :29-52. Technical Report AHFD-04-14/NASA-04-4, NASA Ames Research Center
- An Investigation of the Dissociation between Subjective Measures of Mentiti Work load and Performance. University of Illinois at Urbana-Champaign Report No. NASA-CR-136291
- Open-plan offices: Task performance and mental workload. Journal of Environmental Psychology. 2009;29(2):279-89.
- [Google Scholar]
- Irrelevant speech and irrelevant tones: The relative importance of speech to the irrelevant speech effect. Journal of Experimental Psychology: Learning, Memory and Cognition. 1997;23:472-83.
- [Google Scholar]
- Irrelevant auditory material affects counting. Journal of Experimental Psychology: Learning, Memory, & Cognition. 1998;24:48-67.
- [Google Scholar]
- Irrelevant speech, serial rehearsal and temporal distinctiveness: A new approach to the irrelevant speech effect. Journal of Experimental Psychology: Learning. Memory and Cognition. 1996;22:115-16.
- [Google Scholar]
- Irrelevant speech and irrelevant tones: The relative importance of speech to the irrelevant speech effect. Journal of Experimental Psychology: Learning, Memory and Cognition. 1997;23:472-483.
- [Google Scholar]
- Phonological similarity in the irrelevant speech effect: Within-or between-stream similarity? Journal of Experimental Psychology: Learning-Memory and Cognition. 1995;21(1):103-15.
- [Google Scholar]
- Arousal and vigilance functions in the course of a continuous flight oriented performance task on ground level and their modulation with exposure to non-hypoxic hypobaria simulating cockpit conditions during long duration flying [in Sukhoi-30MKI and Mirage 2000 aircraft]. Institute of Aerospace Medicine AFMRC Project No. 3409/2005
- Evaluation of heart rate variability (HRV) dynamics and its temporal adjustment alter exposure to normo and hyperbaric hyperoxia. Life sciences research board project 2005 Report No. LSRB77/EPB/2005
- [Google Scholar]
- Autonomic modulations and baroreflex sensitivity after short term exposure to hyperoxie hvpobaria. Institute of Aerospace Medicine Departmental Project No. 209/02/2006
- 2008. Kubios PIRV (version 2.0) software developed by Juha-Pekka Niskanen, Mika P. Tarvainen, Perini O. Ranta-aho, Pasi A. Department of Applied Physics. Karjalainen, University of Kuopio, Finland Free software available from http://kubios.uku.fi
- Air-ground communication safety study: An analysis of pilot-controller occurrences. European Organisation For The Safety Of Air Navigation 2004 Report version 1.0
- [Google Scholar]
- Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Heart rate variability Standards of measurement, physiological interpretation, and clinical use. European Heart Journal. 1996;17:354-381.
- [Google Scholar]
- Task combination and the selective intake of information. Acta Psychologica. 1982;50:253-290.
- [Google Scholar]
- Respiration and heart rate variability: A review with special reference to its application in aerospace medicine. Ind J Aerospace Med. 2004;48(11):64-75.
- [Google Scholar]
- Attentional modulation of heart rate variability (HRV) during execution of PC based cognitive tasks. Ind J Aerospace Med. 2003;47(1):1-10.
- [Google Scholar]
