Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights:
an interview with author, Anneke Hamann
by Stefanie Rudrich
Head of Marketing (Brain Products)
We love seeing our solutions used in publications. The popularity of the user research articles we regularly publish, proves that you, our readers, do so too. In the spirit of keeping things fresh, we decided to add a new approach to promote user research and interview the authors. This way, we can get down to the important points and have a fun exchange while learning about their results and other scientific endeavors.
Our first interview partner was Anneke Hamann (German Aerospace Center, Institute of Flight Guidance, Braunschweig, Germany). We spoke about her recent paper (Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights), in which she and Nils Carstengerdes investigated pilots’ cognitive states and analyzed the effects of stepwise increased mental workload while controlling for fatigue using both EEG & fNIRS.
Your research on cognitive states specifically explores the importance of human factors, like mental workload, in the aviation industry. What are some key characteristics you would like to explore with your research?
A. Hamann: In aviation, there is a constant trend towards higher levels of automation. While human operators such as pilots and air traffic controllers still play an important role and will do so in the foreseeable future, their tasks and responsibilities will gradually change from active engagement to a more passive monitoring of systems. At the Institute of Flight Guidance of the German Aerospace Center (DLR), Braunschweig, research is undertaken to understand the challenges the operators will face, and to develop new operational concepts, interfaces and assistance systems to support them. We believe that the physiological assessment of the operators’ current cognitive state will help us to evaluate such new concepts and interfaces, and provide adaptive and intelligent assistance systems with information about the operators’ current needs. We work on defining valid and reliable EEG and fNIRS measures that can be used for such applications, and on understanding and accounting for interactions of different cognitive states. In our current research, we focus on mental workload and mental fatigue, two of the most central concepts of human performance in aviation, and aim to identify physiological measures that allow us to differentiate both states.
What kind of experimental setup do you use to investigate cognitive states in aviation in your participants?
A. Hamann: Depending on the research goal, we use different flight simulators with varying degrees of realism, and utilize expert samples. For this study, however, we explicitly wanted to control the environment, task demands and sample characteristics as much as possible. Therefore, we used a low-fidelity but highly flexible A321 cockpit simulator and recruited a student sample. Our aim was to identify valid physiological measures that differentiate levels of mental workload in a cockpit environment while controlling for confounding effects of mental fatigue. Therefore, we designed a simplified flight task that puts similar cognitive demands on the participants as a real flight would do, but that they could learn easily: The participants monitored and corrected the altitude of their aircraft, and changed its heading in accordance with an adapted n-back task in four difficulty levels. We controlled for mental fatigue by means of task duration and randomization. We recorded the participants’ brain activity with concurrent EEG-fNIRS measurement with a Brain Products LiveAmp-32 with active electrodes, and a NIRx NIRSport2 with 8 sources, 8 detectors and additional short-distance channels.
Why did you decide to combine EEG and fNIRS technology for this study? How did this multimodal approach help you address your research goals?
A. Hamann: Both EEG and fNIRS have advantages and shortcomings. With EEG, you can achieve high temporal precision in your measurements, but movement artefacts and noise from the simulator environment can limit the data quality. There is a large body of research on mental workload assessment with EEG, but tasks and analysis strategies vary across studies. On the other hand, fNIRS measurement has a higher spatial precision than EEG, and is less influenced by movement artefacts. However, systemic artefacts have to be considered and controlled for. Finally, not much fNIRS research has been done in aviation yet, and montage designs and analysis strategies vary considerably. With the combination of both measurements we wanted to make use of the complementary nature of the measurements and cross-validate the results. The real challenge was to design an fNIRS montage around the EEG positions that would optimally cover the areas of interest and still be comparable to other fNIRS-only studies. This, and getting the electrode gel out of the multitude of cables!
Which analyses did you perform on your recorded EEG and fNIRS data?
A. Hamann: We performed spectral analyses on the EEG data. These have the great advantage over evoked potentials that no additional stimulus is needed that could distract the pilot in a real flight. We focused on frontal theta and parietal alpha and beta power because these bands and locations have shown high sensitivity to mental workload in past studies. For the fNIRS data, we used a montage designed to optimally cover dorsolateral prefrontal cortical area which have proven to be sensitive to changes in task difficulty in n-back tasks and driving simulations. We used a GLM-based approach and controlled for systemic and motion artefacts by including data from short-distance channels as an additional predictor. In both datasets, we compared the four induced n-back levels in order to see how many levels of mental workload could be differentiated with which method.
How do you interpret your findings for EEG and fNIRS’ sensitivity to task difficulty?
A. Hamann: We found that increasing mental workload is associated with increasing activation in frontal cortical areas, i.e. increasing frontal theta power and increasing cortical oxygenation. Both methods, EEG and fNIRS, were able to detect changes and differentiate most of the n-back levels. Interestingly, EEG proved to be more sensitive to high demands and could differentiate between the highest n-back levels, but not the lowest. fNIRS, on the other hand, could not differentiate between the highest levels, but indicated changes between the lowest levels where not even performance was impaired. Only with the combination of both data sets could we differentiate all four induced levels of mental workload. Covering the whole range from low to (very) high mental workload will help us to get an overall picture of the pilot’s state, observe changes and be able to intervene before mental workload reaches a level where performance starts to decline. We therefore conclude that EEG and fNIRS are indeed complimentary and mental workload assessment benefits from combining them.
Can you please provide an overview of your future research plans?
A. Hamann: We started out inducing mental workload while controlling for mental fatigue. Adding to this research, we have just completed a follow-up study on mental fatigue in which we used the same setup and tasks, but explicitly controlled for mental workload. With the results gained from both studies under highly controlled conditions, we can draw inferences as to which EEG and/or fNIRS measures are most sensitive to which state and how to differentiate them. These controlled experiments lay the foundation for future research in more applied settings. Next, we will expand our research to expert samples, more realistic flight simulators and scenarios, and non-nominal flight conditions (i.e. when things do not go as planned, like an engine failure). There are still many questions to be answered before adaptive assistance systems based on physiological assessment will find their way into real cockpits, but our results show that EEG and fNIRS are highly promising candidates for these applications.
We’d like to thank Anneke very much for taking the time to answer our questions. We enjoyed it a lot and hope you, our readers, also like this new approach to present our users’ research. We’ll for sure keep our eyes open for more publications to be featured in our newsletter. To make sure your paper can be easily spotted by us, please keep in mind to cite us and our solutions properly.
Author short biographies
- Anneke Hamann is a PhD student at the Human Factors Department of the Institute of Flight Guidance, DLR Braunschweig (Germany). She received her degrees in psychology at University Jena and Technical University Dresden (Germany).
- Nils Carstengerdes is Head of the Human Factors Department of the Institute of Flight Guidance at DLR Braunschweig (Germany). He received his doctoral degree in natural sciences (Dr. rer. nat.) and his diploma in psychology from Trier University (Germany). He is a lecturer for human factors at the universities of Trier and Hildesheim (Germany).
- Publication: Hamann, A., Carstengerdes, N., Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights. Sci Rep 12, 6449 (2022). https://doi.org/10.1038/s41598-022-10044-y
- Webinar: NIRx Journal Club with Anneke Hamann – Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights (Watch the recording [coming soon])