Combining mobile EEG and immersive VR to increase ecological validity in emotion research

This user research article summarizes our publication: El Basbasse, Y., Packheiser, J., Peterburs, J., Maymon, C., Güntürkün, O., Grimshaw, G., & Ocklenburg, S. (2023). Walk the plank! Using mobile electroencephalography to investigate emotional lateralization of immersive fear in virtual reality. Royal Society open science, 10(5), 221239. https://doi.org/10.1098/rsos.221239

Emotions are an important part of our lives, and understanding how they emerge holds great relevance not only for basic neuroscience but also for clinical psychology. Thus, it comes to no surprise that when searching the scientific database PubMed for the search terms “EEG” and “emotions”, more than 11,000 publications can be found. This testifies that emotion research is a major subfield of EEG research in neuroscience. Despite the high number of EEG publications in affective neuroscience, the field is not without problems. One of the core issues in investigating emotions using EEG is how artificial the test situation often is. In real life, emotions are experienced in complex environments that entail many different sensory, motor, and physiological aspects. In contrast, in a typical stationary EEG experiment on emotion processing, participants sit still in front of a computer screen and watch pictures or movies that are thought to induce emotions, e.g., photos of sad or happy faces, pictures of disgusting or dangerous things, or sad movies. The core problem from a research methods point of view is that these paradigms have low ecological validity. Take for example sadness: A typical way to study sadness in an EEG experiment would be to show a picture of the face of a sad person and then assess which effect viewing that stimulus has on the EEG signal. However, when experiencing sadness in real life, there are many more aspects to it than merely looking at sad faces. For example, if someone visits the funeral of a good friend who suddenly passed, they will indeed see many sad faces. However, in addition to this sensory component, there will also be cognitive aspects of sadness (e.g., memories of the friend), motor behavior related to sadness (e.g., hugging other friends at the funeral, crying), and physiological aspects (e.g., increased levels of stress hormones). Thus, just looking at a picture of a sad person does not elicit the same emotional response as experiencing actual sadness in a real-life situation.

Obviously, low ecological validity of many studies on emotions poses a major problem for affective neuroscience. If our aim is to understand the neural correlates of emotions, it is mandatory to develop paradigms in affective neuroscience that have high ecological validity. This was the major aim of our recently published preregistered mobile EEG study (El Basbasse et al., 2023). We compared the neuronal correlates of experimentally induced fear in two paradigms with varying levels of ecological validity. The first task used a naturalistic VR setting with high ecological validity. Here, participants balanced either on a virtual plank on a tall skyscraper (negative condition) to induce fear of heights, or on the ground (neutral condition). The second task comprised a typical fear induction paradigm with low ecological validity during which the participants were presented with fearful or neutral emotional images. Specifically, we assessed to which extent the high ecological validity and the low ecological validity condition differed in terms of the well-known rightward asymmetry of the neural response to negative emotional stimuli.

This is a shortened version of the methods description. For the full method section, please refer to the original publication (El Basbasse et al., 2023).

Participants

Overall, data of 75 participants (45 women, 30 men, mean age: 24 years) were included. The study protocol was approved by the local ethics committee of the Faculty of Psychology at Ruhr University Bochum, Germany. All participants gave their written informed consent at the beginning of the experiment. The experiment was conducted in accordance with the Declaration of Helsinki.

Stimuli & Procedure

The experiment consisted of two tasks that were combined with mobile EEG (mEEG), one with low and one with high ecological validity.

mEEG + pictures task (low ecological validity): This task used 50 neutral and 50 emotionally negative pictures as stimulus material.

mEEG + VR task (high ecological validity): The VR simulation was presented using an HTC Vive head-mounted display (HMD) with corresponding hand controllers (https://www.vive.com/us/) (see Figure 1). Participants walked across a virtual plank which extended from the side of a skyscraper — either 80 floors up (the negative condition) or at street level (the neutral condition) (see Figure 2 for example views of the virtual environment). To increase immersiveness, participants walked on an actual wooden plank in real-life while performing the task (Figure 1).

Figure 1: The mEEG + VR plank task. (a) The first recording started when participants had positioned themselves in front of the plank (elevator). In the VR environment, the view of the participant would have been from within the elevator either on the street level (neutral condition) or 80 floors up (negative condition). (b) The second recording segment took place after participants had taken a step onto the plank (start of plank). (c) The final recording segment was taken at the end of the plank (end of plank).

Figure 2: Example views from the participants’ perspective. (a) Street view from inside the elevator in the neutral condition. (b) Street view in the start of plank recording segment in the neutral condition. (c) View from the building top from inside the elevator in the negative condition. (d) Downwards view from the end of plank recording segment in the negative condition.

mEEG recording

Brain activation was measured with a mobile EEG system (LiveAmp 32, Brain Products GmbH, Gilching, Germany). This system consists of 32 electrodes arranged in line with the international 10–20 system. The position of the Fpz electrode was set as the ground electrode, and FCz served as on-line reference. A wireless amplifier amplified the recorded activation, transmitting it to the recording software (BrainVision Recorder) on a laptop with a sampling rate of 1000 Hz. The impedance cut-off was set to less than 10 kΩ for an adequate EEG signal. Three acceleration sensors implemented in the wireless amplifier measured head and body movements along the X-, Y- and Z-axes. Several EEG caps in different sizes were adjusted for VR-compatible use by MES Forschungssysteme GmbH (https://mes.gmbh/). To this end, five Velcro straps were sewn onto each cap, so that the HMD could be attached without interfering with the electrodes (Figure 3).

mEEG processing

BrainVision Analyzer (Brain Products GmbH, Gilching, Germany) was used for EEG data processing. First, the sampling rate was adjusted, data were filtered, and faulty channels interpolated. An infomax independent component analysis extracted pulse and eye artefacts (i.e., blinking and horizontal eye movement). The reference electrode FCz was interpolated. The data from the plank task were then segmented according to the three markers (S1: elevator; S2: start of plank; S3: end of plank) set with the manual triggers. Segment length of the first two markers varied, since each following marker constituted the end of the prior interval. The segment period for the third marker was 60s. Overlap of segments was not allowed. The average of all channels was then set as the new reference (Avg). A second filter was applied with a low cut-off at 1 Hz and a high cut-off at 45 Hz. Segments with a length of 1 s were created via segmentation. A fast Fourier transformation was applied to extract alpha oscillations (8–12 Hz) from the signal. Last, the average power was calculated for each recording segment in the plank conditions, and alpha asymmetries were determined.

In the picture task, the pre-processing steps were the same as for the plank task except that there were no segments following marker positions. Segments of 1 s were created for the whole length of the paradigm regardless of stimulus markers. Consequently, only one average was calculated for the whole picture condition for later power analysis.