by Eduardo Bellomo, Ph.D. | Scientific Consultant (Brain Products)
and Dr. Mahipal Choudhary | Scientific Consultant (NIRx)
Curious about combining EEG and fNIRS for your mobile research? If the answer is yes, then check out our step-by-step guide on mobile EEG-fNIRS setup with LiveAmp, actiCAP slim/snap, and NIRSport 2.
For detailed information on a stationary EEG-fNIRS setup using actiCHamp Plus, actiCAP slim/snap, and NIRScout, please refer to this dedicated article
Why EEG and fNIRS?
In the last twenty years, the combination of different neurophysiological technologies has attracted the attention of neuroscience researchers. Since co-registration offers the possibility to examine cortical activity more comprehensively than with one modality alone, the field has observed an increase in multimodal measurement techniques. Amongst the many, the co-registration of EEG and fNIRS has a series of functional and practical advantages. On the one hand, EEG directly measures the brain’s rapid electrical activity. On the other hand, fNIRS is linked to the brain’s hemodynamic response, and, specifically, localizes the slower changes in oxygen metabolism that follow neural activation. In other words, EEG and fNIRS signals capture different events linked to the same neurophysiological activity and have complimentary properties: EEG has an exquisite temporal resolution (millisecond precision) but limited spatial resolution; fNIRS has a good spatial resolution (<1 cm) but limited temporal resolution (~ 3 to 6 seconds).
Compared to other neurophysiological technologies, EEG and fNIRS signals do not interfere with each other, making data recording and analysis more straightforward than with other modalities (e.g. fMRI, MEG). While EEG is measuring scalp potentials, fNIRS uses light to measure cortical hemodynamic response. Despite the absence of interference between the two technologies, users should nevertheless keep in mind that EEG-fNIRS co-registrations are still subjected to the same artefacts commonly discussed for individual (unimodal) recordings.
Brain Products and NIRx offer EEG and fNIRS amplifiers that are affordable, lightweight, wearable, and wireless. This combination provides researchers the tools and solutions to conduct multimodal recordings in real-life/mobile settings. Considering the increased number of mobile neurophysiological studies, this article will focus on the hardware combination of Brain Products’ LiveAmp with actiCAP slim electrodes for the EEG part, and NIRx NIRSport 2 for the fNIRS part.
Mobile EEG-fNIRS co-registration
To show you how to combine LiveAmp with actiCAP slim and NIRSport 2 we will walk you through the steps that need to be considered for a successful EEG-fNIRS mobile co-registration.
Montage design and cap setup
In order to combine EEG and fNIRS, we need a cap that is compatible with both types of sensors. The good news is that the actiCAP slim/snap electrodes and NIRSport 2 optodes can be placed adjacent to each other according to the 10-20 system (Jasper et al., 1958) on the same EasyCap. Just make sure you have an EasyCap with enough slits to host both actiCAP snap holders and NIRx grommet bases.
For example, if you plan on using 32 EEG electrodes and an 8×8 fNIRS montage, it is probably best to have 96 or 128 slits. In general, the more slits, the closer EEG and NIRS channels can be to the scalp positions of interest. This might be relevant if, for example, you have a motor task and are using a motor cortex fNIRS montage. In this case, both EEG electrodes and NIRS optodes would compete for central locations (C3, C4, Cz, etc.), but enough slits would ensure the possibility of placing electrodes and optodes as close as possible to the motor cortex. It is up to you, the researcher, to define the desired montage design by deciding whether to prioritize placing an EEG electrode or a NIRS optode on the most relevant channel locations. You can create EEG-fNIRS designs by first defining the fNIRS montage according to the 10-20 system, with NIRSite (NIRx montage design software), and then complement it by selecting the EEG montage based on the same 10-20 system, or vice versa.
Having a defined montage design is crucial because fitting the cap (with actiCAP snap holders and NIRx grommets bases) is a fairly time-consuming procedure that should be carried out prior to the arrival of the participant. Populating the cap follows the same procedure of any unimodal measurement. In our experience, it is more practical to first fit the cap with the NIRx grommets bases in the relevant slits on the EasyCap. Then, the actiCAP snap holders can be placed according to the desired 10-20 system locations, and the actiCAP slim electrodes can be “snapped” into the holders (the procedure is described here or you can check this video).
Note: Because of the nature of the fNIRS signal, using a black fabric cap is recommended, in order to decrease unwanted optical reflection and improve signal quality. Except for the colour, this is the same cap, i.e. actiCAP snap, that comes with the actiCAP slim electrodes (the standard colour is white but black caps can be ordered as a customization).
Checking the signals
Once the montage is defined and the cap is fitted, it is time for your participant to wear it. This procedure is the same as a normal EEG cap setup followed by an fNIRS cap setup. We would recommend first starting to reduce the impedance of the actiCAP slim electrodes with the use of BrainVision Recorder (click here for a detailed description or check this video). Then, the NIRx optodes can be fitted into the grommet holders and signal quality can be checked in Aurora fNIRS.
Triggering and event synchronization via LSL
In order to precisely mark both your EEG and fNIRS data for online and offline analysis, triggering and event synchronization are crucial to any type of co-registration. Since the focus of this article is on mobile EEG-fNIRS, we recommend using software triggering through LabStreamingLayer (LSL). LSL is an open-source system for the unified collection of time series data coming from different sources connected to the same network. In our case, the data stream sources will be the EEG amplifier (the LiveAmp), the fNIRS amplifier (the NIRSport 2), and triggers coming from a stimulus presentation software (in this case PsychoPy). Refer to this blog post on bci.plus for how to view the LSL EEG and Trigger Stream.
Network creation: For LSL to receive all the data streams, the computers to which the LiveAmp and the NIRSport 2 are associated, as well as the stimulus presentation computer, need to be connected to the same network. For this, we recommend creating a private local network either via a wired ethernet connection or through an access point router which grants a stable Wi-Fi connection.
Connecting the streams to LSL: The next step is connecting the data streams in LSL. This needs to be done separately for each the EEG stream, the fNIRS stream, and for the triggers (i.e., stimulus presentation software). This connection is created differently for each device: for the LiveAmp, the connector app is needed; for the NIRSport 2, the relevant checkbox on Aurora must be checked (see Aurora manual, p. 34); for the triggers coming from the stimulus presentation software, the relevant option/line of code must be implemented.
Recording the LSL streams: Once the streams have been created, open LabRecorder, the original LSL acquisition app (LabRecorder.exe) and Aurora, and click “Update” to visualize all streams. You should be able to see the LiveAmp stream, the Aurora stream, and the Trigger Stream. As described here, LabRecorder can store the streams in a single .xdf file, which can then be analysed or converted/exported via MATLAB®/Python. Please note that in the case of fNIRS data, it is possible to record fNIRS and Trigger Stream directly into a .nirs file with Aurora. In this case, LabRecorder would be used to combine only the EEG and Trigger Stream into a single .xdf file.
Monitoring the LSL streams: There is currently no software that can display both types of signal at the same time. Monitoring the fNIRS stream can be done directly from Aurora, which can simultaneously display both fNIRS data and Trigger Stream. The EEG stream together with the Trigger Stream can be monitored with a dedicated Viewer. For more information read this blog post on bci.plus.
Once both EEG and fNIRS data have been recorded and marked with the same triggers, it is possible to analyse them separately with your favourite data analysis software. On the one hand, EEG data stored in the .xdf file can be opened within MATLAB®/EEGLAB or Python and eventually be further converted into BrainVision Analyzer format (see here). On the other hand, fNIRS data can be directly loaded into nirsLAB, Homer2 or NIRS Toolbox, if saved into a .nirs format. Alternatively, similar to EEG data, the file can be extracted from the .xdf file, converted, and exported to one of the aforementioned analysis software.
Note: Please bear in mind that probably because of the differences in the nature of EEG and fNIRS signal, platforms for the streamlined and combined analysis of EEG and fNIRS data have yet to be developed.
We hope you have found this content helpful and that you now feel equipped and ready to start with your EEG-fNIRS experiments. Do not forget to cite us in your next publication so we can see how you are using combined EEG & fNIRS in a mobile setting.