How to verify timing in your EEG setup
by Dr. Alex Kreilinger
Strategic Product Manager (Brain Products)
Before starting a study that involves EEG recording, it is always a good idea to make sure the setup, including software and hardware, works as planned to avoid any unwanted surprises once it’s time to analyze the recorded data. Unfortunately, it is not always obvious if the timing is off. For example, if a hardware trigger does not match the precise time of the corresponding event.
Two undesirable things can happen in this case
First, there can be a constant delay between event trigger and actual event, called latency. Second, the time delay between event trigger and event can vary, which is called jitter. If there is a stable latency, the only effect is that your temporal components may appear sooner or later. For example, a P300 appears some milliseconds later than expected. The effect of a large jitter is worse, though, as averaging waveforms will smoothen the shape of the ERP. This will change the appearance of the ERP and filter out high frequency components.
In a best-case scenario, you want to have no latency and no jitter at all. Since this is virtually impossible, minimizing the two effects should be your next best option. Moreover, if the latency is stable, it is possible to account for it in the analysis if you know the exact value.
In this article …
… we want to demonstrate a specific method to measure delays between the time a hardware trigger is acquired and the actual event it corresponds to. We also want to demonstrate the difference between such hardware triggers and software markers sent via LabStreamingLayer (LSL). To provide an example that everyone can use, we are using PsychoPy, an open-source software based on Python which can be used to run experiments. Using just a bare minimum of code, we demonstrate how to run an experiment specifically designed to point out potential timing issues. In the code, we create flashes on the screen and, at the same time, send a hardware trigger via our TriggerBox that can be recorded with any Brain Products amplifier. At the same time, we also send an LSL marker that can be used as an alternative to these hardware triggers. Please see Figure 1 for a basic scheme showing these three different events and where delays can be expected.
Figure 1: The scheme demonstrates the setup of the timing verification test. PsychoPy creates three events simultaneously: 1) flashing the screen; 2) sending a hardware trigger via the serial interface to the TriggerBox; 3) sending an LSL software marker in a separate LSL stream. Two potential delays are visualized: the delay between command and actual screen refresh, and the delay of the signal as it is passing through the amplifier (this example showing an actiCHamp) and the LSL connector. In the end, an LSL EEG stream that contains the photo signal and the hardware triggers is recorded together with the LSL marker stream in LabRecorder and saved to a single XDF file.
The PsychoPy script for running the experiment
After running the necessary imports for drawing and connecting to the TriggerBox, we define a function that sends hardware triggers to the serial ports and LSL markers at the same time. Then, we define the parameters for the square that flashes on the screen and open the LSL stream with all the required information. After a waiting time of 10 s, we start a loop that lasts for the specified trial number. An important parameter is n_frames. In our example, we set this value to 30 frames, which means half a second for a display with a refresh rate of 60 Hz. In the for loop, the square toggles between black and white every 30 frames. Each time the square goes white, a ‘1’ is sent as a hardware trigger and as an LSL marker. Each time it goes black, the hardware trigger is reset to ‘0’ which is also sent as an LSL marker. Another important detail is that the trigger function, send_triggers, is called on flip with win.callOnFlip(send_triggers, [1]). This means, the triggers and markers are supposed to be sent immediately when the screen refreshes.
Comparing only hardware triggers and screen flashes
The main purpose of the script is to measure the delay between the time of the hardware trigger and the actual event, which is the flashing of the screen. You can easily do this by connecting a TriggerBox to the computer that is running PsychoPy and connecting the TriggerBox output to your Brain Products amplifier. The TriggerBox connects directly with actiCHamp (Plus) and BrainAmp (via the BrainAmp USB 2 Adapter, BUA), or via the Sensor & Trigger Extension for LiveAmp. Now, connect a Photo Sensor to an AUX input and put the sensor on the screen where the flashes appear (per default in the upper left corner). You can also use a StimTrak that generates triggers out of the Photo Sensor signal and connect it to the TriggerBox, but we will assume that the analog signal from the Photo Sensor will be used to generate triggers in the analysis part later.
With this setup, you can record the data any way you want: You can use BrainVision Recorder, or you can use the amplifier-specific LSL connector (see an example for LiveAmp in Figure 2) and LabRecorder. It does not matter, as all the relevant information is only stored in the EEG stream. It does, however, make a difference regarding the software you want to use for analyzing the data, as you will create an XDF file with LabRecorder and BrainVision data format with BrainVision Recorder.
Either way, make sure to specify one AUX channel for the Photo Sensor. You can add EEG channels, but there is no need for them. It is also recommended to select “EEG Channel” for the “LSL Trigger Output Style”.
Figure 2: The settings used for streaming data with the LSL connector for LiveAmp. A single AUX channel contains the photo signal from the Photo Sensor and triggers are sent in an additional EEG channel.
Adding LSL markers to the comparison
If you want to add LSL markers to your analysis, you also need to record the LSL marker stream that is created in PsychoPy. In the previous example, we only focused on hardware triggers. If you want to record the LSL marker stream you must use LabRecorder and select both the LSL marker stream and the LSL EEG stream coming from the amplifier (see Figure 3).
Figure 3: The EEG and LSL stream are selected in LabRecorder and ready to be recorded to an XDF file.
The ideal outcome
What do we expect when we design an experiment in a way that events should happen at the exact same time?
1. The hardware trigger appears at the same time as the Photo Sensor signal starts to rise.
2. The time stamp of the LSL marker in the recorded XDF file is the same as the time stamp of the hardware trigger.
The realistic outcome
Unfortunately, we may not achieve our desired outcome due to several factors:
How this tool should be used
Please keep in mind that this script is very generic and only intended to give you an idea how to verify and test your setup. You may use parts of the code and include it in your own scripts, or you may just get an idea of possible sources for timing inconsistencies and use this information for some further brainstorming. You can also quickly run this script when you want to test some changes in your equipment or settings. We always recommend checking your setup to ensure everything is running smoothly before you start collecting actual data for your study and we hope the exercise provided in this article will find its way into your usual routines.
Analyzing the data
In the provided script, we use MNE-Python to load the XDF file and compare time stamps of the hardware triggers, LSL markers, and the Photo Sensor signal. Please download the analysis script and the XDF files (see box below) that were created with the previously explained setup. There is one file for LiveAmp (“TimingTest_LA.xdf”) and one for actiCHamp/actiCHamp Plus (“TimingTest_aC.xdf”), the main difference being the different signal delays, which you will notice after running the analysis.
Before you can run the script, you need to make sure to install the following packages, for example with a command such as pip install mnelab pyxdf PySide2 rich. Make sure you install at least MNELAB version 0.8.6.
Executing the script (for example in Spyder) will result in a table with some statistics and a figure (Figure 4), demonstrating the nature of the delays for a LiveAmp (left) and an actiCHamp/actiCHamp Plus (right) example. We analyze the delays between photo signal and hardware triggers, between photo signal and LSL markers, and between hardware triggers and LSL markers. The delay between photo signal and hardware trigger is almost the same for both amplifiers, which is to be expected because it is independent from the acquisition device. The delay of ~18 ms indicates that the square flashes only after a full screen update cycle at 60 Hz. The delay between hardware triggers and LSL markers is the main difference between the two recordings. It shows that the LSL stream from LiveAmp has a higher latency (~13 ms) than the one from actiCHamp/actiCHamp Plus (~3 ms). This difference also results in the longer delay between time stamps of the photo signal versus LSL markers, on average more than 30 ms for LiveAmp. These additional 30 ms should be considered when LSL markers are used as starting points for epochs containing events related to the actual flashing of the screen. In chronological order, the LSL markers appear first, followed by the hardware triggers, and then the screen updates come last.
Figure 4: Delays over 300 trials between the three events: onset of the photo signal, hardware triggers, and LSL markers. LiveAmp on the left and actiCHamp (Plus) on the right.
Please feel free to adjust the script to your needs and test with other files you recorded yourself.
The MNE-Python script for analyzing the data
This script was provided by Clemens Brunner, one of the core contributors of MNE and developer of MNELAB. Please check out his website for some very interesting and relevant tutorials regarding Python and EEG.
The script begins with importing all the relevant packages, followed by the definition of a function to summarize the results statistically. The XDF file is loaded and the correct EEG stream is identified (as the sequence of the stream IDs is random). Next, we pick the channels we are interested in: AUX1 is the channel containing the photo signal and the last channel contains the hardware triggers (either “STETriggerIn” for LiveAmp or “Markers” for actiCHamp/actiCHamp Plus). We get the time stamps of the photo signal onsets by thresholding the difference of the analogue signal and discard the final event which is caused by the screen returning to bright at the end of the measurement. The LSL events are directly read as annotations; we only take every second annotation, though, as we are not interested in the zeros. We get the hardware triggers by simply looking at values greater than zero, because this channel is always ‘-1’, except when indicating a trigger, then it is ‘1’ followed by a ‘0’. Now that we have all the indices or time stamps, we can compare the delays between the three modalities and create the statistics and figure.
As always, please don’t hesitate to reach out to us if you have any questions on timing verification in general or require assistance related to your specific setup.