The sound of silence: an EEG study of how musicians time pauses in individual and joint music performance

Compared to other senses, very little is known about taste. In a recent study (Crouzet, S. M., Busch, N. A., & Ohla, K. (2015). Taste quality decoding parallels taste sensations. Current Biology, 25(7), 890-896.), we used mulitvariate pattern analysis of single-trial EEG data to investigate which information about a taste is represented in taste-evoked brain responses.

In everyday language, “taste” is often used in to describe a multisensory sensation that involves taste proper, but also other sensations such as olfaction and somatosensation (think of texture and temperature). By contrast, when we talk about taste in chemosensory research, we refer specifically to the sense that is mediated by dedicated gustatory receptors, mostly on the tongue.

Taste is a peculiar sense. One aspect in which it differs from other senses is the categorical structure of gustatory phenomenology. For example, we could describe our phenomenology of having a cup of coffee as “I see the cup”, and “it smells like coffee”, and “it tastes bitter”. Thus, when we describe our perception in other senses, we usually refer to the object evoking this perception, but we describe taste only with a categorical label: bitter, sour, sweet, salty, or umami (savory). These categories are also referred to as “taste qualities”. Therefore, one important research question in taste research is to clarify how these taste qualities are represented in the human cortex.

Unraveling the neural code underlying taste quality perception remains a major challenge in gustatory research. This is due, in part, to the fact that chemical stimuli delivered into the mouth are notoriously difficult to control with respect to their physical properties, i.e., place of stimulation and onset/offset. Most of our current understanding of the cortical areas involved in human taste perception is based on neuroimaging techniques that are relatively forgiving to imprecise stimulus onset, such as functional magnetic resonance imaging (fMRI) or positron emission tomography (PET). By comparison, only few studies so far have measured EEG and thus little is known about the temporal dynamics of the neural representation of taste (see Ohla, Busch, & Lundstrom, 2012).

A fundamental question that needs to be addressed in taste research is what these neural responses mean – which information is encoded in the signal? What does the gustatory system tell itself and the rest of the brain about the taste and how are these responses related to subjective taste perception? This question is difficult to address with conventional, univariate event-related potential (ERP) analyses. For instance, several studies have demonstrated that the amplitude of the gustatory-evoked ERP is dependent on the intensity of the gustatory stimulus (Ohla, Toepel, le Coutre, & Hudry, 2010; Tzieropoulos, Rytz, Hudry, & le Coutre, 2013). Does this imply that the neuronal processes underlying this ERP response are used by the brain to code intensity, rather than, say, taste quality? Likewise, a study comparing ERPs evoked by different tastants may demonstrate amplitude differences between tastes, but does this imply that the brain uses the underlying neural processes to represent taste category? Generally speaking, univariate ERP analyses test whether the ERP amplitude is bigger in condition A than in condition B. To study whether a neural signal is involved in coding a given feature, an alternative avenue is to test whether the neural signal contains information about the feature. If so, it should be possible to analyze the data recorded on a given trial and decide based on these data which feature was presented on that trial. This approach is called decoding (popularized under the term “brain reading”). The reasoning is that if a neural signal allows us to decode information about a feature, it is likely that the brain uses the same neural signal for coding this feature.

Thus, to address the question whether the taste-evoked EEG response codes information about taste quality rather than other taste features such as intensity or pleasantness, we used time-resolved multivariate pattern analysis (MVPA) to evaluate whether the single-trial, instantaneous topographical pattern of electrophysiological activity carries information about taste quality (Crouzet, Busch, & Ohla, 2015). MVPA leverages information in the topographical pattern on single trials in single subjects, thereby allowing to directly relate on a single-trial basis brain responses with subsequent behavior. Therefore, MVPA in humans offers the opportunity to relate this information to the subjective perception reported by the participants.

Figure 1: Event-Related Responses to Different Tastes Exhibited Differences in Electric Field Strength and Distribution (A) Schematic of an experimental trial. Atomized tastants (indicated in black) were embedded in a regular stream of water sprays (gray). (B) Global field power (GFP) for each taste category (averaged across participants). The time period during which the GFP differed from pre-stimulus levels is marked with a gray line for all tastes and with colored lines for individual tastes. Only p < 0.05 lasting for at least 100 ms are shown. (C) Periods of significant topographical differences (global map dissimilarity) between taste categories are indicated in black. The top row depicts the main effect of taste categories (only p < 0.05 lasting for > 100 ms are shown); subsequent rows represent all pairwise comparisons: salty versus sweet, salty versus sour, salty versus bitter, sweet versus sour, sweet versus bitter, and sour versus bitter (p < 0.05 for > 30 ms). (D) Estimates of neural sources underlying the initial GFP-normalized taste-evoked responses (P1) include the frontal operculum and insula (predominantly of the left hemisphere), the bilateral superior temporal gyrus, and the cuneus for all tastes. (Reproduced with permission from (Crouzet et al., 2015))

To achieve high precision regarding the timing and intensity of gustatory stimulation, we used a computer-controlled spray gustometer. This device delivers unimodal taste solutions at a constant temperature. It does so by embedding taste “pulses” in a regular sequence of tasteless control pulses. The advantage of this technique is multifarious: it prevents dilution or mixing of the tastants, it enables steep rise times, it stimulates a confined area, and it diminishes concomitant lingual somatosensory activation, to name the most critical.

We recorded EEG from 64 channels using Brain Products’ actiCHamp amplifier system. 16 participants tasted salty, sweet, sour, and bitter solutions and performed a delayed four-alternative forced-choice (4AFC) taste discrimination task on each trial. The tastes evoked a significant electrophysiological response starting at 150 ms after taste onset. Time-resolved MVPA was used to evaluate whether the single-trial, instantaneous topographical pattern of these responses carries information about taste quality using either multi-class, one-vs-one, or one-vs-all taste classification. A random sample corresponding to 90% of the trials was utilized for training, leaving the remaining 10% for testing.

In the multi-class analysis, the classifier was trained to provide an answer to the question: “Which taste is this (any of sweet, salty, sour, or bitter)?”. This is equivalent to the four-alternative forced choice tasks performed by the subjects. Taste quality could be decoded significantly above chance level at as early as 175 ms after stimulus onset indicating that taste quality is among the first attributes of a taste processed by the brain.

Participants correctly categorized the tastes on 86% of all trials, and the majority of the categorization errors occurred due to the confusion of salty with sour tastants and vice versa. These confusions allowed us to evaluate whether the perceptual similarity between salty and sour stemmed from neuronal similarity between salty- and sour-evoked response patterns by investigating whether the classifiers were susceptible to the same confusion. Since the multi-class decoding analysis corresponded to the participants’ behavioral task (i.e. answering “Which taste is this?”), we obtained a dynamic neural confusion matrix that represented for each time point the proportion of trials on which the classifier categorized taste X as taste Y. These matrices are a representation of the neuronal space in which taste categories are coded since it shows which taste-evoked neural patterns are similar and thus are frequently confused by the classifier. We also obtained each subjects’ behavioral confusion matrix, which is a representation of subjects’ perceptual space since it shows which taste categories are subjectively similar and so are most frequently confused by the subjects. Neural and behavioral confusion matrices were significantly correlated starting at 175 ms after stimulus presentation. Thus, the time at which the classifiers’ error patterns correlated with that of the participants also coincided with the earliest time at which the neuronal signal carried information about taste quality as indicated by the multi-class analysis. This result confirms that the information used by the classifiers for decoding formed the basis for the perceptual decision making of each participant.

To identify the neural network underlying the earliest taste-evoked response that carried taste quality information, the cortical generators of the P1 component of the taste-evoked response were estimated for each taste. Areas previously associated with gustatory processing in general and taste quality processing in particular including the anterior and mid insula and the overlying frontal operculum, the parietal operculum, the superior temporal gyrus, and the cuneus were activated by all tastes. These observations suggest that the earliest sweep of activation through the gustatory network not only codes that any taste was detected, but already allows for taste quality discrimination.

Figure 2: MVPA Yielded Significant Decoding Starting at 175 ms and a Close Relation to the Pattern of Errors of the Participants (A) For a given participant and time point, a random sample of 90% of the trials was used to train a classifier to discriminate between brain responses (EEG scalp topography) associated with two (one-versus-one classification) or four (multi-class discrimination) tastants. Classification performance was then evaluated by using the remaining 10% of the trials. The entire procedure was repeated for 120 cross-validations. (B) Multi-class decoding: classifiers were trained to provide at each time point an answer to the following question: ‘‘Which taste is this?’’ The black curve indicates decoding accuracy averaged across participants (shaded area: bootstrapped 95% confidence interval; light gray curve: average chance level). The thin (top) and thick (bottom) horizontal lines correspond to the period of significant decoding for individual participants and at the group level, respectively. (C) Representational similarity analysis: behavioral confusion matrices were correlated with neural confusion matrices at each time point to estimate the similarity between taste representations in perceptual space and in neuronal space. Conventions are as in (B). (D) One-versus-one decoding: classifiers were trained to provide at each time point an answer to the following question: ‘‘Is this taste X or taste Y?’’ Conventions are as in (B). (Reproduced with permission from (Crouzet et al., 2015)).

A full account of gustatory processing in humans must factor in the chemical, the neural, and the perceptual domains. All three domains can be represented as a space – an abstract representation in which distances correspond to similarity relations. For example, in the neural space, each dimension corresponds to the activity of one neural unit or, as in our case, electrode. If two tastants evoke neural patterns that are close in neural space, their response patterns are similar. Likewise, two tastes that are close in perceptual space are expected to taste similar. While considerable effort has been made to elucidate how the chemical space is encoded within the neural space, e.g. by identifying receptors for different tastants on the tongue and also by characterizing response patterns of peripheral or cortical gustatory neurons, the mapping of the neural space onto perceptual space – the role of neural response patterns in subjective gustatory perception and gustatory-related behavior – has received little attention so ar. Our study is the first to investigate this mapping in humans by using multivariate pattern analysis of EEG and psychophysical data

In sum, our results provide the first evidence that taste quality is encoded at the earliest level of the taste processing cascade, much earlier than traditionally thought, and that the information encoded in the multivariate neural response patterns is also the basis for gustatory perceptual decision making.