EEG shows brain can simultaneous encode two speech streams

Abstract

Successful speech communication in multi-talker scenarios requires a skillful combination of sustained attention and rapid attention switching. While the neurophysiology literature offers detailed insights into the neural underpinnings of sustained attention, there remains considerable uncertainty on how attention switching takes place.

In this study, using EEG recordings from normal-hearing adults in an immersive multi-talker environment, we measured the neural encoding of two competing speech streams amid background babble. Participants were cued to switch attention between streams every 15-30 s. Neural tracking was assessed via Temporal Response Functions (TRF), confirming reliable decoding of attentional focus.

Our results indicate asymmetric disengagement and engagement processes during attention switches, where the neural tracking of the new target stream emerges before disengaging from the previous target, revealing a transient simultaneous encoding of two speech streams. That transition was closely mirrored by a reduction in EEG alpha power, informing on the cognitive effort during different phases of the attention switch.

We then isolated cortical activity reflecting lexical prediction mechanisms to determine how lexical context is updated after an attention switch, comparing four context-accumulation strategies that were constructed using Large Language Models. Our findings elucidate both the temporal and contextual mechanisms underlying auditory attention shifts, pointing to the possibility that listeners carry out a reset in lexical context after switching attention.

By focusing on dynamic attentional reallocation, this study offers insights into the brain's capacity for flexible speech processing in complex listening environments.

Citation: Carta S, Aličković E, Zaar J, López Valdés A, Di Liberto GM (2026) Competing speech streams are simultaneously represented in the human cortex during attention switching. PLoS Biol 24(7): e3003876. https://doi.org/10.1371/journal.pbio.3003876

Academic Editor: Manuel S. Malmierca, Universidad de Salamanca, SPAIN

Received: July 3, 2025; Accepted: June 12, 2026; Published: July 16, 2026

Copyright: © 2026 Carta et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data supporting the findings reported in this manuscript are freely accessible without restriction. The EEG pre-processed dataset, the resulting analysis files, and the analysis code are publicly available on the open repository Zenodo (https://zenodo.org/records/20569817). The EEG recordings are provided following the Continuous-event Neural Data (CND) format standard. The associated speech stimuli can also be found in the same repository, within the STIMULI folder.

Funding: S.C., A.L.V., and G.D.L. were supported by the William Demant Fonden (https://www.williamdemantfonden.dk/), under grants 21-0628 and 22-0552, and by Taighde Éireann - Research Ireland (https://www.researchireland.ie/) under grant No. 18/CRT/6223. G.D.L. additionally conducted this research with the financial support of Research Ireland at ADAPT, the Research Ireland Centre for AI-Driven Digital Content Technology (https://www.adaptcentre.ie/) at Trinity College Dublin [grant 13/RC/2106_P2]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations:

• EEG, electroencephalography
• EOG, electro-oculography
• EMG, electro-myography
• ERSP, event-related spectral perturbation
• iEEG, intra-cranial electroencephalography
• FDR, false discovery rate
• fMRI, functional magnetic resonance imaging
• ICA, Independent Component Analysis
• IQR, interquartile range
• LLM, large language model
• MEG, magnetoencephalography
• PSD, power spectral density
• RMS, root-mean-squared
• SE, standard error
• SEM, standard error of the mean
• SNR, signal-to-noise ratio
• SPL, sound pressure level
• TRF, Temporal Response Functions

Introduction

To understand speech in multi-talker environments, listeners single out the target speaker from competing sound streams [1-3]. The neurophysiology of this selective attention process has been widely studied with simulated cocktail-party scenarios [4,5], shedding light on how our brains segregate a target stream from competing speech streams, and enabling the transformation of the target speech into linguistic meaning.

While the extent to which masker speech streams are processed remains highly debated [6-8], there is no doubt that there are considerable differences between the processing of target and masker speech, which have been measured with various technologies, such as non-invasive electroencephalography (EEG) [1,9], intra-cranial electroencephalography (iEEG) [10], magnetoencephalography (MEG) [3,11] and functional magnetic resonance imaging (fMRI) [12,13]. That work could pinpoint precise loci in the auditory cortical areas where that segregation emerges [14] as well as measuring the substantial (but not total) suppression of linguistic processing for the masker speech [1,15-17].

However, neurophysiology literature in this field has almost entirely focused on sustained attention tasks [2,10], leaving considerable uncertainty on the neural underpinnings of attention switching.

Dynamic switching paradigms have been widely used in the domain of cognitive control studies to probe for cognitive flexibility and cognitive stability [18]. In those experiments, participants are often required to flexibly adapt their behavioral response depending on new instructions, initiating a task-switch [19-21]. For example, given a single digit, they are required to classify it either based on parity, i.e., whether it is even or odd, or based on relative magnitude, i.e., whether the digit is greater than or less than 5 [22]. In these paradigms, the switch-cost is the increase in reaction time or error rate when switching from one task to the other.

Similar behavioral paradigms have also involved simple speech stimuli in multi-talker settings [23-25]. However, the main interest of those tightly controlled experiments was to model the process of target speech selection as one particular instance of a task-switching problem, i.e., target stream selection could either depend on spatial location or voice identity [23], rather than focusing on the dynamic aspect of attention re-allocation per se in naturalistic multi-talker scenarios. As such, very little is known on how a flexible reorienting of attention might impact speech processing of continuous competing streams.

In recent speech neurophysiology research, experimental paradigms have started to include switches of attention as a tool towards tailored EEG/MEG methodological advances in the domain of attention decoding [26,27], or to investigate how sustained speech attention unfolds for moving auditory objects [28]. However, to the best of our knowledge, only one previous study has specifically focused on the neurophysiology of attention switching in multi-talker scenarios, relating the neural encoding of speech during attentional re-orienting with EEG alpha activity and pupil dilation dynamics [29]. Those findings proved that the neurophysiology of attention switching can be studied non-invasively.

Building on that work, our study sheds light on the exact neural dynamics supporting the steering of attention between two competing speech streams, disengaging from the previous target stream while engaging to the new one.

In this study, we measure the neural encoding of speech using a range of encoding window lengths, as listeners steer their attention from one speaker to another. We test whether engagement with a new speech stream begins before disengagement from the previous target is complete, resulting in a brief period of simultaneous tracking of both streams. Such an asymmetry in the disengagement-engagement processes, even if transient, could support the ability to explore alternative auditory streams while maintaining attention to a given stream [30].

The neural encoding of speech was measured from normal-hearing adult participants using EEG during an immersive multi-talker listening task. Participants were exposed to two competing speech streams from TED talks, presented via two front-facing loudspeakers, while background noise from a 16-talker speech babble played from rear loudspeakers (Fig 1A). An on-screen arrow cued participants to attend to one of the two speech streams and to shift their attention rapidly whenever the arrow changed direction, approximately every 10-30 s (Fig 1B).

Neural tracking of target and masker speech was quantified using the Temporal Response Function (TRF), describing the linear relationship between each speech stream and the neural responses. As an initial validation, we confirmed that the attended stream could be reliably decoded from the EEG, consistent with the extensive literature on sustained attention [9,10,31]. This confirms that the EEG responses in this experiment reflects differential encoding of target versus masker speech (Fig 1C).

• (A) Participants were presented with speech from two loudspeakers placed in front of them with 60° of separation (30° left and 30° right), and with concurrent 16-talker background noise (B1-B4). In each trial, the screen presented an arrow pointing to the target speech stream. Participants were instructed to switch attention as soon as the visual cue changes direction.
• (B) Schematic diagram of one experimental trial. The black area represents blocks of attention either to the left (L) or right (R) front streams. The red arrows indicate the instants where the attention cue switches side (six times per trial). Note that block duration was randomized and always between 15 and 30 s, with trials lasting 3 min.
• (C) EEG data validation was carried out by running an attention decoding analysis. Progressively longer decoding windows were considered (larger windows use more data, typically leading to more accurate decoding scores). Binary classification scores are reported arbitrating between the target and masker streams. The dashed line indicates the 95th percentile of a random distribution calculated by randomizing the classification labels. Statistically significant attention decoding classification scores were measured for all the decoding windows considered, with numerical results comparable with previous studies on selective attention [31,34,35]. Data supporting this figure is available at: https://zenodo.org/records/20569817.

We next addressed two fundamental questions about the neural mechanisms underlying attention switching in naturalistic listening.

First, we asked whether the processes of engaging with a new speech stream and disengaging from a previous one unfold symmetrically (Figs 2 and 3). To test this, we fit encoding TRF models to EEG data, measuring the neural tracking of the two competing speech streams over time. This allowed us to characterize the average encoding dynamics surrounding attention switches, comparing disengagement and engagement processes.

The second objective was to understand how our brains update and use lexical context when switching attention (Fig 4). Building on previous work showing that speech comprehension is supported by contextual predictions [32,33], we formulated four competing hypotheses reflecting different assumptions about how linguistic context is preserved, reset, or selectively updated across an attention switch. Using a state-of-the-art large language model (LLM), we derived quantitative predictions for each hypothesis, resulting in four regressors for lexical surprisal and entropy, separately, differing in their sensitivity to prior context and to the occurrence of the switch. Encoding TRF models were then fit for each hypothesis, allowing us to compare alternative context-accumulation strategies and identify the model most consistent with the observed neural responses.

This study provides substantial new insights into the temporal unfolding and contextual mechanisms guiding attention switching, encompassing both low and high levels of speech abstraction.

Disengagement has longer temporal dynamics compared to engagement.

• (A) Left: Speech tracking encoding for an attention switch from Speaker 1 and 2. The trajectory in the panel represents our null hypothesis, where the disengagement and engagement processes progress in a symmetric manner after the switch-cue (vertical gray line). Right: Results for the neural tracking of Speaker 1 and Speaker 2 across the switching cue. EEG prediction correlations (average across all channels) obtained from a 4-s sliding-window TRF model including Envelope (Env), Word Onset (WO) and Word Surprisal (WS) features. Coloured horizontal bars at the bottom of the plot indicate the attention instruction around the attention switching cue. The turquoise dot indicates the encoding switch of EEG prediction correlations based on Spk1- and Spk2- speech features. The piecewise linear model fit for disengagement and engagement is overlayed on the EEG prediction correlation values. Please note that the broken-line-fit in this plot was performed on the grand-average cortical tracking curves here for illustrative purposes. Please find the estimates at the single-participant level in Panel C. Hexagram shapes indicate the start of the disengagement (blue) and engagement (yellow) processes, while diamonds represent the end of the transitions.
• (B) Left: Diagram of expected results for alpha-band ERSP (event-related spectral perturbation) across the switching cue. Right: ERSP of the alpha band (8-12 Hz) around the switching cue (average of all channels), computed with a 4-s sliding window.