Quality of frequency-following response to speech sounds linked with left prefrontal hemodynamic activity using fNIRS+EEG
At a Glance
Section titled “At a Glance”| Metadata | Details |
|---|---|
| Publication Date | 2018-01-01 |
| Journal | Frontiers in Human Neuroscience |
| Authors | Benjamin D. Zinszer, Todd A. Hay, Alex Athey, Bharath Chandrasekaran |
Abstract
Section titled “Abstract”Event Abstract Back to Event Quality of frequency-following response to speech sounds linked with left prefrontal hemodynamic activity using fNIRS+EEG Benjamin D. Zinszer1, 2*, Todd A. Hay1, 3, Alex Athey1, 3 and Bharath Chandrasekaran1, 2 1 University of Texas at Austin, Multimodal Neuroimaging Initiative, United States 2 University of Texas at Austin, Department of Communication Sciences and Disorders, United States 3 University of Texas at Austin, Applied Research Laboratories, United States Background Speech perception is a complex task requiring multiple cognitive resources to isolate signals of interest and compare them with stored linguistic representations. Consequently, many different neuroimaging technologies have been applied to study neural processing of speech sounds. In the present study, we examine different neural responses to repeated speech sounds by integrating simultaneous electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), two portable neuroimaging technologies on opposite ends of the spatio-temporal resolution spectrum. The frequency-following response (FFR) is a low-latency (approximately 10ms) electrophysiological response (measured with EEG) that phase-locks with an acoustic stimulus. Although it originates primarily in subcortical structures (inferior colliculus), the FFR is measured by a single pair of electrodes on the head, providing high temporal resolution data but no spatial localization. The FFR serves as an index of early auditory processing and has been used to study the perception of Mandarin lexical tones for several years (see Krishnan et al., 2004). Unlike the FFR, cortical hemodynamic responses are spatially localized (discernable at centimeter resolution) but unfold over an extended time window, peaking about six seconds after a speech stimulus has been perceived. Cortical responses to speech sounds measured by fNIRS can reveal important linguistic processes that provide meaning to acoustic stimuli, such as intonated Mandarin syllables (Zinszer et al., 2015). In this study, we aim to understand the relationship between the FFR/EEG and fNIRS measures. We hypothesized that because both signals are modulated by language experience, their differences across participants should correspond between imaging modalities. Method Participants. Seven participants (2 M / 5 F), native speakers of Mandarin Chinese, participated in the experiment. Participants were pre-screened for minimal musical experience, no hearing impairments, and no cognitive or neurological impairments. Procedure. The participants were exposed to 100 ms duration /i/ vowels intonated with Mandarin tones 1 (high-flat), 2 (rising), and 4 (falling), interleaved with 140-150 ms silences via insert earphones. Participants watched a silent, subtitled nature film throughout the experiment, and they were instructed to ignore the sounds. These tone stimuli were organized into Blocks that either repeated the same /i/+tone stimulus 160 times (hereafter, Repeated Blocks) or randomly inter-leaved all three variations of /i/+tone a total of 80 times (hereafter, Variable Blocks). In a single run, participants heard three Repeated Blocks (one for each Mandarin tone) and three Variable Blocks, all randomly ordered. Participants repeated as many of these 6-Block runs as possible in a one-hour window with self-paced breaks in between. Six of the participants completed between 7 and 9 runs. One participant withdrew after 3 runs due to mild discomfort. Measurements. Throughout the experiment, we measured participants’ FFRs with a single-channel frontal EEG montage (plus references at each mastoid). Changes in blood oxygenation were measured using a NIRx NIRScout system with 12 sources and 14 detectors distributed over bilateral superior temporal, inferior parietal, and inferior frontal regions. Analyses. Preprocessing steps applied to the NIRS data consisted of bandpass filtering the data between 0.005 and 0.7 Hz and converting the data to oxygenated hemoglobin (HbO) using Homer2 (Huppert et al., 2009). HbO measurements were normalized to zero mean and unit standard deviation. A perceptron model with one hidden layer was trained to discriminate between Repeated Blocks of tones 1, 2 or 4 and Variable Blocks, given only a single time sample within the first 20 seconds of NIRS measurements for a given Block (since all Block durations were >=20s). Eighty percent of the measurements were used for training with 10% reserved for validation and 10% for testing. The validation set was used to detect overfitting during the training phase. Preprocessing steps applied to the EEG measurements consisted of bandpass filtering between 110 and 160 Hz to isolate the signal closest to the stimulus and normalization to zero mean and unit standard deviation. To measure FFR quality, individual FFRs were averaged together within each Repeated Block, and cross-correlated over a +/- 20ms window with the stimulus waveform. FFR quality for each tone in each run was averaged to give an overall quality for each participant, relatively independent of the number of runs completed. Finally, we examined the relationship between FFR and fNIRS signals across participants by correlating peak FFR cross-correlation with the classification accuracy for fNIRS Blocks. Results Block-classification accuracy was 100% (chance=25%) when all fNIRS channels were used. We repeated the training and classification procedure for the perceptron model with fNIRS data from two limited subsets of channels: four channels over the left posterior temporal lobes classified Blocks with mean of 85% accuracy (sd=5%, min=76%) and four channels over the left dorsolateral prefrontal cortex classified Blocks with mean accuracy of 90% (sd=6%, min=79%). FFR quality (mean of peak cross-correlation between stimulus waveform and FFR) was 0.24 (sd=0.08, range: [0.15, 0.38]). We used Pearson correlation to compare the mean FFR quality for each participant with the fNIRS classification accuracy in the left anterior and left posterior channels for each participant. Classification of the left anterior channels correlated with FFR quality, r=0.56, although this measure did not achieve statistical significance at the current sample size (p=0.19). No correlation was found for the posterior channels (r=0.14, p=0.77). Discussion This preliminary study exploring the connection between hemodynamic measures of speech processing acquired from an fNIRS device and electrophysiological measures of early auditory processing strongly suggests that these measures capture related information about speech processing activity. Because these neuroimaging modalities each operate at different timescales (milliseconds versus several-seconds resolution) and very different spatial resolutions (whole head versus a few centimeters), FFR and fNIRS have the potential to mutually inform one another. Our ongoing work in this area includes the fusion of the EEG and fNIRS data into a single classifier for individual trials which could later provide real-time measures of speech processing to predict performance on a behavioral identification or comprehension task. Acknowledgements We are grateful to Elise LeBovidge and Jacie McHaney of the SoundBrain Lab for their assistance with data collection. References Huppert, T.J., Diamond, S.G., Franceschini, M.A., & Boas, D.A. (2009). HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Applied Optics, 48(10): D280-98. Krishnan, A., Xu, Y., Gandour, J. T., & Cariani, P. A. (2004). Human frequency-following response: representation of pitch contours in Chinese tones. Hearing Research, 189(1-2), 1-12. Zinszer, B. D., Chen, P., Wu, H., Shu, H., & Li, P. (2015). Second language experience modulates neural specialization for first language lexical tones. Journal of Neurolinguistics, 33, 50-66. Keywords: multimodal imaging methods, Speech Processing, Mandarin Chinese, lexical tone, Frequency Following Response (FFR), Electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS) Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Oral Presentation Topic: Neuroergonomics Citation: Zinszer BD, Hay TA, Athey A and Chandrasekaran B (2019). Quality of frequency-following response to speech sounds linked with left prefrontal hemodynamic activity using fNIRS+EEG. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00074 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 02 Apr 2018; Published Online: 27 Sep 2019. * Correspondence: Dr. Benjamin D Zinszer, University of Texas at Austin, Multimodal Neuroimaging Initiative, Austin, Texas, United States, [email protected] Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Benjamin D Zinszer Todd A Hay Alex Athey Bharath Chandrasekaran Google Benjamin D Zinszer Todd A Hay Alex Athey Bharath Chandrasekaran Google Scholar Benjamin D Zinszer Todd A Hay Alex Athey Bharath Chandrasekaran PubMed Benjamin D Zinszer Todd A Hay Alex Athey Bharath Chandrasekaran Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.