Deep auditory encoding with self-attention to predict brain activity
This Python package allows you to train a recurrent DNN (a GRU) and learn a self-attention mechanism that weighs hidden states.
This Python package allows you to train a recurrent DNN (a GRU) and learn a self-attention mechanism that weighs hidden states.
This was my first Kaggle challenge that I took seriously and I managed to get into the top 3% (119th place out of 4550 teams) - nothing to brag about, but a great learning experience nonetheless. You can find my solution here.
Do you have a BIDS-compliant dataset and stimulus and want to train and validate voxel-wise encoding models in a reproducible workflow? Using this BIDS app, this is as easy as writing a single line in your terminal.
Published in Frontiers in psychology, 7, 755, 2016
We use Hierarchical Bayesian Modelling to compare models of decision making under uncertainty with varying degrees of inter-individual differences.
Recommended citation: Boos, M., Seer, C., Lange, F., & Kopp, B. (2016). Probabilistic inference: task dependency and individual differences of probability weighting revealed by hierarchical Bayesian modeling. Frontiers in psychology, 7, 755. https://www.frontiersin.org/articles/10.3389/fpsyg.2016.00755/full
Published in bioRxiv, 2020
In neuroimaging, voxel-wise encoding models are a popular tool to predict brain activity elicited by a stimulus. To evaluate the accuracy of these predictions across multiple voxels, one can choose between multiple quality metrics. However, each quality metric requires specifying auxiliary parameters such as the number and selection criteria of voxels, whose influence on model validation is unknown. In this study, we systematically vary these parameters and observe their effects on three common quality metrics of voxel-wise encoding models in two open datasets of 3- and 7-Tesla BOLD fMRI activity elicited by musical stimuli. We show that such auxiliary parameters not only exert substantial influence on model validation, but also differ in how they affect each quality metric. Finally, we give several recommendations for validating voxel-wise encoding models that may limit variability due to different numbers of voxels, voxel selection criteria, and magnetic field strengths. All code used in the analysis is available at https://github.com/psychoinformatics-de/studyforrest-paper-pandoraencoding
Recommended citation: Boos, M., Guntupalli, J. S., Rieger, J. W., & Hanke, M. (2020). The role of auxilliary parameters in evaluating voxel-wise encoding models for 3T and 7T BOLD fMRI data. bioRxiv https://www.biorxiv.org/content/10.1101/2020.04.07.029397v2
Published in Neuroimage, 2021
Speech comprehension in natural soundscapes rests on the ability of the auditory system to extract speech information from a complex acoustic signal with overlapping contributions from many sound sources. Here we reveal the canonical processing of speech in natural soundscapes on multiple scales by using data-driven modeling approaches to characterize sounds to analyze ultra high field fMRI recorded while participants listened to the audio soundtrack of a movie. We show that at the functional level the neuronal processing of speech in natural soundscapes can be surprisingly low dimensional in the human cortex, highlighting the functional efficiency of the auditory system for a seemingly complex task. Particularly, we find that a model comprising three functional dimensions of auditory processing in the temporal lobes is shared across participants’ fMRI activity. We further demonstrate that the three functional dimensions are implemented in anatomically overlapping networks that process different aspects of speech in natural soundscapes. One is most sensitive to complex auditory features present in speech, another to complex auditory features and fast temporal modulations, that are not specific to speech, and one codes mainly sound level. These results were derived with few a-priori assumptions and provide a detailed and computationally reproducible account of the cortical activity in the temporal lobe elicited by the processing of speech in natural soundscapes.
Recommended citation: Boos, M., Lücke, J., & Rieger, J. W. (2021). Generalizable dimensions of human cortical auditory processing of speech in natural soundscapes: A data-driven ultra high field fMRI approach. NeuroImage, 237, 118106. https://www.sciencedirect.com/science/article/pii/S1053811921003839