Uncovering Patterns of Brain Activity from EEG Data Consistently Associated with Cybersickness Using Neural Network Interpretability Maps

Cybersickness poses a serious challenge for users of virtual reality (VR) technology. Consequently, there has been significant effort to track its occurrence during VR use with brain activity through electroencephalography (EEG). However, a significant confound in current methods for detecting sickness from EEG is they do not account for the simultaneous processing of the sickening visual stimulus that is present in the brain data from VR. Using event-related potentials (ERPs) from an auditory stimulus shown to reflect cybersickness impacts, we can more precisely target EEG cybersickness features and use those to achieve better performance in online cybersickness classification. In this article, we introduce a method utilizing trained convolutional neural networks and transformer models and plot interpretability maps from integrated gradients and class activation to give a visual representation of what the model determined was most useful in sickness classification from an EEG dataset consisting of ERPs recorded during the elicitation of cybersickness. Across 12 runs of our method with three different neural networks, the models consistently pointed to a surprising finding: that amplitudes recorded at an electrode placed on the scalp near the left prefrontal cortex were important in the classification of cybersickness. These results help clarify a hidden pattern in other related research and point to exciting opportunities for future investigation: that this scalp location could be used as a tagged feature for better real-time cybersickness classification with EEG. We provide our code at: [anonymized].