CS Events

PhD Defense

Brain-informed Deep Learning of Human Movements with Neurophysiological Interpretations


Download as iCal file

Wednesday, May 25, 2022, 01:00pm - 03:00pm


Speaker: Neelesh Kumar

Location : Via Zoom


Dr. Konstantinos Michmizos (Chair)

Dr. Dimitris N. Metaxas

Dr. Vladimir Pavlovic

Dr. Georgios D. Mitsis (McGill University)

Event Type: PhD Defense

Abstract: The accurate and reliable decoding of movement from non-invasive electroencephalography (EEG) is essential for informing therapeutic interventions ranging from neurorehabilitation robots to neural prosthetics. However, the main caveats of EEG, namely its low spatial resolution and ill-defined source localization, hinder its reliable decoding, despite progress in both statistical and the recent machine learning methods. In this talk, I will present our brain-informed deep learning solutions for the accurate and reliable decoding of movements from EEG with neurophysiological interpretations. First, I will present a 3-dimensional convolutional neural network (3D-CNN) that, when designed and trained upon the brain's constraints, can decode EEG to predict the movement primitives, namely the reaction time (RT), movement intent, and the direction of the movement. When validated on data acquired from in-house IRB-approved motor experiments, our proposed method outperforms the state-of-the-art deep networks by up to 6.74%. Next, I will show how the deep learning framework can be extended to assess the cognitive engagement (CE) of the subjects when performing a rehabilitation task. Specifically, our method uses i) a deep network that predicts the level of CE for two classes- cognitively engaged vs. disengaged with 88.13% accuracy; and ii) a novel sliding window method that predicts continuous levels of CE in real-time. Next, to widen the application domain of our method to portable brain-computer interfaces, I will present an energy-efficient neuromorphic solution to EEG decoding. Our neuromorphic solution, where a trained spiking neural network (SNN) is deployed on Intel's Loihi neuromorphic chip, achieves the same level of performance as the deep neural networks (DNN) while consuming 95% less energy per inference. Lastly, to guarantee the reliability of our solutions in real-world applications, I will present interpretation techniques that establish correspondence between the features learned by the artificial networks and the underlying neurophysiology. Overall, our approach demonstrates the importance of biological relevance in neural networks for accurate and reliable decoding of EEG, suggesting that the real-time classification of other complex brain activities may now be within our reach.


Rutgers University School of Arts and Sciences

Contact  Konstantinos Michmizos

Webex Link: