Executive Summary : | The World Health Organization (WHO) has identified neurological problems as one of the three long-term effects of Covid-19, along with arrhythmia and depression. A significant number of patients with neurodegenerative diseases, such as brain injury, stroke, and Parkinson's disease, have speech impairment due to nerve damage. The objective is to convert these thoughts to spoken words to improve their quality of life. To achieve this, a database of spoken words and their corresponding EEG data must be collected from normal people, partial and fully speech-impaired individuals. Experiments include building a 1D-Convolutional Neural Network classifier using only EEGthink, building a 2d-CNN classifier using an effective nonlinear representation of EEGthink, using EEGthink and a Transformer-based classifier, and building two sequence-to-sequence models, seq2seq-1 and seq2seq-2, that learn mappings between EEGthink and EEGarticulate. The combined contextual embedding from these models, [CV1:CV2,] is used as input to a DNN classifier that outputs the class labels corresponding to the word. A simple look-up table can then map these labels to spoken words, which become the final output. For partial speech loss patients, a BCI system is built with transfer learning using seq2seq-1 and seq2seq-2 models. For fully speech impaired patients, a separate seq2seq model is trained to obtain a hidden vector embedding, HV, which is then used to find the word. A hardware prototype of this system is built, aiming to improve the quality of life for millions of old age patients and young ones suffering from partial or full speech impairment due to various neurological problems. |
Co-PI: | Dr. Mirnalinee TT, ssN College Of Engineering, Kalavakkam, Tamil Nadu-603110, Dr. Nayeemulla Khan, Vellore Institute Of Technology (VIT) Chennai, Tamil Nadu-600127, Dr. Devaprasad Markandeyan, Chettinad Academy Of Research And Education, Chennai, Tamil Nadu-603103 |