NeuroSight

What is the name of your organization?

Feynovate

What is the name of your solution?

NeuroSight

Provide a one-line summary of your solution.

Portable solutions for multimedia reconstruction and translation from on-the-go neuroimaging to help paralyzed and nervous disorder patients and their caregivers.

Film your elevator pitch.

What specific problem are you solving?

My grandfather was a Parkinson's Disease patient, and when he passed away, I was in the next room unaware. It isn't uncommon for patients suffering from similar neurological disorders to be unable to communicate with their loved ones and caregivers let alone do anything themselves. Caregivers need to predict the needs and wants of paralyzed patients, which can cause inadequate burdens, complications and  uncertainties. One of the optimum solutions for this scenario is a Brain-Computer Interface (BCI) which reads the brain activity of the patient to spell out what they are thinking into words and more recently via generative models into images. Such solutions can alleviate the distress of the patients, their family, friends and caregivers exponentially. Reading the human mind has been a part of science fiction stories for many years. But, with the advent of current technology, it is not only become possible, but also necessary. Such Brain-Computer Interface (BCI) enable patients suffering from paralysis or neurological disorders having speech impediments and severe disabilities to interact directly with computing devices and in turn auxiliary output devices such as speakers, through their brain signals. However, the systems presently in use are inconveniently bulky, have a high latency between input and output, and can be limitedly operated only by professionals. Scientists are trying to find new avenues of reading brain waves for domestic applications, however current solutions are  highly surgical in nature, expensive and again limitedly operated by professionals. Successful and largely documented attempts at reconstructing text and/or images from the brainwave oscillation data using non-invasive techniques have used Functional magnetic resonance imaging (fMRI) data to facilitate their brain decoding, because of its large spatial resolution. But, fMRI imaging itself utilizes bulky instruments, which are expensive and has a substantially large processing time from input to output. Even though such attempts have been highly successful, they are limited in their capabilities to laboratory processing and obtaining results, which not feasible for individual users commercially.

What is your solution?

We are currently developing a working prototype for a ingle end-to-end architecture to examine Electroencephalography (EEG) and Magnetoencephalography (MEG) data, which is comparatively easier, cheaper and smaller but has lower spatial and temporal resolution as compared to fMRI to be applied while reconstructing multimedia. The data is collected by non-invasive sensors on the scalp of the user in a low-cost portable headset, which parses it through a deep-learning autoencoder framework stacked with a generative method, which learns a more compacted representation of EEG and MEG data with visual imagery (VI) and eliminated background activity and employs it to generate, from the semantic characteristics in the spatial features of the data with the visual stimuli to provide image reconstruction evoking from specific brain responses and attempts to reconstruct the stimulated words to mimic natural speech through open vocabulary Sequence-To-Sequence decoding and zero-shot sentence sentiment classification, allowing the decoding of phrases which are previously absent from the training set. The device is paired with a companion app on user’s smartphone for real-time processing and text or image generation for the use of their caregivers in order to better understand what are the needs of the patients, their cause of irritation and provide a means of general communication. The text generation can be, therefore, performed for any translated foreign language spoken by the user. This process can be further utilized to study subconscious responses and help in understanding conditions such as post-traumatic stress disorder (PTSD) and provide alerts during a panic attack, after training a machine learning model to classify certain trigger responses. The study also tries to provide more portable solutions to higher spatial resolution imaging techniques to provide alternatives for fMRI using fNIRS to be used in user-wearable headsets on a daily basis for the same process. The research has footing in previously proven data and currently trying to integrate previous studies with this new method. This procedure proves to better cost-effective as it leverages hardware component which are already in supply from current stakeholders in the industry, but are being used in different avenues. This innovation could further be utilized in diagnosing disorder in their early stages from the collected brainwave data even for seeming healthy individuals.

Who does your solution serve, and in what ways will the solution impact their lives?

Our solution directly serves the patients suffering from ailments such as Parkinson's, Stroke, Paralysis, Epilepsy, Palsy and other neurological disorders and at-risk people who are unable to even communicate their basic needs and necessities to their caregivers and are unable to even talk with their loved ones. The proposed solution is cost-effective in nature to produce and easy to be operated by even non-technically aligned users. The wearable device uses non-invasive sensors and imaging to collect data which is in non way proven way harmful to the users in contrast the currently developing devices. The sensors and other hardware are already in production for sectors such as entertainment by industry stakeholders and can cheaply sourced. This alleviates the disparity that would be forced upon users and patients which are not financially well-off. This technology could provide a step into a brighter and more accessible future for all by enabling any affected patient or distressed individual to avail the need and use their taken-away ability of communicating; providing a voice to the otherwise voiceless individuals. The research for this architecture has been going on for some time and has been already submitted to avenues such as S.T. Yau High School Science Award (Asia), Columbia Junior Science Journal, Stanford Intersect for Computer Science research. We are currently submitted under the Innovation Department of Government of India for an innovation and entrepreneur individual internship to develop innovations and connect with industry leaders and professional researchers for advice. This innovation could even help people in diagnosing and checking for disorder symptoms even when they are not fully formed for macroscopic detection and help in getting healthcare support earlier than dangerous cases.

How are you and your team well-positioned to deliver this solution?

I spent most of my school years taking care of my grandfather who was suffering from Parkinson's Disease and have experience firsthand the effects, restrictions and limitations of such neurological disorders not only for the patients but also for their family, friends and caregivers. During my routines I came to know about how my grandfather couldn't properly convey what he wanted to in his mind and quietly tolerated his ache. So, I am personally attached to this project and intend use all of my determination and passion in making this idea into a common reality. I am a deeply resolute person, and I fear most that after my death my life amounted to nothing substantial , so I want to leave behind a legacy of change for good and for the betterment of society as a whole. Even if one person is able to live a better life because of, my aim would come to fruition. We are currently researching and developing our architecture and consulting professional researchers. In March, during my senior final exams, I was in a road accident and broke my knee so I had to be hospitalized after giving my exams. During my stay at the hospital I got know more about the healthcare system and patients' and caregivers' workings, needs and requirements. My interactions helped develop a more deeper understanding and provide a better map of the future of this project in normal, domestic and hospital environments with experiences from actual caregivers and patients.

Which dimension of the Challenge does your solution most closely address?

• Improve the rare disease patient diagnostic journey – reducing the time, cost, resources, and duplicative travel and testing for patients and caregivers.

In what city, town, or region is your solution team headquartered?

In what country is your solution team headquartered?

• India

What is your solution’s stage of development?

Prototype: A venture or organization building and testing its product, service, or business model, but which is not yet serving anyone

Please share details about what makes your solution a Prototype rather than a Concept.

I am currently conducting and research on this very architecture, decoding and reconstructive generation. The research has been submitted at different venues and I am currently developing a working prototype to implement the findings. At the current stage, the device shows results and more usability as compared to an idea in consideration. I have worked with EEG wave testing and results on myself only, so, the results provide real-world use cases. The major component is the decoding and generative architecture which is currently under work, other than that the wearable headset and the companion app have deployable prototype designs. At the current rate and further support, I project that the device could be launched as local pilot project under the supervision of professions in upcoming months.

Why are you applying to the Prize?

I will not lie, the monetary support could prove to be extremely beneficial in development, implementation, networking and resource accumulation. It could not only help in deploying a real consumer-based product but also help in getting professional support and higher educational opportunities. But, what I am truly excited about is getting an opportunity to meet peers, professionals and network with industry experts at such a highly reputed and influential event as Concordia Annual Summit. I would be inexplicably grateful to be able to attend and connect with people through this opportunity, which itself could provide higher avenues for this project and myself to work with. An I could add credibility to this name event at such a young age for people to take this problem seriously.

Who is the Team Lead for your solution?

Aditya Chaudhary

How is your Team Lead connected to the community or communities in which your project is based?

Aditya just graduated from his studies as a science student with Computer Science major in a public school. The opportunities and resources are low here for students, as the student body of the nation as a whole is mostly focused on clearing exams to get into a college, which hinders the development of ideas outside of the curriculum, or even the resources to actualize them. Hence, when it comes to prototyping, we need to think outside of the box and utilize the means at our disposal to the best extent, which has led us to some exciting and bizarre scenarios as well. This didn't stop us from losing hope altogether, but slowly build towards realizing our goals. He was also the president of our school's technology club and regularly worked with and mentored other students in the school's innovation society. He also takes up volunteering lectures in basic physics and engineering and aeronautics in partnership with local students of other schools who come from economically weaker sections. This has led him to work and socialize with bright young minds and sharing of stories, problems and ideas. His another idea has been submitted under the innovation department of government of India for an incubation internship and to work with industry professionals for further research and development of innovative ideas.