Portuguese PI – Sofia Cavaco (NOVA.ID.FCT; FCT/UNL)
CMU PI – Maxine Eskenazi (LTI)
Research teams: Carnegie Mellon University (CMU); Associação para a Inovação e Desenvolvimento da FCT (NOVA.ID.FCT); Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa (INESC ID)
Organizations: Centro Hospitalar de Lisboa Central, EPE (CHLC); Escola Superior de Saúde de Alcoitão (ESSA); Hospital Dona Estefânia (HDE); Santa Casa da Misericórdia de Lisboa (SCML); Voice Interaction
Main Research Unit: Associação para a Inovação e Desenvolvimento da FCT (NOVA.ID.FCT)
Funding Reference: CMUP-ERI/TIC/0033/2014
Duration: 48 months
Keywords: Speech therapy; interactive environment; multimedia analysis; visual bio-feedback
“This Entrepreneurial Research Initiative, BioVisualSpeech, proposes to research natural and multimodal human computer interaction mechanisms for providing bio-feedback in speech and language therapy through the use of serious digital games. To this end we propose to develop a platform with tools for the speech and language therapist and patient that combines visual feedback, the gamification of the speech-language therapy and multimodal analysis of the patient’s speech and orofacial expressions. The major outcome of the project is aligned with our previous work towards the use of digital games, multimedia and speech recognition to make a real change in the life of groups with special needs, enhancing their better integration in society.”
Sofia Cavaco and Maxine Eskenazi
This Entrepreneurial Research Initiative proposes to research natural and multimodal human computer interaction mechanisms for providing bio-feedback in speech and language therapy through the use of serious digital games. Traditional behavioural treatment approaches for speech and oral motor disorders, can benefit greatly from visual biofeedback for patient motivation purposes, for tracking orofacial motion and for evaluation of the patients speech productions. Existing bio-feedback mechanisms such as mirrors are commonly used in speech-language therapy. These instruments are useful in the sense that patients can watch themselves as they pronounce the sound or do an orofacial movement or activity, and this way better associate their intention to their actions. This project proposes a novel approach to include biofeedback in speech-language therapy by providing the patient with a visual self-monitoring of his/her performance combined with a reward mechanism. This approach offers significant benefits to the patient as well as for the speech and language therapist (SLT). Our approach provides the SLT with a tool to gather and evaluate the patient’s speech progress, and to measure the coordination of the patient facial muscles in terms of orofacial motion.
