Sistema de comunicação alternativa para pessoas com distúrbios neuromotores severos usando redes neurais artificiais

Abstract

Communication difficulties are frequent for many people with severe motor disabilities, making it difficult for them to interact with their families, caregivers and society in general. Augmentative and Alternative Communication (AAC) then aims to compensate for the communication deficit of these people, providing the individual with a better quality of life. However, these individuals with severe neuromotor disorders who have severe movement restrictions find great challenges in the use of several current assistive technologies. In this context, the objective of this research is to present an Alternative Communication System based on Artificial Neural Networks with a user-centered approach and their needs, for use by this public. The input and signal processing are carried out by reading facial landmark points, using the MediaPipe FaceMesh library. The development of the gesture/facial expression classifier is performed through the implementation and comparison of two different models: a Convolutional Neural Network (CNN) model and a Recurrent Neural Network model using Long Short-Term Memory (LSTM) units and dense layers. Dynamic challenges were implemented to conduct a more in-depth analysis of the models’ performance in various contexts, varying parameters such as the quantity of samples and the inclusion of similar gestures. Real-time overall results indicate a consistent performance of the proposed system, suggesting that, in both approaches, the Convolutional Neural Network (CNN) stands out significantly compared to the Long Short-Term Memory Recurrent Neural Network (LSTM) in gesture recognition.