Deep learning para reconhecimento de sinais da LIBRAS como tecnologia assistiva

Abstract

Communication is essential for social inclusion, yet the lack of linguistic accessibility often marginalizes specific groups, such as the Brazilian deaf community. This project proposes the use of deep learning to recognize signals from Brazilian Sign Language (LIBRAS) with the aim of translating them, through model predictions, into written Portuguese. Initially, videos of people signing were collected to form a dataset, which was subjected to processes of extraction and mapping of key body points using open-source tools provided by MediaPipe. The extracted data was processed and used as input for two designed models: one based on Long Short-Term Memory (LSTM) and another on Transformers. The study revealed that model performance is influenced by the alignment methods applied during data processing. The Transformer demonstrated superior results in terms of accuracy and generalization, albeit with higher computational demands. Conversely, the LSTM model showed satisfactory performance in terms of computational efficiency but exhibited limitations as classification complexity increased. One of the primary challenges was the difficulty in building a rich and robust dataset, due to the scarcity of available content for collection and extraction, especially when compared to other natural languages, whether textual or spoken. This limitation partially restricted the models' generalization capabilities. Despite these challenges, the project achieved promising results, suggesting that with enhanced and expanded datasets, its application as assistive technology can be extended to more complex scenarios with broader applicability. This study represents an advancement in the use of deep learning to promote inclusion and accessibility for the Brazilian deaf community.