Detecção de posicionamento do cidadão em Projetos de Lei

Abstract

Background: Comments on political projects on the internet reflect the aspirations of a significant portion of the population. The automatic stance detection of these comments regarding specific topics can help better understand public opinion. This study aims to develop a computational model with supervised learning capable of estimating the stance of comments on legislative propositions, considering the challenge of diversity and the constant emergence of new bills. Method: For the domain studied, a specific corpus was constructed by collecting comments from surveys available on the Chamber of Deputies website. The experiments included the evaluation of classic machine learning models, such as Logistic Regression, Naive Bayes, Support Vector Machine, Random Forest, and Multilayer Perceptron, in addition to the fine-tuning of BERT language models. Automatic data annotation was also performed using the zero-shot approach based on prompts from the generative GPT-3.5 model, aiming to overcome the difficulties related to human annotation and the scarcity of annotated data, generating approximately three times the size of the manually annotated corpus. Results: The results indicate that the adjusted BERTimbau model surpassed the classic approaches, achieving an average F1- score of 70.4% on unseen topics. Moreover, the application of automatically annotated data in the initial stage of BERTimbau fine-tuning resulted in performance improvement, reaching an F1-score of 73.3%. The results present deep learning models as options with positive performance for the task under the conditions of this domain. Conclusion: It was observed that the ability to generate contextualized representations, along with the number of topics and comments trained, can directly interfere with performance. This makes automatic annotation and the exploration of topic diversity with Transformer architectures, promising approaches for the task