Requirements engineering for machine learning-based AI systems: a tertiary study

Abstract

Context: In the last decade, machine learning (ML) components have become more and more present in contemporary software systems. A number of secondary literature studies reports challenges impacting on the development of ML-based systems, including those for requirements engineering (RE) activities. Motivation/Problem: Synthesizing secondary literature contributes to building knowledge and reaching conclusions about the existing RE approaches for ML-based systems (RE4ML), besides the novelty of a tertiary study on that subject. Objective: Through a tertiary study protocol we elaborated on, this paper synthesizes the body of evidence present in secondary studies on RE4ML systems. Method: We followed well-accepted guidelines about tertiary study protocols, including automatic search, the snowballing technique, selection and quality criteria, and data extraction and synthesis. Results: Nine secondary studies on RE4ML systems were aligned to our tertiary study's goal. We extracted and summarized the requirements elicitation, analysis, specification, validation, and management techniques for ML-based systems as well as the great challenges identified. Finally, we contribute with a nine-item research agenda to direct current and future searches to fill the gaps found. Conclusions: We conclude that RE has not been left aside in ML research, however, there are still challenges to be overcome, such as dealing with non-functional requirements, collaboration between stakeholders, and research in an industrial environment.