Analysis and evaluation of experimentation management tools applied to multi-model machine learning in autonomous driving systems

  • Autonomous Driving Systems (ADS) are usually complicated large scale systems and contain several nontrivial interconnected ML models. They are part of the bigger category of Multi Machine Learning Model Systems, which are software systems that rely on more than one ML model. Because the systems share their environment with humans, the application of ADS is very volatile. The problem is that an environment under the influence of humans is not easy to predict. Resulting safety concerns are cause to fulfill certain standards and expectations toward quality that also affect the development phase of the system. To fulfill all the safety aspects, every part of the development process of ADS needs to be evaluated. While there is already research on the topic of testing ADS, we want to take a closer look at experimentation management in ADS with multiple machine learning models. We especially want to investigate whether current methods and tools are feasible for such systems and discover if there is potential for future research. This thesis is an exploratory case study on a small scale subject system that was previously developed as a prototype for a Formula Student race car, the KoopaCar. In this thesis, we want to explore how experimentation management tools can support the development of Multi Machine Learning Model Systems, with a focus on ADS. We want to especially discover how the requirements for experimentation management can differ, and if the requirements of the management of multiple machine learning models are supported by experimentation management tools.

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar
Author:Henriette KnoppGND
Referee:Thorsten BergerGND, Sven PeldszusORCiDGND
Document Type:Bachelor Thesis
Date of Publication (online):2024/02/22
Date of first Publication:2024/02/22
Publishing Institution:Ruhr-Universität Bochum, Universitätsbibliothek
Granting Institution:Ruhr-Universität Bochum, Fakultät für Informatik
Date of final exam:2023/04/17
Creating Corporation:Fakultät für Informatik
Tag:Asset Management; Autonomous Driving Systems; Experiment Managment; Machine Learning
Institutes/Facilities:Lehrstuhl für Software Engineering
Dewey Decimal Classification:Allgemeines, Informatik, Informationswissenschaft / Informatik
faculties:Fakultät für Informatik
Licence (English):License LogoCreative Commons - CC BY-NC 4.0 - Attribution-NonCommercial 4.0 International