The Agile‑AI project, funded under grant 01 IS 19059A and carried out by the Bauhaus University Weimar from 1 November 2019 to 30 November 2022, aimed to bring agile software development practices to the creation of artificial‑intelligence systems, with a particular focus on the iterative design and execution of AI experiments. The project was divided into three interrelated sub‑projects: the Experiment Specification Language (ESL), the Experiment Execution Platform (EEP), and the Experiment Retrieval Engine (ERE). This article concentrates on the technical achievements of the ERE sub‑project, while also summarising the collaborative framework that enabled the work.
The core technical contribution of ERE was the design and implementation of a retrieval engine that can search, filter, and analyse both historical and ongoing AI experiment series. The engine incorporates a specialised query language that allows researchers to specify search criteria based on input data characteristics, experiment metadata, and the quality of results produced. To support efficient retrieval, the team developed novel index structures tailored to the high‑dimensional and heterogeneous nature of experiment artefacts. An evaluation framework was also introduced, enabling systematic assessment of retrieval effectiveness and the impact of different indexing strategies. Although the report does not publish explicit performance figures, it indicates that the engine was benchmarked against a representative set of experiment collections, demonstrating significant improvements in retrieval speed and relevance compared to generic search systems. The evaluation also highlighted the engine’s ability to handle incremental updates, a key requirement for continuous experiment pipelines.
In addition to the retrieval engine itself, the ERE sub‑project produced a set of reusable components that integrate with the broader Agile‑AI ecosystem. These components expose APIs for experiment metadata ingestion, result quality scoring, and real‑time query execution, thereby facilitating seamless interaction with the ESL and EEP modules. The modular design ensures that the retrieval engine can be adopted independently by other research groups or industrial partners interested in experiment management and reproducibility.
The collaborative dimension of the project was anchored by the Bauhaus University Weimar, which served as the principal investigator and project coordinator. The project was part of a larger consortium that included several research institutions and industry partners, although the report does not enumerate all collaborators. Roles within the consortium were distributed along the three sub‑projects: the ESL team focused on language design and compiler construction, the EEP team developed distributed execution and scaling mechanisms for cluster environments, and the ERE team concentrated on retrieval algorithms and evaluation. The project’s timeline was structured into iterative sprints, mirroring agile practices, with regular cross‑team reviews to ensure alignment and knowledge transfer.
Funding for the project was provided by the German research council, as indicated by the grant identifier. The allocation covered personnel costs, computational resources, and dissemination activities, including conference presentations and journal publications. The project’s outcomes are intended to be open‑source, promoting reproducibility and fostering a community of practice around agile AI development. By delivering a specialised retrieval engine and integrating it into a cohesive agile framework, the Agile‑AI initiative has advanced the state of the art in experiment management and set a foundation for future research in reproducible AI systems.