ATDLLMD: Acceptance test-driven LLM development
| dc.contributor.author | Faragó, David | |
| dc.contributor.editor | Herrmann, Andrea | |
| dc.date.accessioned | 2024-07-26T10:37:42Z | |
| dc.date.available | 2024-07-26T10:37:42Z | |
| dc.date.issued | 2024 | |
| dc.description.abstract | Since the capabilities of Large Language Models (LLMs) have massively increased in the last years, many new applications based on LLMs are possible. However, these new applications also pose new challenges in LLM development. This article proposes an acceptance test-driven development (ATDD) style, baptized ATDLLMD, where the LLM’s training and test sets are extended in each iteration by data coming from validation of the previous iteration’s LLM and system around the LLM. So the validation phase supplies the additional or updated data for training and verification of the LLM. ATDLLMD is made possible by two major innovative solutions: applying the innovative CPMAI process, and applying our own verification tool, LM-Eval, leading to a red-train green cycle for LLM development, which resembles ATDD, but integrates data science best practices. | en |
| dc.identifier.uri | https://dl.gi.de/handle/20.500.12116/44202 | |
| dc.language.iso | en | |
| dc.pubPlace | Bonn | |
| dc.publisher | Gesellschaft für Informatik e.V. | |
| dc.relation.ispartof | Softwaretechnik-Trends Band 44, Heft 2 | |
| dc.relation.ispartofseries | Softwaretechnik-Trends | |
| dc.subject | Large Language Model | |
| dc.subject | LLM | |
| dc.subject | development process | |
| dc.subject | test-first | |
| dc.subject | LLM evaluation | |
| dc.subject | LLM testing | |
| dc.subject | data-centric AI | |
| dc.subject | business-centric AI | |
| dc.title | ATDLLMD: Acceptance test-driven LLM development | en |
| dc.type | Text/Conference Paper | |
| mci.conference.date | 15.-16. Februar 2024 | |
| mci.conference.location | Gummersbach | |
| mci.conference.sessiontitle | Treffen der GI-Fachgruppe Test, Analyse und Verifikation von Software (TAV 49) | |
| mci.reference.pages | 13-17 |
Dateien
Originalbündel
1 - 1 von 1