Logo des Repositoriums

Auflistung nach Schlagwort "LLM"

1 - 5 von 5
  • Konferenzbeitrag
    ATDLLMD: Acceptance test-driven LLM development
    (Softwaretechnik-Trends Band 44, Heft 2, 2024) Faragó, David
    Since the capabilities of Large Language Models (LLMs) have massively increased in the last years, many new applications based on LLMs are possible. However, these new applications also pose new challenges in LLM development. This article proposes an acceptance test-driven development (ATDD) style, baptized ATDLLMD, where the LLM’s training and test sets are extended in each iteration by data coming from validation of the previous iteration’s LLM and system around the LLM. So the validation phase supplies the additional or updated data for training and verification of the LLM. ATDLLMD is made possible by two major innovative solutions: applying the innovative CPMAI process, and applying our own verification tool, LM-Eval, leading to a red-train green cycle for LLM development, which resembles ATDD, but integrates data science best practices.
  • Konferenzbeitrag
    Case study: Using LLMs to assist with solving programming homework assignments
    (Proceedings of DELFI Workshops 2024, 2024) Deriyeva, Alina; Dannath, Jesper; Paaßen, Benjamin
    Nowadays, students have the option of using LLMs for assistance in solving homework assignments. Moreover, most LLMs, like ChatGPT, are also trained on large sets of source code and thus can be used to assist in programming exercises. In this paper, we present a case study based on data collected over the course of 1.5 semesters, where students of three programming-related courses were explicitly permitted to use such models while solving homework assignments. In a qualitative evaluation, we observe that there might be a difference between targeted requests for an answer to specific questions and requests for a complete solution from an LLM. Particularly, targeted requests might be pedagogical feasible and enhance the learning experience. Additionally, we discuss the potential of LLM applications in programming education, with a focus on the intermediate level and beyond.
  • Konferenzbeitrag
    Enhancing Chatbot-Assisted Study Program Orientation
    (Proceedings of DELFI Workshops 2024, 2024) Dieing, Thilo I.; Scheffler, Marc; Cohausz, Lea
    As university dropout rates increase, implementing innovative solutions is crucial to reduce attrition. Aligning students’ interests with their study programs enhances academic success, satisfaction, and retention. This paper presents a novel approach using open-source Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) to develop a semi-open-domain knowledge chatbot. The chatbot generates informed responses and recommendations to diverse student queries by retrieving relevant data while maintaining ethical standards and avoiding biased responses. When testing five model combinations on 70 prompts partially from real study advisors, results demonstrate that the RAG approach with the Mixtral LLM and RoBERTa embedding model offers superior performance. Our method for handling critical user prompts further indicates a significantly improved response quality. These findings advance service-oriented chatbots in education, aiming to reduce student attrition through accurate and helpful program recommendations.
  • Konferenzbeitrag
    Large Language Models are Pattern Matchers: Editing Semi-Structured and Structured Documents with ChatGPT
    (AKWI Jahrestagung 2024, 2024) Weber, Irene
    Abstract: Large Language Models (LLMs) offer numerous applications, the full extent of which is not yet understood. This paper investigates if LLMs can be applied for editing structured and semi-structured documents with minimal effort. Using a qualitative research approach, we conduct two case studies with ChatGPT and thoroughly analyze the results. Our experiments indicate that LLMs can effectively edit structured and semi-structured documents when provided with basic, straightforward prompts. ChatGPT demonstrates a strong ability to recognize and process the structure of annotated documents. This suggests that explicitly structuring tasks and data in prompts might enhance an LLM’s ability to understand and solve tasks. Furthermore, the experiments also reveal impressive pattern matching skills in ChatGPT. This observation deserves further investigation, as it may contribute to understanding the processes leading to hallucinations in LLMs.
  • Workshopbeitrag
    Limitations of ChatGPT in Conceptual Modeling: Insights from Experiments in Metamodeling
    (Modellierung 2024 Satellite Events, 2024) Muff, Fabian; Fill, Hans-Georg