Logo des Repositoriums
 

Assessing Large Language Models in the Agricultural Sector: A Comprehensive Analysis Utilizing a Novel Synthetic Benchmark Dataset

Kurzanzeige
dc.contributor.authorKästing, Marvin
dc.contributor.authorHänig, Christian
dc.contributor.editorKlein, Maike
dc.contributor.editorKrupka, Daniel
dc.contributor.editorWinter, Cornelia
dc.contributor.editorGergeleit, Martin
dc.contributor.editorMartin, Ludger
dc.date.accessioned2024-10-21T18:24:12Z
dc.date.available2024-10-21T18:24:12Z
dc.date.issued2024
dc.description.abstractThis paper provides a comprehensive study of Large Language Models (LLMs) for question-answering and information retrieval tasks within the agricultural domain. We introduce the novel benchmark dataset BVL QA Corpus 2024 specifically designed to thoroughly evaluate both commercial and non-commercial LLMs in agricultural contexts. Using LLMs, we generate question-answer pairs from paragraphs extracted from domain-specific agricultural documents. Leveraging this newly developed benchmark dataset, we assess a selection of LLMs using standard metrics. Additionally, we develop a prototype Retrieval-Augmented Generation (RAG) system tailored to the agricultural sector. This system is then compared to baseline evaluations to determine the degree of alignment between actual performance and initial upper limit estimations. Our empirical analysis demonstrates that RAG systems outperform baseline LLMs across all metrics.en
dc.identifier.doi10.18420/inf2024_113
dc.identifier.isbn978-3-88579-746-3
dc.identifier.pissn1617-5468
dc.identifier.urihttps://dl.gi.de/handle/20.500.12116/45085
dc.language.isoen
dc.publisherGesellschaft für Informatik e.V.
dc.relation.ispartofINFORMATIK 2024
dc.relation.ispartofseriesLecture Notes in Informatics (LNI) - Proceedings, Volume P-352
dc.subjectLarge Language Models
dc.subjectRetrieval-Augmented Generation
dc.subjectAgricultural Information Retrieval
dc.subjectBenchmark Dataset
dc.titleAssessing Large Language Models in the Agricultural Sector: A Comprehensive Analysis Utilizing a Novel Synthetic Benchmark Dataseten
dc.typeText/Conference Paper
gi.citation.endPage1286
gi.citation.publisherPlaceBonn
gi.citation.startPage1279
gi.conference.date24.-26. September 2024
gi.conference.locationWiesbaden
gi.conference.sessiontitleKoLaZ-24-Kolloquium Landwirtschaft der Zukunft 2024: Digitale Souveränität in der Landwirtschaft, der Lebensmittelkette und dem ländlichen Raum: Trotz, mit oder durch KI?

Dateien

Originalbündel
1 - 1 von 1
Vorschaubild
Name:
Kaesting_Haenig_Assessing_Large_Language_Models.pdf
Größe:
420.44 KB
Format:
Adobe Portable Document Format
Herunterladen

Sammlungen

P352 - INFORMATIK 2024 - Lock in or log out? Wie digitale Souveränität gelingt