Logo des Repositoriums
 

Towards natural language understanding for intuitive interactions in XR using large language models

dc.contributor.authorSkyba, Kevin
dc.contributor.authorPfeiffer, Thies
dc.date.accessioned2024-09-03T13:00:44Z
dc.date.available2024-09-03T13:00:44Z
dc.date.issued2024
dc.description.abstractThis paper presents a voice assistance system for extended reality (XR) applications based on large language models (LLMs). The aim is to create an intuitive and natural interface between users and virtual environments that goes beyond traditional, predefined voice commands. An architecture is presented that integrates LLMs as embodied agents in XR environments and utilizes their natural language understanding and contextual reasoning capabilities. The system interprets complex spatial instructions and translates them into concrete actions in the virtual environment. The performance of the system is evaluated in XR scenarios including object manipulation, navigation and complex spatial transformations. The results show promising performance in simple tasks, but also reveal challenges in processing complex spatial concepts. This work con- tributes to the improvement of user interaction in XR environments and opens up new possibilities for the integration of LLMs in XR environments.en
dc.identifier.doi10.18420/vrar2024_0021
dc.identifier.urihttps://dl.gi.de/handle/20.500.12116/44480
dc.language.isoen
dc.publisherGesellschaft für Informatik e.V.
dc.relation.ispartofGI VR / AR Workshop
dc.titleTowards natural language understanding for intuitive interactions in XR using large language modelsen
dc.typeText/Workshop Paper
gi.conference.date17. - 18. September 2024

Dateien

Originalbündel
1 - 1 von 1
Vorschaubild nicht verfügbar
Name:
GI_VRAR_2024_paper_21.pdf
Größe:
1.34 MB
Format:
Adobe Portable Document Format