Logo des Repositoriums
 
Workshopbeitrag

Towards natural language understanding for intuitive interactions in XR using large language models

Vorschaubild nicht verfügbar

Volltext URI

Dokumententyp

Text/Workshop Paper

Zusatzinformation

Datum

2024

Zeitschriftentitel

ISSN der Zeitschrift

Bandtitel

Verlag

Gesellschaft für Informatik e.V.

Zusammenfassung

This paper presents a voice assistance system for extended reality (XR) applications based on large language models (LLMs). The aim is to create an intuitive and natural interface between users and virtual environments that goes beyond traditional, predefined voice commands. An architecture is presented that integrates LLMs as embodied agents in XR environments and utilizes their natural language understanding and contextual reasoning capabilities. The system interprets complex spatial instructions and translates them into concrete actions in the virtual environment. The performance of the system is evaluated in XR scenarios including object manipulation, navigation and complex spatial transformations. The results show promising performance in simple tasks, but also reveal challenges in processing complex spatial concepts. This work con- tributes to the improvement of user interaction in XR environments and opens up new possibilities for the integration of LLMs in XR environments.

Beschreibung

Skyba, Kevin; Pfeiffer, Thies (2024): Towards natural language understanding for intuitive interactions in XR using large language models. GI VR / AR Workshop. DOI: 10.18420/vrar2024_0021. Gesellschaft für Informatik e.V.. 17. - 18. September 2024

Schlagwörter

Zitierform

Tags