Towards natural language understanding for intuitive interactions in XR using large language models
| dc.contributor.author | Skyba, Kevin | |
| dc.contributor.author | Pfeiffer, Thies | |
| dc.date.accessioned | 2024-09-03T13:00:44Z | |
| dc.date.available | 2024-09-03T13:00:44Z | |
| dc.date.issued | 2024 | |
| dc.description.abstract | This paper presents a voice assistance system for extended reality (XR) applications based on large language models (LLMs). The aim is to create an intuitive and natural interface between users and virtual environments that goes beyond traditional, predefined voice commands. An architecture is presented that integrates LLMs as embodied agents in XR environments and utilizes their natural language understanding and contextual reasoning capabilities. The system interprets complex spatial instructions and translates them into concrete actions in the virtual environment. The performance of the system is evaluated in XR scenarios including object manipulation, navigation and complex spatial transformations. The results show promising performance in simple tasks, but also reveal challenges in processing complex spatial concepts. This work con- tributes to the improvement of user interaction in XR environments and opens up new possibilities for the integration of LLMs in XR environments. | en |
| dc.identifier.doi | 10.18420/vrar2024_0021 | |
| dc.identifier.uri | https://dl.gi.de/handle/20.500.12116/44480 | |
| dc.language.iso | en | |
| dc.publisher | Gesellschaft für Informatik e.V. | |
| dc.relation.ispartof | GI VR / AR Workshop | |
| dc.title | Towards natural language understanding for intuitive interactions in XR using large language models | en |
| dc.type | Text/Workshop Paper | |
| gi.conference.date | 17. - 18. September 2024 |
Dateien
Originalbündel
1 - 1 von 1
Vorschaubild nicht verfügbar
- Name:
- GI_VRAR_2024_paper_21.pdf
- Größe:
- 1.34 MB
- Format:
- Adobe Portable Document Format