Enhancing Human-in-the-Loop Adaptive Systems through Digital Twins and VR Interfaces

Abstract

This work has been published as a full paper at SEAMS'21. In the context of self-adaptive systems, there are situations where human involvement in the adaptation process is beneficial or even necessary. For such ''human-in-the-loop'' adaptive systems, two major challenges, namely transparency, and controllability must be addressed to include the human in the self-adaptation loop. Transparency covers the context information about the adaptive system and its context while controllability targets the decision-making and adaptation operations. As existing human-in-the-loop adaptation approaches do not fully cover these aspects, we investigate alternative human-in-the-loop strategies by using a combination of digital twins and virtual reality (VR) interfaces. Based on the concept of the digital twin, we represent a self-adaptive system and its respective context in a virtual environment. For integrating the human in the decision-making and adaptation process, we have implemented and analyzed two different human-in-the-loop strategies in VR: a procedural control where the human can control the decision making-process and adaptations through VR interactions and a declarative control where the human specifies the goal state and the configuration is delegated to an AI planner. We evaluate our approach based on an autonomic robot system that is accessible through a VR interface.