Knierim, PascalHein, DimitriSchmidt, AlbrechtKosch, Thomas2021-05-132021-05-132021https://dl.gi.de/handle/20.500.12116/36431Current interaction modalities for mobile Augmented Reality (AR) are tedious and lack expressiveness. To overcome these prevalent limitations, we developed and evaluated a multimodal interaction concept by pairing a smartphone as an input and output modality for mobile AR. In a user study (n = 24), we investigated the effects on interaction speed, accuracy, and task load for (1) virtual object manipulation as well as (2) interaction with established graphical user interfaces (GUIs). Our findings show that a smartphone-based AR controller results in significantly faster and more accurate object manipulation with reduced task load than state-of-art mid-air gestures. Our results also indicate a significant enhancement for using the physical touchscreen as a modality compared to mid-air gestures for GUI interaction. We conclude that interaction in mobile AR environments can be improved by utilizing a smartphone as an omnipresent controller. Additionally, we discuss how future AR systems can benefit from tangible touchscreens as an additional and complementary interaction modality.enAugmented RealityHead-Mounted DisplaySmartphoneObject ManipulationMulti-Display InteractionText/Journal Article10.1515/icom-2021-00032196-6826