An Evaluation of Smartphone-Based Interaction in AR for Constrained Object Manipulation

Kristoffer Waldow, Martin Misiak, Ursula Derichs, Olaf Clausen und Arnulph Fuhrmann.

In: Proceedings of the 24th ACM Symposium of Virtual Reality Software and Technology (VRST), Tokio, Japan 2018


In Augmented Reality, interaction with the environment can be achieved with a number of different approaches. In current systems, the most common are hand and gesture inputs. However experimental applications also integrated smartphones as intuitive interaction devices and demonstrated great potential for different tasks. One particular task is constrained object manipulation, for which we conducted a user study. In it we compared standard gesture-based approaches with a touch-based interaction via smartphone. We found that a touch-based interface is significantly more efficient, although gestures are being subjectively more accepted. From these results we draw conclusions on how smartphones can be used to realize modern interfaces in AR.


Preprint: PDF

DOI: 10.1145/3281505.3281608


The three different evaluated interation methods in AR. Left: A transformation method with gaze, gestures and a bounding box with virtual handles. The user can move the hand vertically to scale and horizontally to rotate the object. Middle: Similar to the variant on the left, except there is no gaze selection. The transform selection is directly identified by the users’ gestures once their hand starts moving. Right: Smartphone as touch-input for the transformation. By swiping vertically or horizontally on the display, the object is scaled or rotated.