Skip to content

MirrorTouch: Combining Touch and Mid-air Gestures for Public Displays

Author: Müller, J., Bailly, G., Bossuyt, T., & Hillgren, N.
Published in: ACM MobileHCI
Year: 2014
Type: Academic articles

In this paper we present a series of three field studies on the integration of multiple modalities (touch and mid-air gestures) in a public display. We analyze our field studies using Conversion Diagrams, an approach to model and evaluate usage of multimodal public displays. Conversion diagrams highlight the transitions inherent in a multimodal system and provide a systematic approach to investigate which factors affect them and how. We present a semi-automatic annotation technique to obtain Conversion Diagrams. We use Conversion Diagrams to evaluate interaction in the three field studies. We found that 1) clear affordances for touch were necessary when mid-air gestures were present. A call-to-action caused significantly more users to touch than a button (+200%), 2) the order of modality usage was different from what we designed for, and the location impacted which modality was used first, and 3) small variations in the application did lead to considerable user increase (+290%).

Download Publication


Connected HIIG researchers

Jörg Müller, Prof. Dr.

Former Associate Researcher

    Explore current HIIG Activities

    Research issues in focus

    HIIG is currently working on exciting topics. Learn more about our interdisciplinary pioneering work in public discourse.