Expressiveness, which we define as the extent to which rich and complex intent can be conveyed through action, is a vital aspect of many human interactions. For instance, paint on canvas is said to be an expressive medium, because it affords the artist the ability to convey multifaceted emotional intent through intricate manipulations of a brush. To date, touch devices have failed to offer users a level of expressiveness in their interactions, which rivals that experienced by the painter and those completing other skilled tasks.
We investigated how data about wrist movement – provided by a motion sensor, similar to those found in many smart watches or fitness trackers – can be used to expand the expressiveness of touch interactions. We created Expressy, an implementation that uses a wrist-worn inertial measurement unit to detect and classify qualities of touch interaction that extend beyond those offered by today’s typical sensing hardware. We also created a range of example applications that demonstrate the enhanced, expressive interaction capabilities made possible by Expressy.
The Expressy project was conducted in collaboration with Gerard Wilkinson, Ahmed Kharrufa, Bradley Pursglove, Hendrik Haeuser, Nils Hammerla, Gavin Wood and Patrick Olivier from Newcastle University; and Steve Hodges from Microsoft Research Cambridge.
Wilkinson, G., Kharrufa, A., Hook, J., Pursglove, B., Haeuser, H., Hammerla, N., Wood, G., Hodges, S. & Olivier, P. Expressy: Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions. In Proc. of CHI ’16, ACM, to appear. Download
Wilkinson, G., Wood, G., Hook, J., Nappey, T., Kharrufa, A., Pursglove, B., Haeuser, H., Hammerla, N., Hodges, S. & Olivier, P. Interactivity: Using Expressy to Demonstrate Expressiveness in Touch-based Interactions. In Ext. Abs. of CHI ’16, ACM, to appear. Download
Wilkinson, G., Green, D., Wood, G., Kharrufa, A., Hook. J., Pursglove, B., Haeuser, H., Hammerla, N., Hodges, S. & Olivier, P. Video Showcase: Using Expressy to Showcase Expressiveness in Touch-based Interactions. In Ext. Abs. of CHI ’16, ACM, to appear. Download