Spatial Audio Through Body Movement

Thoughts prompted by Professor Lee Jung-mo’s presentation on embodied cognition at the Korean HCI conference in 2009.

I was once greatly impressed to discover a cheap mono speaker on the Wiimote, the controller for the Wii game console. Conventional spatial-audio solutions have included the extravagant approach of surrounding a room with multichannel speakers (5.1, 7.1, and so on), or the cumbersome approach of recording sound using dummy-head techniques and playing it back through earphones. The Wii achieves this in a surprisingly innovative way.

It makes sound come from the controller in your hand.

For example, when you swing the controller to hit a baseball, the controller makes a “crack” sound. Because it is a gestural interface, the controller is at precisely the point in space where the sound should originate at the moment it occurs. This produces very natural 3D spatial audio at very low cost.

This shift in perspective probably owes much to someone’s inspired ingenuity. I wondered whether establishing an embodied-cognition perspective and developing something like “embodied cognitive engineering” on that basis might reduce, even slightly, our dependence on flashes of genius when developing interfaces like this.