The 'Elasti-face' booth invited participants to place their head in an oval. Different luminosity zones of the face analysed with camera tracking produced distortion sounds as the face is moved and pulled, responding to distinctive regions of the face and encouraging playful interaction. Mitchell's large fluffy dice were colour-tracked in a prescribed zone on the floor and held Wii controllers encased inside that sent bluetooth motion data of tilt, yaw and rotation sensitivity back to the controller computer processing ambient sound loops in a relaxing fashion. Various dimensions of the sound were transformed as the dice were moved around and depending on which number surface was facing upwards. The reactivision table idea uses camera recognition of iconic symbols on the base of objects to locate multiple objects as they are moved around the table, here used as a sound mixer altering panning, timbral filtering, loop loading, loudness according the the objects placed and their position on the table. The sculptural pinwheels mounted on tall copper tubing were lightweight paper fans that responded to breath. As people blew the different controllers, they manipulated natural wind and airy sounds of the installation. The retro-looking space invaders game used traditional arcade game-style interaction to generate real time music giving individualised feedback on the game performance. A virtual fishpond projection on the floor uses boid flocking algorithms to control fish motion (flocking) behaviour in response to the user's footsteps, colour-tracking the mandatory Aussie hat for positioning and modifying the fishes' swimming over time. The gestural piano totally shatters conventional expectation of piano operation by removing the usual keys and using arm gestures (camera-tracked) to play the piano strings with solenoids and microphone to pick-up and process sounds, thereby producing the ultimate prepared piano implementation with unusual nuances and possibilities normally unheard from the musical instrument. The spatial sound controller using coloured balls (or any colour-tracking object) alters sound according to positioning left/right, up/down and proximity to the built-in iSight camera on the Macbook.
The students' brief was fairly flexible but invited them to develop one of (1) an augmented conventional physical musical instrument, i.e. a hyper instrument, (2) an interactive installation appropriate for gallery or similar, or (3) a data-driven informative ambient display (sonification or auditory graphing). In most other contexts, the audio stream operates separately in all areas sound and digital media, while concerned with all media types, often favours visual communication or linear media. This class provides students with an opportunity to explore sonic communication much more deeply and look at sound as an interactive and responsive medium and also consider its interrelationship in multi-modal display. The exhibition itself was quite ambitious involving 21 loudspeakers, a handful of data projectors, 5 audio interfaces, Wii controllers, 6 webcams for video tracking, Arduino boards, endless cabling and aesthetic props, physical computing and installation infrastructure. For the most part the staff provided and setup the equipment and the students constructed their booths, boxes, modified an old piano, made wind-blown pinwheels and the interface materials themselves. This was the first time we held the studio and its resulting exhibition. We were stoked by the large number of students, staff, friends, family and visitors who attended. It was naturally a fairly congested and noisy environment once all the works were underway but no interaction design is complete without some usability feedback and interaction experience.
Full-sized flickr photo posting
