Feature: Striking a Collaborative Chord

Hamamatsu, a Japanese city of Music, played host to NIME 2004 in the sunny month of June this year. The New Interfaces for Musical Expression conference series started as a workshop at CHI 2001 (Seattle, USA), an origin which has shaped the focus of subsequent conferences. The series is primarily aimed at exploring the use and design of new user interfaces that are used to create music, and has even spawned University courses.

Given participants' wide range of interests and backgrounds it is very easy to be distracted by the plethora of novel interaction devices which take on a whole range of shapes and sizes from small devices that you 'strangle' to embedded systems which play music based on people's movement in a space. The key to such conferences seems to be to try to take a step back and understand how it all fits together in terms of exploring the space of musical interaction and user interfaces in general.

The conference itself was hosted at SUAC (Shizuoka University of Art and Culture), a campus which afforded many glimpses into Japanese culture such as the rolling alpine meadow on top of the main building with a range of bizarrely sized benches, some so large that one felt like a 5 year old child when seated, and a spotlessly clean campus which is something you don't tend to find in many Universities.

The conference was impeccably organised apart from a few of the usual problems associated with connecting computers to data projectors - maybe there is a conference series on such issues, and surely there must be blogs about the trials and tribulations of connecting an array of laptops to a data projector. In fact, the conference organisation even included a 'navigator' to guide you on your ten minute walk from the railway station to the conference venue.

The conference this year included two keynotes - Robert Moog (yes, he of the Moog Synthesizer fame) and Toshio Iwai. Moog's talk was half about technical developments and half about his views on how people interact with electronic musical instruments. The technical developments focussed on development of the Moog synthesizer in the 1950s (the first electronic synthesizer which has since had a lasting impact on all forms of music) and the theremin (a strange instrument developed in the 1920s which if played by moving one's hands near the instrument; watch any 1950s sci-fi movie and you're sure to hear a theremin making eerie noises).

Moog's view on how performers interact with their instruments was rather more esoteric - arguing that your mind merges with your instrument as you play it. Such a view is inspired by Rupert Sheldrake, author of books as 'Dogs that Know When Their Owners are Coming Home, and Other Unexplained Powers of Animals'.

I certainly don't feel that my mind merges with my computer when I'm writing, but maybe we as user experience designers should be aiming for such levels of intimacy.

Iwai's keynote was inspirational as he stepped back from technological considerations to discuss the relationship between music and its visual representations - exploring the nature of punched cards in simple player-piano style instruments. He demonstrated quite neatly the multiple interpretations and use of visual representations by playing a roll of paper with happy birthday encoded as holes backwards through a small player piano. This produced a melancholy tune conjured up ideas about other transformations such as folding the paper, playing it upside down, and so on, which can be achieved without any musical knowledge. Iwai also treated us to a performance of music using children's games controllers which, for me, really summed up what new forms of interaction should be about - not necessarily technologically astounding, but something which can be used to really engage with the music and the audience.

Developing new interfaces for music expression using computers allows us to explore two aspects of interaction not typified by conventional musical instruments - non-determinacy, and embodied collaboration. With conventional instruments such as a guitar, we can be pretty sure of the sound that will be produced when we play it in certain ways, whereas with a computer based instrument we could develop musical instruments which take our input as a suggestion and then play music that some algorithm feels is appropriate.

We were treated to a performance by a violinist and one of LEMUR's 'guitarbots' which consisted of four independent single-stringed slide guitar units controlled algorithmically to accompany the violinist. Similarly, Katsuhusa Ishida and colleagues presented work on algorithms to automatically correct musical melodies as they are played in order to allow novice players to improvise musical pieces. However, such 'correcting' systems raise questions about learning - if we are automatically corrected in our playing of pieces, how do we learn, and what should the boundaries of correction be.

Related to these issues, Sergi Jorda presented a paper on learning curves for new interfaces which highlights one of the interesting aspects of designing expressive interfaces - that there is a difference between efficiency of use and learnability, and these need careful consideration given the target user group.

The second aspect of using computers in the development of musical interfaces is that collaboration can be embodied into the instrument. With a conventional instrument such as a flute, a player collaborates with other players through visual and auditory cues shared within a physical space. When we develop new interfaces we have the chance to construct instruments where the collaboration takes place within the instrument itself. For example, Sidney Fels and collaborators demonstrated the Tooka - a two player musical instrument where sound is controlled by pressure in a tube between the two players' mouths, and a set of buttons for each player. The configuration is such that the single output sound is a product of the two players' inputs - the joint production takes place within the instrument whilst visual interaction helps co-ordinate the activity. The instrument itself is a bendy tube that two players blow into from each end which looks pretty strange at first sight, as was confirmed when Linda Kaastra and Sachiyo Takahashi treated us to a performance one evening.

From a more technological perspective, Ge Wang and Perry Cook gave a paper on 'on-the-fly programming' as a way of creating music in collaboration with other people. In this situation collaborators program signal generators, filters, and mixers which can be combined in real time to create music. The key is that data, or signals, can be shared between different people's code so providing a space for collaborative real time music generation by programming.

There were many, many interfaces which were highly strange, and focussed on more deterministic and individual control of music. For example, Ajar Kapur and his collaborators presented the eSitar which augments a conventional sitar with pickups to monitor the position of fingers and plucking. Such information can then be used to create sitar style sounds as well as graphical accompaniment to the music. It would be interesting to see how such instruments could be developed to change the nature of the instrument itself, say by mapping it to different sound styles or by developing a two or four player sitar. Similarly strange was the use of sound in karate training as outlined by Masami Takahata and colleagues. In their work karate students wear sensors which are tracked by computer and used to generate sound based on the acceleration of various body parts in order to provide real-time, non-verbal feedback about their moves. Maybe this could be used in driving instruction to encourage the development of a rhythmical driving style.

There were a whole host of other interfaces and devices at the NIME conference which I haven't the space to outline but which highlight the wide range of possibilities there are to supporting creativity with computers. The conference series itself has been running for three years now and to me it had the feeling of growing maturity which I hope will drive it to explore expressive interaction in a range of new, exciting, and rigorous ways.

Nick Bryan-Kinns
Queen Mary, Univesity of London

'Internet addiction' linked to Depression
Source: BBC, 9 February 2010
 
There is a strong link between heavy internet use and depression, UK psychologists have said.

Could *You* be more Usable?
Source: UN, 8 February 2010
 
Bet you could.

Stowe Boyd on 'Steampunk' thinking about the Future of Computing
Source: Stowe Boyd's blog via Experientia, 6 February 2010
 
Are established metaphors of user experience holding us back from new ways of structuring our interaction through computers?

Nokia's User Experience Programme
Source: UN, 5 February 2010
 
Nokia has put together a rich and informative website covering the key elements of user experience.

Interfaces magazine: latest issue available now
Source: HCI News Service, 4 February 2010
 
The latest issue of Interfaces is now available in pdf format, free from the Interaction Website.

A Lighter Brigade of Chargers
Source: UN, 3 February 2010
 
Lots of gadgets, one charger. At last.

Mobile Touch Screens could soon Feel the Pressure
Source: MIT Technology Review, 2 February 2010
 
A quantum switch could add pressure sensing to mobile screens.

Usability, Usability, Usability: why the iPad will Succeed
Source: Econsultancy, 1 February 2010
 
The tech critics love it, hate it, love it again, shrug it off. What do usability experts say?

British Airways - at last some good news
Source: Loop11, 30 January 2010
 
In a recent website usability study for the world's leading airlines, the British Airways website proved to be the most user friendly, with Malaysia Airlines and Virgin Atlantic having the lowest user experience rating.

Computation of Emotions in Man and Machine
Source: Royal Society, 29 January 2010
 
Advances in computer technology now allow machines to recognise and express emotions, paving the way for improved human-computer and human-human communications.