Category: Interaction Design

It’s been some days since I returned from Barcelona. The conference was incredibly interesting and I am grateful to be a part of this experience. Together with my fellow interaction design colleagues, I share some nerve wracking adventures as well as beautiful memories. 

When reminiscing about the OFFF Conference I remember a few special moments which stuck in my mind. 

Gavin Strange

The very first talk I been to on Thursday was with Gavin Strange and wow, I was blown away. It is by far the most amazing speech of the conference. I really enjoyed it, because his presentation was so diverse, it was funny, inspiring, experiential and emotional. The visuals and the intro was eye candy in a unique way. Gavin works as an Art Director in Bristol, UK and he worked for quite some time in the industry. During his presentation he made clear, that he struggled to find his “own” style, as every designer has to find their “unique” style to be successful. But after some time he quit searching for it and this is where he became free in his doing. Gavin works in several areas for instance animation, art direction, graphic, music etc. Recently he discovered a musical interface and even if he has no clue how it works, he enjoys experimenting and he also presented his musical progress live on stage. He also knows, that he is not good at it and has just started, but he didn’t care and thought it was funny to play music for the audience. 

The main takeaway of his speech is that it is okay, if you have not found your “own, unique” style as a designer and always keep on experimenting and exploring. You don’t have to be good at it but progress comes with practice. 

Gemma O’Brien 

Another talk I think was really refreshing was Gemma O’Brien. She is an artist and designer known for her letterings and murals. I enjoyed finally seeing a woman on the main stage presenting her work and I did not regret it. The way of her speech was pleasant and inspiring. She is very successful and worked together with global brands. Gemma talked about the journey of becoming an artist and the obstacles she faced during her career. Also about how afraid she was using colors as she was only painting analog. As soon as new technology came around she started exploring painting with colors and how she loved it. Even when she thought she achieved everything on her bucket list at a very young age, she decided to make a cut and do something completely else, like running a marathon. 

What I have learnt from her is, that it is okay to be hesitant with certain things but you should try and overcome that fear because you might actually love it. Also, that sometimes a break is needed to find new inspirations and energy. Collect artwork from others and let them inspire you (as she was showing the audience her collection book). 

Joshua Davis 

Joshua Davis is very successful in the field of generative art. He started quite young when technology was by far not the technology we know today. Joshua mainly showed his work from the recent years and the brands he worked with. In my opinion, you really can tell, that he made a lot of money by selling his NFT art. Also he worked with some interesting brands e.g. trashcans.What Joshua Davis wants to tell is that it is okay to work with brands you think is not your dream collaborator but sometimes you have to lower your expectations, when it pays your bills. 

Gab Bois 

Gab Bois is a photographer/product designer from Montreal, Canada. I was super excited for her speech on Saturday, as I follow her on Instagram for a while now and was always curious about her process. Gab talked about her childhood and how she started creating art since then. Also she spoke about her process of finding inspirations for the art as well as how she is creating the final products. I think it was really interesting to see how you can create things with ordinary components. 

David Carson 

Probably the most discussed speech of the conference. Later this evening after the talk we sat together in the apartment and had a discussion about his presentation. I have to say I am quite torn apart as I was expecting a lot more of his presentation and less toe incidents. But also I haven’t known him and his designs before. In my opinion those people who have known him and have been fans for some time, they celebrate everything what he does on stage – doesn’t matter how weird it is. Also as Mr. Fabry said, this is what he does on stage and he wasn’t surprised at all. So there are two parties – the ones who know and celebrate and the ones who do not know and are shocked. 

The key takeaway of his speech is: You can do EVERYTHING on stage, when you reach a certain point in your career ladder. 

Now: Enjoy some of my pictures 🙂 

Links:

https://www.gemmaobrien.com

https://joshuadavis.com

https://www.gabbois.com

http://www.davidcarsondesign.com

https://www.jam-factory.com

Overall, the article presents a well-designed and informative project on the concept of timbre and the implementation of the Timbre Explorer synthesizer interface that enables real-time control of four salient dimensions of timbre, i.e., attack time, brightness, spectral flux, and spectral density. The interface has a graphical user interface with live visualizations to help users understand the effects of each dimension. The applications of the Timbre Explorer include educating users about sound synthesis, the concept of timbre, and the frequency domain, as well as providing a practical shortcut for synthesizers and advancing our understanding of timbre through further perceptual studies. The article describes the background of the study and its influence on the Timbre Explorer’s design, including the previous timbre space instruments. The author also explains the Timbre Explorer’s synthesis model, which controls the base, unfiltered spectrum of the sound, the frequency filter, the initial spectro-temporal evolution of the timbre, and the temporal amplitude envelope. 

One possible improvement could be to include more options for the user to customize the interface’s appearance and layout to better suit their preferences and needs. Additionally, some users may prefer a more tactile or physical interface, such as a MIDI controller, to better control the Timbre Explorer’s parameters.

One potential criticism of the article is that it could have provided more detail on the practical applications of the Timbre Explorer beyond its educational purposes. While the author briefly mentions its potential as a performance instrument and for further perceptual timbre studies, it would have been interesting to read more about its potential use in the music industry or other fields.

Overall, however, the article effectively communicates the purpose and design of the Timbre Explorer and provides valuable insights into the concept of timbre and its practical implementation in music synthesis.

http://doi.org/10.21428/92fbeb44.92a95683

The article Feeling the Effort of Classical Musicians – A Pipeline from Electromyography to Smartphone Vibration for Live Music Performance provides an insightful overview of the live-stream MappEMG pipeline project, in which a mobile application was developed to mimic the muscle response of classical music performers for audience members. This project began with the notion that these “gestures”, or invisible muscle movements made by musicians while performing, are integral to the musician’s sense of place within a piece, and would increase the audience’s sense of immersion if they could be shared.

The project utilizes EMG sensor mapping to track muscle movements, “From an artistic perspective, EMG gives a direct access to the performer’s intention in terms of implied musical effort, which is expressed through actual physiological effort”. These movements are then reproduced as vibrations through the mobile device. Sound-based vibrotactile feedback has already been used for collaborative composition, audience interaction, and greater immersion for those with hearing impairments.

I found this case very compelling as I had never heard of a similar project. It is also interesting to see a real world use case involving the tools we are using now, such as Max8. I appreciate the drive to create a more immersive audience experience, and also to explore a new element of the performance in the musicians’ gestures.

References

Verdugo, F., Ceglia, A., Frisson, C., Burton, A., Begon, M., Gibet, S., & Wanderley, M. M. (2022). Feeling the Effort of Classical Musicians – A Pipeline from Electromyography to Smartphone Vibration for Live Music Performance. NIME 2022. https://doi.org/10.21428/92fbeb44.3ce22588

ID task – Literature research

The topic of this article is the idea, development and evaluation of the On Board Call, a handheld device for imitating wildlife sounds.

“The On Board Call device is a handheld electronic gestural instrument that synthesises sounds resembling wildlife calls. It is designed to encourage deep listening and personal expression through imitation of natural sounds and as a performance tool.” This is the description of the device by the developer. Rather than playing back recorded wildlife sounds the On Board Call creates an imitation by synthesized sounds. The user creates these sounds with the board through a user interface on the device.

The article spends a lot of time explaining the technicalities of the hardware and software. It does state some reasoning for the choices that have been made, but I would be more interested in the arguments behind the gestures and interface layout. This is what the user interacts with, and will threrefore be critical for their experience. The article does for example not explain how well the ergonomics, learnability or feedback is.

Despite missing information about the argumentation behind the interaction between user and device, the article does explain that user trials have been carried out. It is not completely clear what the results and measures taken after the trials was when it comes to the interface, so this would have been interesting to hear more about.

Overall the project seems interesting, but a natural next step would be to test the interface more. It seems this has been designed out of ease for the producer, not out of usability.

Source:
https://nime.pubpub.org/pub/uofmcznd/release/1

ID Task 2: Literature Research

For this task i choose the article Debris: A playful interface for direct manipulation of audio waveforms published in 2021 by Frederic Anthony Robinson. It gives an overview of Debris which as the title suggests is a fun and intuitive software interface for manipulating audio waveforms. Its playful and engaging design aims to make audio editing a more creative and enjoyable process. The direct manipulation tools and real-time feedback help users understand the effects of their actions on the waveform. This encourages the user to experiment more throughout their creative process.

The interface is user-friendly, with clearly labeled tools and easy access to different features. The variety of tools available, such as looping, reversing, and layering tracks, provide users with many options for creating unique audio effects.

However, Debris may not be suitable for more advanced audio editing tasks due to its limited functionality. It is also only available as a standalone application for macOS, which limits its accessibility for some users.

Overall, Debris is a good tool for anyone looking to experiment with audio editing in a fun and intuitive way. It is well-suited for beginners and anyone looking for a creative audio editing experience.

Source:
https://nime.pubpub.org/pub/xn761337/release/1

For the Interaction Design class, I chose the article “COSMIC: A Conversational Interface for Human-AI Music Co-Creation” by Yixiao Zhang, Gus Xia, Mark Levy, and Simon Dixon as artificial intelligence is becoming more and more involved in our everyday lives.

The article describes the software COSMIC, developed as an interface for the creation of music between humans and AI. The AI assistant provides the user with musical ideas, which are developed and refined during communication. The machine learning model is based on a large data base and learns to generate new ideas while using the system. The idea is finally presented to the user and adapted to the user’s preferences by feedback from the user.

The interface has a very clean design and is divided into the five categories:

1. message tabs
2. status panel
3. conversation panel
4. input panel
5. preview panel

The interface has a minimalistic design with a clear structure. The design resembles already known platforms and the used icons are clear and easy to understand.

Without going into the technical details, the system is based on the following architecture.

In my opinion, the system arouses my interest to test it. Especially as a person who has no musical background, I can imagine getting a simplified and explorative generation of music ideas in it.

Sources:

COSMIC: A Conversational Interface for Human-AI Music Co-Creation

by Yixiao Zhang, Gus Xia, Mark Levy, and Simon Dixon

https://nime.pubpub.org/pub/in6wsc9t/release/1

The design of a bespoke musical interface designed to engage the public’s interest in wildlife sounds
by Andrew R. Brown

I liked that this project used smooth real-time mapping of human gesture data to synthesis the sound of wildlife calls. As seen in the P.L.A.C.E performance https://www.youtube.com/watch?v=uKgLrrI-MEU it forces the musician to move in unusual ways to produce sounds which makes the performance interesting to watch. I liked the idea of the inventor to design the Call as an a compact and inexpensive device design with minimal gestural dimensions in order to be suitable for use by the general public in community workshops or for infrequent personal interaction, thus sparking creativity. 

The User Interface of the Opject seems simple enough to be easily learnable. For the hardwear design I like that it is a handheld objektiv and that the size was kept as compact as possible to minimize resource usage in production. As the software was built with an Arduino it still feels very much like a replicable project.
The gestural interface spoke to be as we already produced our own prototypes last semester with our phones using the sensors. So I could imagine myself being able to construct something like this someday.

links:

https://nime.pubpub.org/pub/uofmcznd/release/1
https://www.youtube.com/watch?v=uKgLrrI-MEU