Student Project: "INMUSE"

• Download English-US transcript (PDF)

[SQUEAKING] [RUSTLING] [CLICKING]

CHARLENE: Hello, everyone. My name is Charlene.

THERESE: My name is Therese.

CHARLENE: And we're here to present INMUSE, a music capturing game. So you're going for a walk, and you put on Bose Air glasses and turn on your phone, and open up the INMUSE app.

As you walk with your Bose Air glasses, you will hear a melody to your right. You turn towards the melody to get a better hearing of it. And if you like it, you double-tap on your glasses to capture the melody.

If you feel the melody is not something that you like, you can shake your head and let it go. And by the end of the walk, you will have to capture a series of melodies and compose your own little soundtrack that you experienced during your walk. So here's a little video of a [INAUDIBLE].

[LAUGHTER]

AUDIENCE: [INAUDIBLE]

AUDIENCE: All right.

[LAUGHTER]

CHARLENE: [INAUDIBLE]

[STEADY BEAT PLAYING]

[MUSIC PLAYING]

AUDIENCE: Yeah, that was great.

CHARLENE: Great, so now we can do the demo. So the demo is set up where if you take 10 steps and it's in a very high sensitivity for the step counter because we're doing this indoor to trigger a sound, and it's ready for whoever want to try it out.

AUDIENCE: [INAUDIBLE]

CHARLENE: We have a very simple UI to indicate how many melodies you encounter, how many steps you took, and then how many melodies captured. The whole experience is aimed to be phones down, heads up, and just for you to walk. You don't really want to be distracted, first of all. It's really just to enhance and to kind of capture the moment as you walk.

So let me turn and get on the app. There you go. You should connect, and then you can just slip the phone right in your pocket.

AUDIENCE: So this is not head direction dependent, meaning one direction or the other--

CHARLENE: That doesn't matter.

AUDIENCE: Doesn't matter.

CHARLENE: Doesn't matter. So the audio is spatialized. We experiment a couple of times with spatialized music in order for the user can distinguish between the new sound and the sound that they captured.

But during our testing, we did find out it does make the walking experience more uncomfortable because they would like to face the direction they're walking instead of keep turning their head around. So spatialization in the moving-- when they're moving isn't as reasonable for the user to do as a request. So we cut that out.

THERESE: Yeah, and one thing that the more and more we worked on this game we wanted to focus on is distinguish it from just playing your normal music and have it more of a meditative and almost reflective experience, and really make sure that you're picking the sounds that speak to you. So that way at the end, when you have this song, it's kind of connected to a memory or whatever specific feeling that you're having at the time.

CHARLENE: One big caveat-- all the music that's in the app is something that we composed, and we're not composers. So the music itself isn't-- it's like the one we pick out that's open source online and the one that we just dropped our own little drum set drum beat. So they're not a wide set of it. So it's not super descriptive of everyone's experience at that time. So don't hate on the music too much.

[LAUGHTER]

AUDIENCE: Sounded good.

CHARLENE: Oh, good. So implementation evolution-- so throughout our user testing, the how to experience-- is it working well?

AUDIENCE: Cool.

CHARLENE: Yeah, great. So feel free if you want to try it later on, come on and try it.

Originally, we had several implementation in terms of having spatialized music where we describe having the new audio coming in different direction. The previous iteration when we first started the idea is that in order to capture the music, we wanted to actually face the direction of the music. You'd kind of be looking at it with the frame because you're hearing it in the direction of the sound.

AUDIENCE: Should I turn it off?

CHARLENE: Yeah. But then as we user test, we realized again having-- while moving forward and dictate what a user's head direction be is not a reasonable thing to ask. So we cut that part down. The user just had the option later on to change how many steps they want to take. There's a range between low amount of step, depending on the environment [INAUDIBLE], to the higher amount of step for the music to be generated.

So they've gone for a long walk. They don't want to be bombarded with a bunch of new music coming up. They select a longer generation. If they are indoor, for example, if you just want to take a short walk around the office and it's snowing outside, then you do a short iteration, short generation.

THERESE: Yeah, so one of our main research questions during this process was how we can create a mind space using only audio to make the player more meditative and reflective. So at first-- so we drew a lot of inspiration from a lot of different places. So I think, obviously, first is Pokémon GO.

So originally, we were thinking have a phone AR component, and you walk around. And in addition to hearing the sounds, also have a little sprite character where you would put your phone there and then capture it on the phone. But we found that really drew a person out of their own head and put them in more of a virtual world, which was kind of the opposite of what we wanted.

Yeah, so then another thing that we started thinking about was meditation, and how when people meditate, a very common way to do that is to count your breathing. So one way that we wanted to implement that with audio is have a consistent cadence. So you kind of match your steps to it, and it's that rhythmic, meditative state. And we found that it really helped out the users to get into that mindset.

And then during some play testing, we found in the beginning, people were really stressed out. And it's a lot of things going on, new sounds, a lot of decisions that they have to make. And people were saying that it was really intense. But the more that they played it, it became almost second nature, and players were really able to get into that mindset as soon as they put on the glasses, and wanted to make these decisions and make their own creation.

CHARLENE: So the user that we've being asking around have been people who are really into composing music, and people who are more interested in just having music in their life in general. And as we work with the instructor, there was a differentiation between composing music and almost like the experience of birdwatching, where you have spur of the moment encounter, and you can choose to capture it or let it go. And we want to emphasize the feeling of you can save the music, but you don't want to-- very spur of the moment, instead of keep tweaking it and premeditating over the experience, and recomposing it over and over again.

THERESE: Yeah, so that leads me to our next question, which was about control. So how does the level of control that you give the user affect the user experience? So as you were saying, we had a lot of different musical backgrounds of play testers. So that can be frustrating for people who don't know a lot about music. Or it was just kind of changing the experience in a way that we didn't want it.

So also, if you give the user the option of as soon as you get a sound, you can change all these different things about it, it made walks much longer, and drew people out of the experience. Because you don't want to wait until the end to be like, oh, that thing that I caught like 10 minutes ago, what can I do with it? So it was kind of derailing the experience.

And presenting it more as a kind of birdwatching experience really helped align everyone's expectations so that everyone would enjoy it, and kind of dealing with what you get and making the most of what you get also supported our goal of having it as a meditative experience, and kind of analogous to life. You just make the most of what you get, and have something beautiful in the end. So it was really exciting.

CHARLENE: So the future iteration of what we want to do is more-- right now, the current set of music is at a particular BPM. But sometimes on a slow day, you want a slower melody. Or if you're on a fast day, you want a higher, pumpy one. We would like to have the option for the user to choose that in the beginning.

Ideally, it's a input that's more intuitive. So it's to have them input-- so it's hard to-- for someone who is not musically inclined, if I tell you, oh, put it into BPM of your choice, they're not going to be able to have an idea of 140 BPM versus 120 BPM and how that feels. Instead, we want to be able to incorporate more the sensors to address the gyroscope or accelerometer to track your head nod to the beat that they want. So if they want a faster beat one, they could shake their head a little bit faster. If they want a slow beat one, it'd be a mellower head shake. And from the tracking of the beat, you pre-select what music you want to be able to offer to the user.

So in the future, we want to be able to passively track users' emotional state with the physiological sensor that's embedded onto the AR frame, and this way have a more continuous experience.

THERESE: Yeah, and then right now, it's a very personalized experience. And one thing that we wanted to implement in the future is start to build a community around that. So right now, at the end of your walk you have this song, and you can just save it for yourself. But in the future, we'd like to have it as an open platform for a community of other users on the app. And then maybe someone has this really nice beat. And then someone else could-- they could collaborate and add vocals and make songs, or just share with your friends.

CHARLENE: So the whole idea started out with the feeling that music can capture emotion or a certain moment. And we want to be able to incorporate that into everyone's life. And that's the goal of INMUSE, hopefully. Yeah, and that's it.

[APPLAUSE]

Free Downloads

Video

• Internet Archive (MP4 - 26MB)

Caption

• English-US (SRT)