Student Project: "Going in Blind"

• Download English-US transcript (PDF)

[SQUEAKING]

[RUSTLING]

[CLICKING]

PROFESSOR: Take it away.

GWEN EDGAR: All right, we would like a volunteer. And I believe we have someone who enthusiastically offered.

RIO LAVIGNE: Yep. So we would prefer you not to look at the computer screen.

GUEST SPEAKER: OK.

RIO LAVIGNE: And just stand--

GWEN EDGAR: --facing the audience.

RIO LAVIGNE: Yeah, the audience. This will be great. Try not to disconnect the wire.

GUEST SPEAKER: OK.

RIO LAVIGNE: We apologize. This is still very much a prototype.

GUEST SPEAKER: Oh, I'm glad you picked Rondos. These fit me better.

RIO LAVIGNE: Same. And this may take a moment to--

GWEN EDGAR: Should we plug it in?

RIO LAVIGNE: Oh, yes. Now-- no looking. No looking. No looking. Right, so as you can tell, this game is meant to be played without any visual input. In fact, do you hear things, by the way?

GWEN EDGAR: This is the game view. For those that aren't super familiar, this is our scene view. Because we wanted you to see what was going on. This is what the phone would show.

RIO LAVIGNE: OK, so wait just a moment. You should be-- I'm going to just check. Hold on. Face forward. OK, great, you're calibrated. So I turned off the screen so you can see what our play tester is looking at.

GWEN EDGAR: Well, facing towards.

RIO LAVIGNE: Facing towards. Yes, so our play tester hears a hey over here. In this game, you move using head tilts. So you tilt your head forward or down to move forward. You tilt your head backwards to move backwards.

GWEN EDGAR: And this is the player.

RIO LAVIGNE: So the teal dot is the player. And you can turn to face the sound source and tilt your head down to move towards the sound source. If you hit a wall, you'll go bonk.

GUEST SPEAKER: Is that what that big bonking, knocking sound was now?

RIO LAVIGNE: Yes. We still have to work out the balances between all the sounds.

GUEST SPEAKER: I seem to have hit a wall. But I'm moving backwards.

RIO LAVIGNE: Tilt your head back. Yep.

GUEST SPEAKER: Is the over here spacialized?

RIO LAVIGNE: Yes. Yes.

GUEST SPEAKER: It's behind me. How do I go backwards?

RIO LAVIGNE: Tilt your head back.

GUEST SPEAKER: Oh no, not that way. I like the footsteps.

RIO LAVIGNE: Yes.

GUEST SPEAKER: It's asking me what I'm doing. Like, what are you doing? OK, this is really hard.

RIO LAVIGNE: All right, congratulations, you found the first one. Now there's another one.

GUEST SPEAKER: Oh, more, correct.

RIO LAVIGNE: Oh great, yeah.

GUEST SPEAKER: OK, someone else want to try? Or should I keep going?

RIO LAVIGNE: Keep going for a little longer.

GUEST SPEAKER: I'm doing something else now?

RIO LAVIGNE: Yeah, you're going to try to find the target again. So this is actually our calibration level. So you got it.

GWEN EDGAR: We found that the calibration level was really useful for giving a frame of reference for distances and how far-- what the sound is and how it sounds further away.

RIO LAVIGNE: Yep. Yeah, so maybe try find one more target. And we can move to the next. What is going on? OK, well, that happens sometimes.

GUEST SPEAKER: [INAUDIBLE] this way. Nope, too far. Too far. I can tell it's over here, but I don't know how to get there.

RIO LAVIGNE: If you-- because the-- you can turn yourself, by the way. You can rotate. And you will now face that direction.

GUEST SPEAKER: There's a wall right here. I kept knocking. I'm going to go this way. Why do I keep hearing a [INAUDIBLE]? There we go. No?

AUDIENCE: Might be that the sound source is very close to a wall.

RIO LAVIGNE: That's true, yeah. Well, we don't want to take up too much time from our presentation. No, pff, we anticipated people to have difficulties. Here, do you want to take a look at what the game looks like? So you're the teal player.

GWEN EDGAR: So the first one was here. And then it moved up here. And then it went across.

GUEST SPEAKER: Oh, it moves.

RIO LAVIGNE: It did move.

GUEST SPEAKER: Oh.

GWEN EDGAR: Surprise. OK,

GUEST SPEAKER: Yep. Yep.

RIO LAVIGNE: And we actually want to showcase one more level. This time I'll put it on, put the glasses on. Because it's-- unless-- do you want to try?

GUEST SPEAKER: No.

RIO LAVIGNE: Nobody's very good at it, by the way, when they start out. But people do get used to it.

GUEST SPEAKER: Yes.

RIO LAVIGNE: Oh, I'm bonking into a well. Fun times. So we also want to showcase a little bit of level 2, maybe.

AUDIENCE: So you've had people go through this?

GWEN EDGAR: Yes. People--

AUDIENCE: Do they recognize the sound behind them versus in front of them OK?

GWEN EDGAR: So this is one of the points we were going to bring up before. It's very difficult to tell in front or behind. But as you move, you can hear whether you're getting closer or further away.

AUDIENCE: And is that done with louder? Or is that done with some other sound effect?

GWEN EDGAR: That's done-- so we actually were wanting to do something where, when you're facing it, it actually gets a little bit louder. So you know you're looking in the correct direction. Right now that's not very dynamic.

RIO LAVIGNE: Yeah, I'm having technical difficulties. We'll show off level 2 later.

AUDIENCE: Are you using any particular spacializer for the sound? Or is this just like panning?

RIO LAVIGNE: Well, it's--

GWEN EDGAR: We have a different rolloff, but we're just using what Unity afforded us.

RIO LAVIGNE: Well, Unity plus Bose AR. So as you noticed, later on, you can turn yourself and then move.

AUDIENCE: Right.

GWEN EDGAR: Yes. Thank you all for being here. This is our presentation for going in blind. I'm Gwen Edgar, and this is my teammate.

RIO LAVIGNE: Rio Levine.

GWEN EDGAR: And welcome. So first, we're going to start you off with the background on blind gaming.

RIO LAVIGNE: Right. So actually, blind gaming has been around basically since video gaming was a thing. When I'm talking about blind gaming, I mean electronic gaming. So starting in the '70s, Atari came out with this game called Touch Me. And it had flashing lights. And it made noises. So it was actually accessible to people who couldn't see.

Also, all of these text-based role-playing games were getting popular. And if you had a text to speech software, a visually impaired player could play those as well. But console games, action games were very limited. But gamers are going to game. So blind gamers played video games anyway.

So they could use text to speech, and just based off of sound cues, navigate through very complex levels, completing games like 2D Punch Out, Super Mario 64. A blind gamer has even completed Ocarina of Time, if you've ever played that game, it's crazy without visuals. Even Resident Evil 4 was accidentally accessible. Because if you pressed pause, your character would reorient themselves to face towards the goal that they were trying to get to.

Fighting games, like 2D fighting games, are actually pretty accessible, because they have binaural sound. You can tell which side your opponent is on. So Mortal Kombat, Street Fighter-- both pretty accessible. And blind players have even been known to play first-person shooters. So as long as there are enough sound cues, it seems like blind players can play these games. And that brings us to--

GWEN EDGAR: --our audio-only platformer, which, as you can see, you see nothing. You have to use only audio cues to navigate throughout the space, because it's using a lot of binaural inputs. So you're able to tell in what direction different things are.

As you all know, we're using the Bose glasses for this project. And there are a couple of gestures that are automatically integrated, such as nodding your head, shaking your head, and double-tapping. We were exploring these different things. And so first, we were trying to do movement with head nods-- so step, step, step, step. And we found this got really tiring.

And it was hard to know, were you taking a small step? Were you taking a big step? How far am I going? Like, how do I get through these different types of movements?

So we decided to look at something else, which, as you saw, you tilt your head down to move forward. If you're looking this way, you go this way. You tilt your head back to go back. And this allowed for a certain smoothness of walking. And the foot-- you hear footsteps as you move forward, which gives a sense of you being in the place and a sense of that movement that you're playing towards. And as you turn your head, your direction will change. You might have seen a couple of times that there was a curve happening because of motions like this.

So that brings us back to the Bose glasses. We were using the gyroscopes a lot, which meant we couldn't use the head nod or shake, because they were kind of using the same thing. And if you're trying to move forward, you can't do this. And those were the sensors. So that left us with one thing, the double-tap.

Now what does any platformer need?

RIO LAVIGNE: Well, Gwen, I think it needs platforms.

GWEN EDGAR: Exactly. What do you need for platforms? Jumping. So we decided to move that double-tap towards jumping. And so we were thinking about, OK, so we're going to double-tap to jump. But we want to follow the platform. But if we're looking at a platform up here, we might start walking backwards. And if we're looking at a platform down here, we might start walking forwards.

So we were thinking, all right, all right, we'll have a jump mode. So you double-tap first, you enter jump mode, and then you don't move around. So you can look at all your targets and not worry about falling over an edge or going out of the zone.

But this got really confusing. We had it such that three missed jumps, you'd exit. It wasn't integrated very well. It didn't feel very smooth. It was kind of clunky. People were like, why did I stop moving? So we decided to move away from that, and instead, just jumped. And what this really meant was that we could only design levels in certain ways that looking at the platforms would be within your range of motion without trying to go outside of it any way.

RIO LAVIGNE: Without, like, moving.

GWEN EDGAR: Along with the jumping though, even more, was that if we just tried to apply a force, the player could go spiraling off into space. Because they didn't know that there wasn't something for them to land on. And if we allowed them to jump at any point, they might jump onto a wall, and then fall, or go through the wall, or fall down. And they don't know why they failed.

So instead, what we did is we allowed jumping onto a platform. So if you're looking at a platform, you just jump, and it's all good. And if you don't, it's like, nope, and doesn't let you. And we found that worked pretty well. And it didn't upset players. That was very important to us.

RIO LAVIGNE: Yes. So OK, we've gone over what you can do within our game. Here are some of the challenge we faced-- challenges we've faced when developing it so far. First challenge-- player orientation. Fortunately, it worked right off the bat when we did our demo. That's good. But in general, if you just start the game without doing any kind of calibration, you're looking in some random direction. So you need to make sure to face the player forward.

And since the last time we gave this presentation, we found a method that faces you forward towards the z-axis. But there's no way to face and make the default any other direction, which is fine.

GWEN EDGAR: We just built all of our levels along the positive z-axis.

RIO LAVIGNE: Yep. Next challenge--

GWEN EDGAR: Properties of audio sources and the placements where we put them-- this was discussed in some of the Bose lectures and in some of the psychoacoustics lectures that we're talking about. But yes, forward and behind, you can't really differentiate. So if you're on a platform and there's something moving behind you, there's a platform moving in front of you, you can't really follow either platform. Because there's the-- you can't focus on either source. They get very muddled in your head.

RIO LAVIGNE: If it's not clear, the platforms are making sound.

GWEN EDGAR: Yes, I'm sorry, I forgot to mention that. The platforms make sounds so you can follow them. So you'll get on a platform. We turn the volume down on that platform, or else it's just overpowering. And when you leave it we turn it off such that you can always detect what's in front of you.

And that works really well. Because the audio that is moving with you doesn't bother at all with the other moving sources. So if you're on a platform, it can still make its noise. And you can follow one in front of you.

RIO LAVIGNE: Yes.

GWEN EDGAR: And ambient noise also is fine to use.

RIO LAVIGNE: Mm-hmm.

GWEN EDGAR: But also, as discussed, you can tell something's left or right. And you're able to turn towards those.

RIO LAVIGNE: Yep.

GWEN EDGAR: Limited inputs, we were talking about this earlier. In a lot of games, you have keyboards. So you can have special abilities, and different modes you can enter, and a bunch of menus and whatnot. And with this, we had the two. And so we were working a lot with those. And that was just one of the challenges we faced.

RIO LAVIGNE: Yep. Also, as you might have noticed, your method of movement is imprecise. You don't know where you're going. You're not really sure if you're going straight forward. You don't know where exactly-- how far things are away from you.

GWEN EDGAR: You might think you're going forward, but your head is tilted to the side a bit. So you're actually going in this direction.

RIO LAVIGNE: Fun problems to think about.

GWEN EDGAR: So one of the first things with this was the platforms. Because players had a lot of trouble locating exactly where the platform was, we had to make a huge collider. So this entire green box, if you hit-- if you're gazing anywhere inside of it, you'll go to the platform on your jump. It was almost impossible to hit this even when looking at the screen.

RIO LAVIGNE: Yes.

GWEN EDGAR: And in graphic games, it's also lenient. Because you jump, and you can fall onto it. You don't have to jump exactly to where it is. So it follows.

RIO LAVIGNE: And to help with the imprecise, like, I think I'm moving forward, but maybe I'm listing to the left or right, we made it so that if you were going fast enough and you're within 15 degrees of forward, you will go straight forward on the z-axis. This made one of our levels, which was essentially a very long hallway, much more-- much less frustrating-- I won't say easier, but less frustrating-- for players. Because then they could be like, oh, I'm actually going forward.

As mentioned, don't use different heights. So right, like--

GWEN EDGAR: Yes, so as players were playing more and more, they were able to tell-- they were able to learn more about the sound and tell where it was. But when you're starting the play, it's really difficult to try to detect where anything is in this axis. And I think that, based on all our play tests, many players were in a kind of restful position when trying to follow it, which meant that they couldn't see anything above them, first of all. Because restful is apparently tilted down a little bit and not looking forward, and also that-- I'm sorry, that the-- we couldn't build levels upwards right now. So we were trying to build them more straightforward so the players were able to jump back and forth.

RIO LAVIGNE: So--

GWEN EDGAR: --playtests.

RIO LAVIGNE: Yeah. So on Monday, I got 11 players to playtest the game. On Wednesday, I got one more player to playtest based on some of the feedback we got. And oh, by the way, everyone said that it was an interesting experience. And some of them even said that it was fun. A lot of them did say it was fun, so that's good.

GWEN EDGAR: Yeah, so one of the first things that popped up a lot was distance detection or radar. So you have no idea how close you are to the wall. You experienced that. Or you don't know how close or far something might be. We were thinking of adding something. Like, perhaps you could have something like a cane tapping, and you'll hear different noises. Or oh, there's a gap here. And we could take away barriers around edges with something like this, or something like rear assist parking. So as you get close to walls, it goes, beep, beep, beep, beep, beep, beep, beep, beep.

RIO LAVIGNE: Yep.

GWEN EDGAR: Frame of reference.

RIO LAVIGNE: Oh yeah. So the first level went through a couple of iterations. In our most recent iteration, you have to find multiple targets before you progress to the next level. And this is to give the player a chance to really get more in the game, understand, this is the target. This is the sound the target makes. This is how I locate that sound. This is how I move forward. And moving backwards is also an option.

GWEN EDGAR: The target after the one that you saw was directly behind. So the idea was trying to give players that range of motion and the noises.

GUEST SPEAKER: Was I listening to more than one target at a time?

GWEN EDGAR: No.

RIO LAVIGNE: No.

GUEST SPEAKER: OK. There were two voices. But that was just one target, right?

GWEN EDGAR: It was one target, yes. The target would spawn after the next one was destroyed, last one was destroyed, yep.

RIO LAVIGNE: Yeah, so front or behind versus left or right. Damage, death, and other punishments-- so we talked about players spiraling into the void after they jump. What do you do when a player dies? You put them at the beginning of the level, right?

GWEN EDGAR: Obviously.

RIO LAVIGNE: They failed. Well, this is confusing. Because players don't really know why they failed. They--

GWEN EDGAR: Or where they are now.

RIO LAVIGNE: Yeah. Because now you've just moved them without them moving themselves. So they've completely lost where they are. So we decided to take out damage, and dying, and all of that. We have a level with obstacles in it. If you hit the obstacle, you say ouch. You get pushed back a little bit. And this doesn't disorient players. If players are worried about dying, they're much more afraid to move and explore their world as well. So we took that out.

GWEN EDGAR: Level complexity-- as I mentioned, during the playtest, players were saying that as they were going through, they were getting better at navigating. They were feeling more confident. And they were getting more used to the sounds they were hearing. So they're able to detect different noises better and the patterns. So we were wondering how far we can go with this.

If we have people that know how to play the game, how many levels of complexity can we add? Can we have a staircase of platforms? Can we make it so you need to navigate almost kind of-- platforms almost like a maze? What can we do with this? Can we have enemies? How many enemies? What kind of things can be used? So the idea is, really, how complex can these end up being made? And this was not something that we could follow in the scope of this class, but that we would like-- be very interested in pursuing.

RIO LAVIGNE: And on average, it took players about 10 minutes to get through all of the levels that we had.

GWEN EDGAR: The three levels. We only saw the first one in this round. But there's a level with the platform and a long hallway with swinging blades.

RIO LAVIGNE: Yep. Yes.

GWEN EDGAR: And last, level guidelines-- these are-- we have three main level guidelines that we want-- that we take as our contributions and we want to give to you as main takeaways from one project, the first one being to offer the player one audio target at a time. That can be trying to navigate, looking for a source. That can be trying to jump on the platform. That can be trying to find the swinging blade. After you have gotten past that, you can present a new target as long as the old one has been removed.

Second--

RIO LAVIGNE: Always have a linear and flat path to follow, at least especially in the beginning. Because you can't really detect when something is above or below you, especially if you haven't heard the sound before. So especially with these beginning levels, make sure things are just in front of the player.

GWEN EDGAR: So this doesn't mean that you can only move forward in the level. But it means you need to have a linear progression path. So you can-- you can go backwards. You can move around. But if you've used an element that needs to go away, or else be specifically brought back for reuse--

RIO LAVIGNE: Yep.

GWEN EDGAR: And third, give clear and immediate feedback. When you're looking at something, you're able to see, oh, I missed the platform, or oh, I got hit by something, or oh, I didn't make the jump, or oh, I ran into a wall. But if you don't have that immediate feedback here, people are just like, what happened? It doesn't sound like anything happened. Or oh, things sound further away now. They don't have a good orientation point.

So footsteps when you're actually walking, the nope when you can't jump, the bump against the wall when you hit it-- we're very clear in good ways to give that feedback.

RIO LAVIGNE: Yep. And OK, well, that's our presentation.

GWEN EDGAR: That's our game, Going in Blind.

RIO LAVIGNE: Yes.

[APPLAUSE]

Free Downloads

Video

• Internet Archive (MP4 - 47MB)

Caption

• English-US (SRT)