BRIANNA: Well, it is right about 9 o'clock.
It only takes a second or so for folks to get on,
so I'll just say a little bit about who we are.
If you aren't familiar with us, AccessCSforAll
is that NSF grant based at the University of Washington
and the University of Nevada, Las Vegas
focused on increasing the accessibility of K12 computer
science education.
So through these webinars, we're trying
to get folks together from time to time
to learn about accessible tools that exist out there.
So today, we have Lauren Milne from Macalester College
is going to tell us about Blocks4All,
and I'll let Richard do the introduction.
If you need the captions, there's a link in the chat.
RICHARD: Great.
Thank you, Brianna.
I will say that most of the people on this call
are part of the AccessCSforAll development partners,
so many of you have committed to working
on developing tools and software for people with disabilities.
So thank you all for coming, and it's my pleasure
to introduce Lauren Milne.
She was a former student of mine at the University
of Washington.
And while she was there, she was very inventive
and invented this thing, Blocks4All.
Of course, I challenged her to do it.
You know, I knew all the block languages
were inaccessible to blind children,
and so they were left out of this things like the code.org
activities.
They just couldn't do them, because they
weren't accessible.
And she met the challenge, and today she's
here to tell us about what's been going on during her time
as a graduate student and also in the last couple of years
at Macalester College where she's an assistant professor.
Macalester by the way, is in Minnesota
in St. Paul, Minnesota, so she loves Minnesota.
I don't know why, but she does.
[LAUGHTER]
LAUREN MILNE: Hello, can people hear me?
Yeah.
All right, yes, so I am calling from Minnesota.
Thanks for-- once again-- thanks for the introduction, Richard.
So I'm going to talk about Blocks4All today,
and I thought I'd start by talking a little bit as Richard
mentioned about blocks languages in general.
So they started off--
let's see-- can I access this?
Scratch is a very popular block based programming language
where you basically can drag and drop different blocks of code
to program avatars.
And so these block based languages
have become more and more popular for teaching children
especially, but any novice how to code.
And as Richard mentioned, the Hour of Code activities--
here, I pull up their websites-- and if you look,
almost all of them are blocks.
So Dance Parties, blocks, language one.
Minecraft is a blocks language one.
Make a Flappy game blocks.
Star Wars blocks or JavaScript.
So these are becoming more and more popular.
But as you can kind of see, they're very visual in nature.
You're dragging these puzzle piece like units of code
around, and they don't--
we did a survey of all these block based languages--
and pretty much none of them had even rudimentary accessibility
with screen readers, in particular, we looked at.
So almost all of them you can't even
access the blocks themself.
Like nothing will be read aloud if you try and click
on the blocks with the screen reader on.
So because of that, we decided to develop our own block based
programming language just to show that it is indeed
possible to make it accessible.
So we end up developing on an iPad.
We really like the idea of having one where.
We found out a lot of children's visual impairments
were already using iPads.
They come built in with Voiceover,
which is a great screen reader.
But we also really like the idea of having a touch screen
so that we could convey some information spatially,
so a lot of the how blocks were laid out
or how the code was laid out was really visual and spatial
in nature, and we wanted to preserve that.
So I'm going to go ahead and share my screen.
I think this should work with my iPad.
It was working earlier.
Let's see.
Can people see that?
BRIANNA: Yep.
LAUREN MILNE: All right.
So here is my iPad with the main screen of Blocks4All,
and I currently have Voiceover on,
and I think it should even be able to hear the audio,
but let me know.
All right so basically what we have is a Blocks workspace,
and on the left side of the screen is the tool box,
and the tool box has all of the different blocks
you can place in the workspace.
So I'm just going to iterate.
I'm going to swipe through with Voiceovers,
so I'm just swiping with my finger,
and it should go through the different categories.
IPAD: Animals category.
Control category.
Double tap to explore blocks in this category.
Drive category.
Double tap to explore blocks this category.
LAUREN MILNE: So I can essentially
go through the categories and then go back.
IPAD: Control category.
Animal category.
Double tap to explore blocks in this category.
LAUREN MILNE: Animals category.
So I double tap to select that.
IPAD: Selection.
Toolbox.
Back button.
Animals.
Heading.
Make cat noise.
Make crocodile noise.
Make dinosaur noise.
Tool box.
Double tap to place blocking workspace.
LAUREN MILNE: And essentially now on my blocks
are successful in my toolbox.
I'm going to go ahead and place the make dinosaur
noise in my workspace, so I double tap to select.
And this-- if you're not familiar with Voiceover--
is all standard how you interact with things with Voiceover.
So I'll double tap to select the dinosaur block.
IPAD: Make dinosaur noise selection.
Select location in workspace to place it.
Main menu button.
Place make dinosaur noise at beginning.
LAUREN MILNE: And then I'm going to place it in my workspace
here.
IPAD: Make dinosaur noise.
Toolbox.
Double tap to place block in workspace.
LAUREN MILNE: So as I said, now we basically
have make dinosaur noise placed in my workspace.
So I wanted to comment a little bit about what
are the changes we made.
How does this differ from a standard blocks based language.
One, the blocks are actually accessible.
They actually are read aloud when you touch
them, which is very important.
We tried, we actually make the blocks
themselves are resizable, so if you have low vision,
you can make them bigger.
If you adjust your dynamic text in Voiceover over,
they become bigger.
And in general, we made them large icons that
were relatively easy to see.
The other thing we did is that we got rid of drag and drop.
So traditionally with the blocks based languages, almost
all of them are drag and drop, which does not work very well
at all with screen readers.
So instead, we have select-select drop.
So you select a block of the tool box around the workspace,
and then you select where you want it to go.
So I go back and--
IPAD: Make crocodile noise.
Make cat noise.
LAUREN MILNE: Maybe select--
IPAD: Toolbox.
Double tap to place block in workspace.
LAUREN MILNE: Make cat noise.
I once again select it.
IPAD: Make cat noise selection.
Select location in workplace to place it.
LAUREN MILNE: It's selected, and I can place it.
IPAD: Place make cat noise before make dinosaur noise.
Workspace block one of one.
Double tap.
Make cat noise.
LAUREN MILNE: Right there.
IPAD: In tool box.
Double tap to place block in workspace.
LAUREN MILNE: So that's the basics of how the app works,
and then of course the other thing
we did is that we wanted to make the output accessible.
So when you think about Scratch or some
of the other ones, a lot of times
they just have a visual avatar on the screen
that you can control.
But if you're visually impaired, that may not be that useful.
So we end up using a robot, and I don't have my awesome robot,
but I have a slightly less awesome-- let me see how to--
this is the robot, Dot, and it actually--
so the application works with the Wonder Workshop robots.
There's Dash, which is like a bigger one and moves around
and Dot, which is just a stationary robot that
pretty much just makes noises.
And I just have Dot there.
But if I go back to my iPad.
Zoom share.
OK.
IPAD: Play button.
LAUREN MILNE: Now if I go to the play button.
IPAD: Make your robot go.
Stop.
[MEOWING]
[ROARING]
Play button.
LAUREN MILNE: I don't know if you could hear that,
but the robot meowed and then made a scary dinosaur noise.
So we did a number of different blocks we have, so basic--
IPAD: Tool box.
Back button.
LAUREN MILNE: Noise making blocks.
We have some--
IPAD: Tool box.
Heading.
LAUREN MILNE: Common control structures.
IPAD: Control category.
Double tap.
Selection.
Tool box.
Back button.
Heading.
Repeat.
Toolbox.
Double tap to place block in workspace.
LAUREN MILNE: We have a repeat block so that you can
repeat things multiple times.
Repeat
IPAD: Forever.
In tool box.
Double tap to place block in workspace.
LAUREN MILNE: We even have a repeat forever block,
so you can repeat--
I think it technically is not forever--
I think it maybe stops after 100 repeats or something.
And we realized this summer when we added that block that we now
needed a way to stop repeating after someone added that
to your workspace.
IPAD: In tool box.
Double tap to place block in workspace.
LAUREN MILNE: We have conditionals,
so if statements-- we're working on adding else statements.
And then this one I think is--
IPAD: Wait for time.
In tool box.
Double tap to place block in workspace.
Repeat.
LAUREN MILNE: But I'll go ahead and add--
IPAD: In tool box.
Double tap to place block in workspace.
LAUREN MILNE: Talking over Voiceover.
I'll go ahead and add a repeat block
so you can see what looks like.
So I select it.
IPAD: Repeat selection.
Select location in workspace to place it.
Place repeat before make dinosaur noise.
Workspace block two of two.
Double tap to add repeat block here.
Repeat.
In tool box.
Double tap to place block in workspace.
LAUREN MILNE: All right.
So now I have--
IPAD: Make cat noise.
LAUREN MILNE: Make cat noise.
IPAD: Workspace block one of four.
Repeat two times.
Workspace block two of four.
LAUREN MILNE: My repeat block.
IPAD: End repeat.
Workspace block three of four.
LAUREN MILNE: And an end repeat block.
IPAD: Double tap to move block.
LAUREN MILNE: And right now there's
nothing inside my repeat block, so it wouldn't actually repeat
anything, so I can move--
IPAD: Make dinosaur noise.
Workspace block four of four.
Double tap to move block.
LAUREN MILNE: I'll move the dinosaur
block inside the repeat block
IPAD: Make dinosaur noise.
Place make dinosaur noise before end repeat.
Workspace block 3 of 3.
Repeat.
In toolbox.
Double tap to place block in workspace.
LAUREN MILNE: All right.
So I just wanted to point out sort
of similar to some of the other block languages.
We now have the sort of nested blocks
are nested inside controlled structures.
So this dinosaur block looks like it's inside
of a repeat block.
And it's kind of lifted up vertically.
So when I nav it-- so I'm going to navigate across the screen.
And it will actually read about of where this.
RICHARD: Lauren, can I just add a little bit
to what you're saying?
Just because things are happening fairly fast.
So, Lauren, is using the one finger explore.
So you can explore the screen with one finger.
And if you touch something that is like a block,
it'll read out what it is.
And then, if somebody has just read out,
she can double tap anywhere on the screen,
and a that'll select that block.
And now you're in a different mode.
You're in the mode of waiting to see where you're
going to place the block.
And so what you did--
the last thing she did was she found the dinosaur, which
was already in the program.
She selected it.
And then she placed it before the end of the repeat loop.
And all that was done by selecting with one finger
explore, and select with one finger,
and then double tap to select it, and so on.
So hopefully, that's clear.
So you we're going pretty fast there, Lauren, by the way.
LAUREN MILNE: Sorry.
Please feel free to interrupt me if I'm going too fast.
Yeah.
The other thing I want to note is
that there is on Apple now with Voiceover,
they have an alternative.
So one of the big things, as I mentioned,
was drag and drop is really hard to work with a screen reader.
And Apple has a drag and drop alternative,
which is essentially, if you're a Voiceover expert,
you select from what's called the rotor menu.
But when we were testing, we tested
with kids that were between five and twelve years old.
And we found that they had a ton of trouble
with accessing items in the rotor menu,
or even sort of the more advanced features of Voiceover.
So we really tried to make the app work kind of exclusively
with just sort of touching items on the screen,
and then double tapping to select.
Those are sort of the only two gestures
you need to be able to perform to use it with Voiceover.
RICHARD: Sorry, can I ask a question?
I don't want to inerrupt.
LAUREN MILNE: Well, yeah.
RICHARD: Could you talk a little bit more about that?
The kind of problems you saw with kids.
I've read the papers and stuff.
But I'm just curious about your firsthand experience
because I was--
LAUREN MILNE: Yeah, so it was really interesting.
In the initial study, we had one student
who was Voiceover expert.
But actually preferred to use the keyboard a lot
with his iPad.
But when using rotor and stuff, he didn't really have trouble.
But a lot of the students were really learning Voiceover
while they're doing it.
I had some students who even had trouble with double tapping
and would use the keyboard control to double tap instead.
But the rotor action was really challenging.
Yeah.
They had never used it before.
I tried to walk them through it.
But I think it was just sort of--
RICHARD: Yeah, that's pretty tricky.
So let me answer a little bit more on that.
So the choices that Lauren made for the interface
were things that kids were either learning or already knew
about Voiceover.
So she wasn't inventing a whole new interface.
She was building on their knowledge,
or their current knowledge, or the knowledge they're
building about using an iPad.
So that's a huge win.
If you can build an interface that
builds on what blind kids are already doing or learning,
then you're going to be more successful.
LAUREN MILNE: Yeah.
Yeah.
And I'll also know that my app is a lot more verbose,
I think, than, who typically you'd
want to design for the Voiceover experience.
And, once again, it was working with these kids
who are really learning Voiceover at the same time.
And so I sort of gave more direction
in the verbal commands when I said.
RICHARD: And just in case it's not--
no one saw it, too.
Jeff Gray asked the question, is there
a specific target age range that your work has focused on?
LAUREN MILNE: Yeah, so when I developed,
I was really looking at--
it was a pretty broad range because I
wanted to do user testing, and it's very hard to get students
that I wanted.
There's very few of them in Adrian.
So I really was.
But I was looking at 5 to 12-year-olds.
And I think a lot of the block based languages are I
think scratch whatever.
There's maybe a broader range.
And I think a lot of the things that we learned in Blocks
For All can really be extended to sort of older kids.
But I think the app itself and probably the dash robots
are maybe a little bit more suited for the elementary age.
Thanks.
I was not looking at the chat button.
RICHARD: Oh, no.
It's all good.
I'm really curious about this work.
And I'm just trying to--
I saw it pop up and thought I'd just say it.
LAUREN MILNE: Yeah.
So a couple of things.
So I actually had, besides what Richard mentioned,
I work at Macalester College.
So I work with undergrads.
And they added a whole bunch of features over the summer.
And one in particular was modifiers.
So [INAUDIBLE] happened with the blockface languages,
you can sort of modify expressions on your blocks
directly with the blocks.
So how we ended up deciding to deal with modifiers,
you can see this repeat block.
It has a two block right above it.
So we basically just placed any modifiers
directly above the block.
But it modifies.
Oh, and I was going to show you what
gets read aloud when you want to see what's inside this repeat
block.
So, once again, as Richard said, I'm
just going to touch the blocks.
And you can see what block I'm touching.
It'll have that black outline.
That's what Voiceover is currently focused on.
So I'll move across the screen.
And it should read aloud the block I'm touching.
IPAD: Make cat noise.
Workspace block 104.
Repeat two times.
Workspace block 204.
Two button.
Double tap to set number of times to repeat.
Inside repeat.
Making dinosaur noise.
Workspace block 304.
Double tap to end repeat.
Workspace block 404.
Double tap to move block.
LAUREN MILNE: So once again, sort of explicitly laying out
the structure of the blocks we found
was really important because you don't have the visual cues.
You can't see that the dinosaur block is nested inside
of the repeat block.
We tried to convey that super clearly
through these surrounding blocks.
So as you move off, you would know it's inside of the block.
RICHARD: Lauren, can you change that to two to a three,
for example?
LAUREN MILNE: Yes.
RICHARD: And demo that.
IPAD: Double tap to set number of times to repeat.
LAUREN MILNE: So if I double tap, I go into this.
IPAD: Increment button.
Repeat.
Back button.
Repeat.
LAUREN MILNE: I'll swipe through.
IPAD: Increment button two.
Increment button.
Repeat increment.
Increment.
Four.
Back button.
LAUREN MILNE: And then go back.
IPAD: Back.
Toolbox.
Heading.
LAUREN MILNE: And that's sort of standard.
You'll see I'll do some other modifier blocks to show you.
But that's sort of standard.
We had it essentially take you to another screen.
I think I'm going to try and change it
so that you don't necessarily get taken to another screen.
You can do it directly on the screen
so you don't have a context switch.
But that's how it currently works.
So now it will repeat four times when
it makes the dinosaur noise.
Yeah.
Let's see, what are some other fun ones?
IPAD: Drive category.
Double tap to explore blocks in this category.
Selected.
Toolbox menu.
Drive forward.
In toolbox.
Place drive forward at n.
Toolbox menu. back button.
LAUREN MILNE: So there's a bunch of-- it
won't work with this little green robot.
With the other robot, you can drive the robot around.
This one has multiple things that can be modified.
So I'm going to click on the drive forward block.
IPAD: Drive forward 30 centimeters at normal speed.
Workspace block 505.
30 centimeters.
Normal button.
30 centimeters.
Back button.
LAUREN MILNE: And we can modify the distance and speed.
IPAD: Distance.
Selected distance.
Current distance is 30 centimeters.
Distance slider.
30%.
Adjustable.
40 centimeters.
50 centimeters.
60 centimeters.
And these are standard Apple widgets
that you can use to adjust that work pretty well with Voiceover
so you can sort of swipe up and down
to adjust the amount of slider or hit
the incremental or decrement buttons here.
IPAD: Speed.
Slower.
Normal.
Faster.
Button.
Faster.
Back button.
Back.
Toolbox.
Heading.
LAUREN MILNE: So now if you look at it--
IPAD: Drive forward 60 centimeters at fast speed.
Workspace block 505.
LAUREN MILNE: And now it drives forward.
IPAD: Double tap to move block.
LAUREN MILNE: Quickly 60 centimeters
at a fast speed, et cetera.
What else?
We don't have any questions.
We have the drive blocks.
IPAD: Lights category.
Selected.
Toolbox menu.
Back button.
LAUREN MILNE: The robot has some nice different lights.
You can turn off different lights
and set them to different colors, which
some of us students use if they had some vision.
IPAD: Lights category.
Look category.
Double tap.
Selected.
Toolbox menu.
Back button.
LAUREN MILNE: A lot of these we just added.
But you can have the robot turn his head and look at things.
There are some sensors on the robot.
IPAD: Look forward.
Look toward voice.
In toolbox.
Double tap to place block in workspace.
LAUREN MILNE: This block is looked towards voice.
So it'll actually, when it gets to that block on the program,
it will listen for your voice, and then look for it.
You have to time it perfectly, though.
IPAD: Toolbox menu.
Back button.
Look category.
Motion category.
Double tap to explore blocks in this category.
LAUREN MILNE: Motion.
We have some more complex options.
IPAD: Toolbox options.
Back button.
[INAUDIBLE]
In toolbox.
Double tap to place block in workspace.
Non.
LAUREN MILNE: And non.
IPAD: In toolbox.
Double tap to place block in workspace.
Toolbox menu.
Back button.
Motion category.
Double tap to explore blocks in this category.
LAUREN MILNE: We were sort of expanding-- as I said,
we wanted to make the output really accessible
for students with visual impairments.
So we have lots of sound making blocks.
IPAD: Voice category.
Sound category.
Speed category.
LAUREN MILNE: And speed.
IPAD: Variables category.
Double tap to explore blocks in this category.
LAUREN MILNE: Very similar.
IPAD: People category.
Functions category.
Double tap to explore blocks in this category.
LAUREN MILNE: Something I want to talk about
that we just added were the ability to create functions.
So something that is a limitation of Blocks For All
is that, essentially, you only have one island of code
because we located all of the blocks
on the bottom of the screen.
And in a lot of block-based languages,
you can have sort of multiple groupings of blocks.
And in particular, you can create functions, right?
So a separate grouping of blocks does a specific action.
And because we wanted all of our blocks
to be on the bottom of the screen because, as I said,
we wanted to sort of take advantage
of this spatial nature of how the blocks are laid out.
And the only way that people without vision could find it
is if it was located on one edge of the screen.
We basically were limited to one island of code initially.
But we ended up adding functions.
And essentially, you have multiple different workspaces
or functions that you can get to to sort of account for this.
So I'm go ahead.
I pre-made a function.
So I'll go ahead and add that function block to my program.
IPAD: Toolbox menu.
Back button.
Party.
LAUREN MILNE: The party function.
IPAD: Double tap to place block in workspace.
Party selected.
Select location in-- place party before and repeat.
Workspace block four of five..
LAUREN MILNE: And add that to my block.
IPAD: [INAUDIBLE] locking work space.
LAUREN MILNE: But how I created that, we
see at the top of the screen there's
a Main Menu button which takes you
back, lets you do some settings, and add the robot.
And then there's a Functions menu.
IPAD: Functions menu.
Button.
Click to create [INAUDIBLE] Return to main workspace.
Button.
LAUREN MILNE: And essentially, the Function menu
allows you to create new functions,
or add a new function, or whatever.
So I already have one function on here, called Party.
IPAD: Party.
Button.
Party.
Toolbox.
Heading.
LAUREN MILNE: But if I click on Party,
you can see I went to a new screen that's
populated with totally different set of blocks
that I had originally created.
And so this screen.
IPAD: Function start.
Workspace block one of seven.
Double tap to move block.
LAUREN MILNE: Starts with a function, and all the blocks
are nested inside of the function.
This is--
IPAD: Set left ear light purple.
Workspace module seven.
Purple button.
Double tap to set light color.
LAUREN MILNE: Changes left ear light.
IPAD: Orange button selected.
Orange tap button tap tool box.
LAUREN MILNE: Et cetera.
So these are all encapsulated in my Party function.
So if I just go back.
IPAD: Functions menu.
Button.
Return to main workspace.
Return to main toolbox.
LAUREN MILNE: To main workspace.
And I press play.
Let me actually go here.
IPAD: Make dinosaur noise.
Workspace block three of six.
Make dinosaur noise.
Workspace block Place make dinosaur noise before party.
Workspace block three of five.
Place in trash.
Button.
Play.
LAUREN MILNE: Right back to the end.
I should make a cat noise, and then I should start partying.
So let me press play
RICHARD: Four times.
[CAT MEOW]
LAUREN MILNE: So oh-- you can't see the party because he's
changing light colors.
Sorry.
Let me stop sharing this.
Can you see it now?
[TOY CHATTERING]
It's making-- oh, yeah.
There was a light change, and it was making those weird robot
sounds.
It's like the party sound.
And it will continue to do so unless I press stop
[TOY CHATTERING]
Wow.
RICHARD: I have a question.
LAUREN MILNE: Yes?
RICHARD: Suppose your program runs beyond the screen--
you know, on bottom.
What happens?
LAUREN MILNE: Oh, yeah.
So I'll add some blocks to the--
[TOY CHATTERING]
It's still partying.
Let me stop this.
IPAD: Play.
Animals category selected.
Make cat noise.
Make cat noise selected.
Place Make cat noise at end.
Toolbox menu.
Back button.
Make crocodile noise.
Place make crocodile noise before end repeat.
Toolbox menu.
Back button.
LAUREN MILNE: So now my program runs beyond the screen.
Essentially, this is a scrollable view, on the bottom.
So there are a couple of ways you
can navigate it with Voiceover.
The easiest is essentially with Voiceover,
you can either sort of touch things
and they'll be read aloud on the screen,
or you can swipe through all of the elements on the screen.
And so, like, if, for example--
IPAD: Make cat noise.
Workspace block seven of seven.
LAUREN MILNE: I select this one--
IPAD: Double tap to move block.
Play button.
Make cat noise.
Workspace block seven of-- page three of three.
Page one of three.
Make crocodile noise.
Make crocodile-- place make crocodile noise before
make cat noise.
Workspace toolbox menu.
Back button.
Make crocodile noise.
Work-- make cat noise.
Workspace block eight of eight.
Double tap to move block.
LAUREN MILNE: [INAUDIBLE].
So essentially, if I swipe between elements,
it will automatically go to the next element, or you can--
I think it's a three finger swipe
you can do on the workspace, that will sort of shift it
over a page.
But yeah-- so you can access blocks
off the edge of the screen.
What else?
We added some variables--
IPAD: Toolbox menu.
Back button.
LAUREN MILNE: --recently So--
IPAD: Toolbox.
Heading.
Motion category.
Noise category.
Sound-- Speed category.
Variables category.
Selected.
Toolbox menu.
Back button.
LAUREN MILNE: And here--
so Wonder Workshop, the people who make the robots,
have their own app.
And we wanted to basically add in all the functionality
in our app that they had on theirs.
They do, like, a robotics competition every year.
And we talked to a team that wanted
to compete with some visually impaired students.
And so we wanted to add in essentially
the same functionality they have.
And one of the important things for this robotics competition
was be able to set variables.
So we now have, essentially--
IPAD: Set variable.
In toolbox.
Set variable selected.
LAUREN MILNE: Set variable block.
IPAD: Select location in-- place set variable before
make crocodile noise.
Set variable.
Set variable orange set to zero.
Workspace block seven of nine.
LAUREN MILNE: And the variables were--
IPAD: Double tap to move block
LAUREN MILNE: --fruit themed.
But essentially--
IPAD: Orange equals zero.
Button.
Double tap to set variable value.
Orange equals-- back.
Button.
LAUREN MILNE: I can pick one of these five variables, all fruit
themed.
So cherry--
IPAD: Cherry button selected.
Cherry.
Set your variable value.
Text field.
Double tap to edit.
Set your variable value.
Text field.
Five.
Six.
LAUREN MILNE: And then I can set--
IPAD: Set your variable value.
56.
Text field.
LAUREN MILNE: --set this variable to 56.
IPAD: Back button.
LAUREN MILNE: And it's the same, just modified [INAUDIBLE]
block.
IPAD: Toolbox heading.
Set variable Cherry set to 56.0.
Workspace block seven of nine.
LAUREN MILNE: So now Cherry is set to 56.
IPAD: Double to move block.
LAUREN MILNE: And I'm just going to clear out my workspace.
IPAD: Repeat four times.
Workspace block two of nine.
Repeat four times.
Place in trash.
Button.
Play.
Motion category.
Noise category.
Variables category selected.
Toolbox menu.
Back button.
Look up or down.
LAUREN MILNE: And I can--
IPAD: Toolbox.
Double tap to place block in workspace.
LAUREN MILNE: Now I can use that variable
to do a couple of different actions.
So I can move forward by Cherry amount,
or turn right or left by Cherry amount, or, you know,
move dashes ahead by Cherry amount, up or down
or left or right.
So--
IPAD: Look up or down selected.
Select location in workspace to place it.
Place look up or down before make cat noise.
Workspace block five of five.
Look up or down.
Set look up or down variable.
Button.
Look up or down-- back.
Cherry button selected.
Cherry back button.
Back toolbox heading.
LAUREN MILNE: So what I just did,
I set the variable Cherry equal to 56.
And then later in my program, when I get to the look
up or down block, it'll check what
that variable has set that to, and look up or down
in that amount.
So that's what it is.
RICHARD: Uh, Lauren-- question.
Do these variables have scope, or are they global?
LAUREN MILNE: They're global.
[LAUGHS] Which, I think is--
yeah.
They're global.
[LAUGHS] And I actually think it's--
[INAUDIBLE]
So the other thing I wanted to point out, something
we've worked with is--
and we haven't actually tested this
with people who use this control-- but in addition
to being accessible with Voiceover,
it works well switch control, and it's mainly
because if something is accessible with Voiceover,
it's usually pretty accessible with switch control.
They use it at the same grips into the program.
So switch control, essentially, you
can hook up your iPad to some external switch.
So maybe you can only use a button, or something,
or there are all sorts of different switch controls.
And you can configure your iPad to work with whatever switch
control you use, or configure it to work
however you want it to use--
to use.
So I'm going to just demo it with switch control,
essentially, where it will sort of iterate over
selectable items on the screen, and I just tap the screen once
to get it to stop there and select something,
if that makes sense.
So traditionally, you'd probably have some external device
instead of tapping the screen.
But this should just demo how it works.
So I'll switch to switch control.
IPAD: Alert.
Accessibility-- Switch control.
Button.
Voice over auth.
LAUREN MILNE: So if you see now, there
is this blue outline that's iterating over the screen.
And if I select--
so I just selected the menu.
And now it's iterating over items on the menu.
This is going-- OK.
Not enough of a--
[INAUDIBLE] switch control.
[COMPUTER BEEPS]
So I can essentially place items in the workspace
using the switch control.
So once again, this is sort of--
if you only are able to do some kind of, like, binary input,
or timed input, it should work for you.
Although, as I said, we haven't tested that.
We'd like to work with people who use switch control to make
sure it does actually--
RICHARD: Can I add a little bit about the switch control?
So this is built into the Apple products.
And if you have an application that has a bunch of items--
for example, on the screen, like this--
then once you activate the switch,
then it will just scan through the items
in a hierarchical fashion.
For example, at one point, it selected the whole program,
and then if you select at that point,
or switched at that point, you get to go into the program.
So there is a natural tree structure.
And the organization-- for programming
using a switch, the obvious research question is, you know,
what should be that organizational structure
to make programming as easy as possible for somebody that's
using a switch?
Especially young people.
So is that, Lauren, pretty much what--
LAUREN MILNE: Yeah.
Yeah.
[INAUDIBLE] Yeah.
Sort of scanning through the program.
There is an alternative switch, where you can sort of scan
from left to right and top to bottom,
so you can sort of select your x- and y-coordinate
with the switch.
Yeah.
There's all sorts of different ways of doing it.
RICHARD: But I think the hierarchical way
makes a lot of sense for this.
LAUREN MILNE: Yeah.
RICHARD: You know, I go to the tool box,
and then you want to go to the program, and then back again.
You know?
That kind of thing.
You don't want to go through every item in the tool box.
LAUREN MILNE: Yeah.
RICHARD: You know, that kind of thing.
So you know, that has to be tested with children
to see how well they can do programming using a switch.
In Minnesota, there is an organization called-- what's
the name of the organization?
LAUREN MILNE: The Pacer Center.
RICHARD: The Pacer Center-- which I visited at the last 6C.
And they have a lot of children who use switches there.
So hopefully that will work out to be a nice research project.
LAUREN MILNE: Yeah.
Yeah.
So yeah.
I also want to mention--
so you can sort of tell, this is very
much like a work in progress.
Like I said, I have students working on it.
So things like the functions and the variables,
we sort of added the functionality.
But I think there's still some improvements
we can make on those.
So we're working on that.
RICHARD: Are you ready for questions, Lauren?
LAUREN MILNE: Yeah.
I'm ready for questions.
I just want to mention, if you're
interested in beta testing, or just trying out the app,
you can send me an email.
It's on test flight, which is, like, the beta testing
service from Apple.
Yeah.
And let me know if you have questions or suggestions.
I'd love to hear them.
BRIANNA: I'll drop Lauren's email in the chat,
if anybody wants it.
If that's OK with you, Lauren.
LAUREN MILNE: Yeah.
SPEAKER 1: I have a question.
So you showed using the Cherry variable block
as the input to the move head up or down.
Can you also put, like, a number literal-- like the slider thing
that you showed with the repeat block in there?
Or is it just a variable input?
LAUREN MILNE: Sorry.
For specifically, like, the move head up or down one?
SPEAKER 1: Or, like, any of the blocks
that accept the variable parameters.
Could you also put just, like, a number directly,
or does it have to be through a variable?
LAUREN MILNE: No.
It has to be through a variable.
But we do have, basically, the same block.
So there's like an up look up or down block
that's in a separate place on the menu, that takes
a number from a slider menu.
So they can find that.
[INAUDIBLE]
Like, I think this is just the standard--
well I'm not sure [INAUDIBLE] say this.
But there is one look up or down block that you can just
put a number slider.
But it's a separate block.
Which is the same way that I think the dash--
the Wonder Workshop app-- works.
They have these separate variable apps.
So good question.
SPEAKER 2: Lauren, would it be possible
for you to show how you're interacting with your iPad
with your hand or finger?
If possible.
[INTERPOSING VOICES]
LAUREN MILNE: So maybe I'll stop sharing my screen
and maybe I'll just--
SPEAKER 2: Yeah.
LAUREN MILNE: ---put my iPad like this.
[LAUGHS] All right.
There's that.
Let me turn on Voiceover really quick.
RICHARD: In the speaker view, you
can change it to showing her--
right.
So it can be bigger.
LAUREN MILNE: Oh, yeah.
So you see.
OK.
And it will be less loud, because I
think the audio doesn't get shared this way.
But yeah-- I can kind of see what I'm doing.
So essentially, as I touch things,
they should be read aloud.
RICHARD: I can't hear it.
LAUREN MILNE: Yeah.
So the audio is not shared, because I'm not
sharing the screen.
There.
Volume is all the way up now.
IPAD: Make cat noise.
Workspace block one of nine.
Double tap to move block.
LAUREN MILNE: Could you hear that?
BRIANNA: Yeah.
RICHARD: Yes.
LAUREN MILNE: OK.
So once again--
IPAD: Repeat forever.
Workspace block two of nine.
End repeat forever.
Workspace block three of nine.
Drive forward 60--
60-- 60 centimeters.
Fast button.
Double tap to set-- set variable Cherry set to 56.0.
Workspace block five of nine.
Double tap to move block.
LAUREN MILNE: So that's me just sort of by touch--
I can touch anywhere on the screen.
IPAD: Move left.
In toolbox.
Double tap to place block in workspace.
LAUREN MILNE: And I will read what's under there.
And then, to select it, you can double
tap anywhere on the screen.
So if I double tap right now, it should select
whatever I was just looking at.
Oops.
IPAD: Drive forward selected.
Select location in workspace to place it.
LAUREN MILNE: I didn't quite double
tap fast enough, because I'm doing it backwards.
[LAUGHS] But OK.
So now, if I select--
I would listen to select where I want it to go.
So say I want it inside the repeat block.
IPAD: Place drive forward before repeat forever.
Workspace block-- place drive forward before
repeat the [INAUDIBLE].
Workspace block three of eight.
LAUREN MILNE: So that's where I want to--
IPAD: Double tap to add drive forward block here.
LAUREN MILNE: --place it.
Then I can double tap.
IPAD: Toolbox menu.
Back button.
LAUREN MILNE: And it's now inside there.
And then the other thing that I want to note
is, you can either sort of do this
by touch, so you sort of know where things are, or--
IPAD: [INAUDIBLE] in tool box.
Double tap to place block in workspace.
LAUREN MILNE: --you can swipe through,
so this will go to the next element.
IPAD: Look straight.
Look left.
Look right.
Look forward.
LAUREN MILNE: The next selectable element
on the screen.
So that's Voiceover.
Did you want to see switch control?
Or does that answer your question?
SPEAKER 1: Yeah, that's fine.
I just wanted to see how you were actually
activating and moving things.
LAUREN MILNE: Yes.
SPEAKER 1: Thank you.
LAUREN MILNE: Mm-hmm.
RICHARD: So just to repeat, there are two methods.
One was one finger explore, which is standard in Voiceover.
And the other one is flick through a list,
which she did secondly.
And those are both standard in Voiceover.
LAUREN MILNE: [INAUDIBLE] Yeah.
And if people are interested--
I mean, so the other thing we kind of talked about
was the rotor movements, that we found kids had trouble with.
Which is, it's like you put your hand
like you're turning a rotor.
So if you think about, like, old iPods,
when they used to have rotors on the--
IPAD: Containers.
Headings.
LAUREN MILNE: And so typically, on iOS apps,
there's, like, a menu of options that you
can access through the rotor, that tends
to be specific to the app.
But yeah.
As I said, that gesture was challenging
for a lot of people-- the kids we were working with.
So--
RICHARD: I think if they were, you know,
a Blocks4All for older kids-- which, you know,
I would rather have them learn Quorum,
personally-- but you know, maybe the rotor could come back
and be a way of changing modes.
LAUREN MILNE: Yeah.
I could definitely imagine-- yeah,
you could maybe have a set of settings, or something,
like, if you're more advanced, or-- it's sort of
built into it.
But yeah.
I think it seems like--
to us, I guess, or to me, at least--
the utility of these block-based languages
seem to be more useful.
I mean, especially for thinking about children, or people
with visual impairments.
It seems more important for the younger children,
and perhaps older kids could learn text-based languages.
ANDREAS STEFIK: For-- just a technical question.
For the dash robot, itself, like, how hard
was it to get code integrated into it?
Like, how do you connect to the robot?
LAUREN MILNE: Well, it's pretty easy.
They have, like, an open API, and they've been really-- like,
we've talked to them.
Yeah.
They've been really open about sharing code,
and sort of opening up functionality to us.
ANDREAS STEFIK: Oh, really?
LAUREN MILNE: Yeah.
Yep.
They've been great.
But I think they have--
ANDREAS STEFIK: Is it, like, Bluetooth calls, or like--
LAUREN MILNE: Yeah.
It's Bluetooth calls.
Yep.
ANDREAS STEFIK: OK.
LAUREN MILNE: It's over Bluetooth.
And they have iOS.
I think they have Android API.
So yeah.
That's right.
[LAUGHS] I can show you the code.
I of course found a bug, like, an hour before.
[LAUGHS]
ANDREAS STEFIK: That's the reality of [INAUDIBLE]
LAUREN MILNE: [INAUDIBLE] [LAUGHS]
ANDREAS STEFIK: You'd expect nothing less.
In fact, if you didn't have a bug an hour before,
we'd be more suspicious.
LAUREN MILNE: [LAUGHS]
BRIANNA: How expensive are the dash and dot robots in terms
of schools getting them?
LAUREN MILNE: Good question.
I think they're maybe $100--
the blue one, which is actually the more fun
robot that moves around--
I think it's maybe $120, or something.
BRIANNA: That's not much.
LAUREN MILNE: Yeah.
And then this little green one is a lot cheaper.
Yeah.
Should get kickback, or something, for advertising
for them.
But yeah.
The other cool thing-- so I mentioned
about making the output accessible.
And I did test with, like, five-year-olds,
and I will say that they--
one of them, in particular, was, like, throwing the robot,
and playing around.
And in general, yeah, they're just very robust robots.
So, like, I did a lot of user testing
where they move the robot around,
and you could feel the robot as it was moving.
It's not, like, a delicate robot.
RICHARD: Lauren, this is Richard.
Bryan has a question.
Have you considered whether and how
this should be used in an inclusion classroom setting?
That would be, I guess, sighted and blind students
in the same classroom-- which is a common case.
Would you recommend using the app for all students?
And what student-device ratio would you recommend?
How many different devices do you need for the students?
LAUREN MILNE: So I actually did--
I did a little bit of testing--
[DOG BARKS]
Sorry.
My dog's barking.
RICHARD: Is that your dog?
LAUREN MILNE: [LAUGHS] But it wasn't an inclusive class.
It was at the Washington State School
for the Blind is where I did some classroom testing.
And there, I found not having, maybe,
like, two students on a device was about good enough.
I think it's just hard to share an iPad
with more than two students.
But I did-- I mean, as you can see,
the app does have a lot of visuals,
so it would work with some who is sighted.
They'd also be able to use it.
And on a lot of these that are testing now--
a lot of times, at the end of the session,
I'd work with kids who had siblings who were sighted,
and they would come and join, and sort of
build the app together.
But yeah.
I don't have a ton of data on that.
But I think having around two kids,
I think, should work just fine to have visually impaired
student and sighted students.
The sighted student would--
I mean, if they have Voiceover on--
would have to sort of get used to the Voiceover.
But I think that's not--
RICHARD: Bryan, I would just add that if there
were multiple robots in the class,
you know, you get the choice.
You know, all those robots will show up on a screen,
and you get to choose the one that you're using.
And other people could use other ones.
So there's not going to be a conflict.
There could be a conflict, but making sure people
choose different robots as their destination.
LAUREN MILNE: Yeah.
And I think that the robot can only
be connected to one device.
Each robot can only be connected to one device at a time.
Technically, I allow you to connect multiple robots
to the same device.
But yeah-- you could have multiple robots
and iPad combinations.
Good question.
I should ask my husband.
So my husband is actually a sped teacher
who does inclusive classrooms.
And I think he's using-- he hasn't used this app,
but he's used the dash robot app.
And I think he's been able to deal with multiple--
RICHARD: Multiple robots.
LAUREN MILNE: Yeah.
RICHARD: It would be kind of interesting to have
people write a program where the robots are interacting.
And I wonder if that's possible.
LAUREN MILNE: Yeah.
Yeah, they definitely are.
So, as I kind of showed you, the block-- like,
it can turn towards a voice.
You can definitely have the robots--
there's a couple of different sensors.
It can listen for noises.
On the blue robots, it can detect if they're
about to bump into something.
So you could sort of imagine an interaction like that.
BRIANNA: I just dropped a link in the chat
to the video we made a couple of years ago.
So if anyone is curious to see a student using
it-- do you want to say a little bit more about Boone, Lauren?
LAUREN MILNE: Oh, yeah.
He was a fantastic student who-- yeah,
he was definitely a Voiceover expert.
And he was-- yeah, he was great using the app,
and he could build very complex programs.
And picked up programming very quickly.
RICHARD: In fact, this video features
Boone and his parents, actually, talking about their son.
LAUREN MILNE: And I will say-- so he had a younger brother who
really enjoyed--
they played together.
usually at the end of sessions, they would--
his younger brother was sighted, and they
would build programs together.
RICHARD: So BRIANNA, is that linked to the STEM
For All video showcase?
BRIANNA: No.
That's just on our website.
I can find it on the STEM For All.
RICHARD: And the reason to go to the STEM For All video showcase
is that you can see, you know, hundreds-- actually,
there's this--
I forget what it's called, now.
They merged them all together-- like, five years of videos.
So there's, like, 1,000 videos.
And use that to search for Blocks
For All to find this one.
I think it's called Multiplexer, something like that.
STEM For All Multiplex.
Do a search.
And then we could send it an email, later, too.
What's neat about that is that it
tracks, around the world, who's actually
watching different videos.
And I've noticed that our video has been watched by over 1,000
different people, in--
I don't know--
50 countries around the world.
So yeah.
Lauren is becoming famous.
LAUREN MILNE: [LAUGHS] Well--
yeah, I don't know.
I mean, I think the frustrating thing
is that there are very few accessible programming
languages and programming apps, especially for kids.
ANDREAS STEFIK: Yeah.
Especially at young ages.
Right?
Because, I know that, you know, Bootstrap
has been adding some stuff in for Blocks
to make it accessible, which is great.
But it's definitely geared to older-- not younger,
as yours is.
I was going ask--
I know you're on--
I forgot what they call it on Apple.
Test Flight, right?
LAUREN MILNE: Yeah.
ANDREAS STEFIK: That's what you're on right now.
Are you planning on making that into something
more official on Apple, so that people can use it
or what are your plans?
LAUREN MILNE: Yes.
Yeah, I would like to--
so this is my second year of teaching,
and my first year working with students.
I think there's a lot of debugging
that I need to do before sort of releasing it
widely outside of the beta one.
And the nice thing about Test Flight
is you can collect error reports and stuff pretty easily.
But yeah-- I'd like to deploy it on the appstore.
ANDREAS STEFIK: Because I know that--
I ask because I know there's a lot of teachers
in the blind and visually impaired community nationwide
that, you know, like you said, once they kind of get
their kids up to a certain age-- like, at least,
like, middle school, or something-- you know,
the kids can read, and like, they
can use a screen reader-ish.
Right?
So a text-based language isn't so bad.
But like young kids, there's very little to help them.
The only other project I know of is Code Jumper.
But it hasn't even been released, yet.
LAUREN MILNE: Yeah.
ANDREAS STEFIK: Schools are kind of in a bind.
I know there are some schools that
try to do, like, LEGO robots in elementary school.
But it's just real tough at that age.
LAUREN MILNE: Yeah.
And I know some schools where they were actually, like,
3D printing--
you know, they'd have their visually impaired students
just use printed blocks.
But they weren't actually programming.
You know, they were doing it, and then
they'd have a sighted student, like, transcribe the thing.
Yeah.
And I mean, I have had-- feel free to redirect people to me.
But I've had a number of teachers,
you know-- a lot of teachers are doing the beta testing.
ANDREAS STEFIK: I was going to say,
I was going to encourage you-- even if it's imperfect,
if you get even, like, you know, the top 10 most devious bugs
out, even having an imperfect release
might actually help teachers.
LAUREN MILNE: Yeah.
ANDREAS STEFIK: You know, it might
work for the common cases, or something,
and have some other flaws.
So I wouldn't-- you know, I wouldn't worry too much
if there's a couple of bugs.
It does happen.
LAUREN MILNE: Yeah.
RICHARD: I think what Stefik is saying--
and he's been through this many times--
he just released Quorum Studio.
Of course, it went through a lot of early testing
with a closed group, like something like Test Flight.
And they got a lot of bugs out.
But then they released it, and sure enough, there
were more bugs.
ANDREAS STEFIK: Yeah.
And one was a crash bug, and that makes me really mad.
LAUREN MILNE: Oh, yeah.
We have a good crash bug.
But it only appears if the app, like, disappears from memory--
which it takes, like, a couple of months for that to happen.
So it only appears, like, if it's been on your phone
in particular situations for a couple months.
So it--
ANDREAS STEFIK: Now, that sounds like it's deploy ready.
I mean, reality is nothing's perfect.
You know?
Even if we spent all day, every day doing only bug fixing,
we'd never get to new features.
I mean, it's complicated.
Any kind of engineering like that is a tricky--
RICHARD: There's something called "diminishing returns."
You know, after a while, you've got to put it out, and then
get ready for, you know--
ANDREAS STEFIK: I mean, only you know if it's ready.
I'm just saying, I know that there's interest.
So--
LAUREN MILNE: Oh, good.
Yeah.
I think-- yeah.
We'll see.
I have an interesting semester, but I'm
going to try to get it out soon.
RICHARD: One thing Lauren and I think that-- and we
talked about this in the past-- is to create an hour of code
based on this, and people would have to, of course,
have the robot.
You're not going to provide that.
But once you have the robot, then
you just download the program.
And by then, you should have it public.
It would be-- anybody could download it.
LAUREN MILNE: Yeah.
RICHARD: And I think Apple would be very helpful in promoting
it, because they are an accessibility
focused organization.
And so I'm sure they can help with sort of--
maybe even doing some of the debugging before it comes out.
LAUREN MILNE: Yeah.
ANDREAS STEFIK: Also, I'll say one last thing.
Just thank you for doing this work.
Because, like, we need more of it at that age.
So just from one person in the community,
I appreciate the work you do a lot.
LAUREN MILNE: Thanks.
RICHARD: Are there any other questions from anybody?
We're close to the witching hour, here.
The 10:00 hour.
BRYAN DJUNAEDI: This is Bryan from code.org.
Can you hear me?
RICHARD: Yeah, Bryan.
Go ahead.
BRYAN DJUNAEDI: Hi.
I had a question about methods.
When you're about reading Blocks For All,
or any other visual coding language or tool for students
with visual impairments, what do you
consider when you're thinking about how effective it is?
So, like, do consider the speed at which they complete a task,
or if they get stuck, or--
like, what dimensions do you, like, I guess, use to quantify
or qualify the success of the product?
LAUREN MILNE: Great question.
And I'll say that this is a really interesting population
to work with, because it's very small.
So, like, for example, when I was doing my initial testing,
and I wanted to work with sort of academic Braille readers--
so this would be students who are visually impaired enough
that they would read Braille, you know,
probably using a screen reader--
but are sort of at grade level for studying.
I think there is something like 12 of them
in the entire Seattle public school system.
You know-- from K to 12.
So I ended up testing with kids in a pretty wide range
of ages, a relatively wide range of visual impairments,
and some of the students were sort of at grade level,
and some of them were a little behind grade level.
So it's really--
I collected, like, quantitative data in terms of,
like, how long it took them to build-- you know,
I had certain tasks.
Like, they had to build a program.
I think I had some debug tasks.
Like, can they figure out why the robot is doing--
you know, isn't going in a square, or something like that.
But--
RICHARD: You also had a task where
they had to explain what the program does.
LAUREN MILNE: Yep.
Yeah.
And mainly what I would end up reporting is sort of the more--
I was more interested in can they actually do it
once they've been trained, and less in, like,
how long exactly it takes them to do something.
You know, sort of the benchmark is they can't--
currently, you know, there is no other sort
of accessible program out there.
So there was-- yeah.
Sort of, "is it possible?" was the first question
we were really trying to answer.
RICHARD: So Bryan, there is a paper in the CHI Conference
2018.
And maybe we can send a link for that,
as well, around to everybody.
BRYAN DJUNAEDI: Oh, cool.
Yeah.
RICHARD: It talks about how the evaluation was done.
ANDREAS STEFIK: For evaluating any kind of programming
languages in general, too-- oftentimes, when
the feature is brand new, what Lauren
did made sense to my brain.
You try it first.
And then, once you do that, if you want to make it better,
you can do comparative studies.
But people don't usually do that till the next step.
That's what I usually see.
So, like, I can imagine her doing
a study on her function thing, or something,
or comparing one--
whether it's at the bottom with the tactile perception,
or in the middle, which I imagine makes sense.
I mean, one or the other might work out,
depending on the group.
But I imagine that sort of thing would have
to be done after the initial--
hey, can anybody use this thing at all?
RICHARD: So Brianna put the link to the paper in the chat.
And it's behind the firewall-- the ACM firewall,
or whatever you call it.
Paywall.
So, you know, people at universities and places
like Google, and other places, should
be able to get to it pretty easily.
BRYAN DJUNAEDI: Awesome.
Thank you.
BRIANNA: I think we are right about at our time.
Thank you, Lauren, for taking the time to put this together,
and sharing with us.
And thanks, everybody else, for coming.
RICHARD: And Lauren, that was a great demo.
Thank you.
BRYAN DJUNAEDI: Thank you.
LAUREN MILNE: Anyone can feel free to email me.
ANDREAS STEFIK: And congrats for another reason that no one
said.
RICHARD: OK.
Bye, everyone.
LAUREN MILNE: Bye.
BRIANNA: Bye.
BRYAN DJUNAEDI: See you.