BRIANNA BLASER: So thanks, everybody,
for joining us today.
My name's Brianna Blaser from AccessCSforAll .
And as many of you know, we like to get together
to talk about accessible technology for computer science
education.
And so we're excited to have Janette and Jordyn, from Apple,
join us today to talk about Swift Playgrounds.
And Richard is going to give an introduction.
RICHARD LADNER: Yeah.
I don't know Janette, but I do know Jordyn because many years
ago she participated in one of our programs
with access computing, led by Jeff Bigham.
And that was her first introduction
to programming in computer science.
And now she's at Apple after graduating from, I forget,
Michigan State.
Is that correct, Jordyn?
JORDYN CASTOR: Yeah.
You got it right.
I was afraid.
I was like, he's going to say Michigan.
He's going to say Michigan.
And I was like, no, don't do it.
RICHARD LADNER: And then, Janette,
maybe you could just say a little bit about yourself.
JANETTE BARRIOS: Sure.
Happy to.
But actually, Richard, I think you better
describe your background for Jordyn first.
Because it's just so cool.
RICHARD LADNER: Well my background.
I don't exactly know what it is.
The sun rising above the earth as
seen from the moon, or something like that.
JANETTE BARRIOS: Yes.
I thought that she would appreciate that.
It's very sort of stellar and very cool.
So yes.
I'm Janette Barrios.
My role is accessibility marketing and community manager
at Apple.
And so I help support getting our accessibility messages
out there and have gotten to work with Jordyn on the coding
programs, specifically, and some of the resources
we'll share with you.
So happy to be here and meet you.
A very legendary person not only Jordyn's life,
but in really bringing STEM education forth
with accessibility and as well as others in the call
that are committed to this as well.
Super nice to meet you guys.
RICHARD LADNER: And Jordyn, I didn't give you
a proper introduction.
Maybe you could say a few more words about yourself.
JORDYN CASTOR: Yeah, sure.
So I think your introduction sounded great.
I don't have too much to add.
The first program that I attended was,
yeah, was Jeff Bigham's program at the 2007 NFB Youth Slam.
And so that was a STEM academy that
was put on by the National Federation of the Blind.
And Jeff and Richard were teachers there.
And I wrote my first computer program.
And it was probably just like this little chat
bot with a couple of function calls.
You know, like no big deal.
But to me it was a huge deal.
It really opened up my world and taught me that, you know,
I could participate in computer science.
I learned I loved technology ever
since I was in second grade and our family
bought our first computer.
And so ever since then I was like,
man, I have to do something with technology
growing up and in college.
And so I attended this STEM academy.
And I was like, oh my goodness.
This is so empowering.
And from then I just knew like, this
is what I wanted to do with the rest of my life.
And so, yeah.
I went to Michigan State and was the first blind person
to graduate with a computer science degree
from Michigan State.
And then I went to pursue my computer science career.
So it was all thanks to Jeff and Richard for getting me started.
RICHARD LADNER: Thank you for that accolade.
OK.
Brianna should we just let them go?
BRIANNA BLASER: Yeah.
I'll turn it over to you all now.
Thanks.
RICHARD LADNER: Swift Playground.
JANETTE BARRIOS: All right.
So here's our official titles on this opening slide.
We've kind of already introduced ourselves
so we'll move forward here.
But Jordyn, I think you also gave some of your intro here.
But I wonder if you might just share, you know,
your title was there on that first slide, but just maybe
a little bit more about, you know, your role
and working on some playgrounds or whatever
you're comfortable showing up front here.
And just kind of your job in a sense, right?
JORDYN CASTOR: Yeah, absolutely.
So you know, right now my job at Apple--
so I've been working a lot with voiceover and Braille support
for our various products.
And so currently I've been doing a lot of work
with Swift Playgrounds.
And Swift Playgrounds is an amazing app
that teaches people how to code through sort
of game play sort of role.
And so my job in that was to make sure
that Swift Playgrounds was accessible to people
who are blind, not only in the aspect of voiceover,
but also with Braille support.
And so it's honestly been one of the most rewarding things
that I've worked on at Apple so far.
And I've also been working a lot on voice
over for Braille support for Mac and iOS.
And it's a dream come true for me, honestly.
It's a very humbling experience.
JANETTE BARRIOS: That's awesome, Jordyn.
And I can say, you know, working at Apple and seeing Jordyn
as well as other engineers involved in accessibility are
just so committed to making that out-of-the-box experience.
When you have that iPhone in front of you,
have that iPod in front of you, you're
setting it in front of a student or sending it
to a student in our world today, knowing
that they can pick that up, turn on voiceover
or any of these other features and just
be able to use it and have it work.
A lot of that is due to the work of these engineers
that, you know, run these things through the paces and test.
And so, as we dive into this, I think
Jordyn's just such a great example of what's possible when
you give students the tools.
So I'm going to dive a little bit here
into this topic of coding and, why is it important?
You know, we know that these are skills that are in demand.
The World Economic Forum talks about how
creativity, critical thinking, analysis, learning, leadership.
All of this can come out of that training and computer science.
And I think for students with disabilities,
this also might mean literacy.
Right.
Some of these core competencies that we all
want people to have that we might take for granted,
whether that's actually learning to code in Braille, right.
Using Braille just as a way to practice how to code.
Or that can mean different things for different people.
With sign language, right, it's learning the signs
for the programming language.
So that can mean literacy.
But it also means jobs.
And we know that there's a major employment challenge here.
So according to the NFB, for working age adults
reporting significant vision loss, over 70%
are not employed full time.
That's just a shockingly large number
and doesn't need to be that way.
There are over 500,000 open computing
positions in the United States.
This is from co.org.
For 2020, an estimated one million computer programming
related jobs in the US are expected to be unfulfilled.
So this is a huge job opportunity.
And I think a lot of people just don't know what this means.
Living in Silicon Valley and normally
where Jordyn and I sit in our offices,
we are surrounded by engineers, builders, designers, coders.
And it's what we live and breathe.
But we know that's not the norm everywhere, right.
People have just no or very little exposure
to what coding is for one, but two, what are the jobs
that you could have?
And so we wanted to call it up front.
Like this isn't just about of course,
the amazing engaging learning that we want students to have,
it's also about employment in the long term.
And it's not just about jobs.
Right, Jordyn?
JORDYN CASTOR: Absolutely.
So yeah.
It's not just about jobs.
There's so many different skills that a student
can gain from coding.
So for me, you know, I see coding
as kind of like an art form.
And so as people with disabilities,
I think sometimes it's hard to, you know,
express ourselves creatively.
Especially, I feel like for me as a blind person,
I forgot to mention that.
But for people who don't know me, I'm totally blind as well.
A whole lot of you don't know who I am.
It's interesting because normally you're, you
know, in a room with a bunch of people.
And now I'm just on Zoom.
But, yes.
I'm totally blind.
And so I see coding really as this art form
where you can express yourself creatively.
You can lay out your functions, and your classes,
and things how you want.
You can design soundscapes.
You know, any problem that you have,
you can design your own creative solution to that problem.
And coding is really cool because, you know,
not only does it give you that really creative format
of expression, but it really helps you
with other skills outside of programming.
So, you know, in the life of someone who is blind,
a lot of times you have to think about mental maps,
and cardinal directions, and you know,
routes to get from point A to point B,
and having that exploratory nature.
And so with coding, you're kind of laying out
this mental map in your mind, how are my functions laid out?
You know, how do I want to create this file?
You practice your Braille skills when you're coding.
Brailling while coding and reading your code in Braille
allows you to feel how the syntax is laid out, to feel
how the punctuation is--
I have a cat and she's decided that she really wants
to come into the FaceTime.
Deciding how things are laid out.
And students, you know like I said
before, they're giving specific routes how to get from point A
to point B. But in the real world, you know,
it's about exploration.
It's really about, how do you explore to get from one point
to the other?
And so with Byte and with Playgrounds
you're really exploring the screen
and laying out the mental map of how to solve those problems.
So you're learning and you're keeping up
a lot of different skills as you're learning to code.
JANETTE BARRIOS: So Jordyn, I think
that's the perfect segue to this next slide.
Which shows a young woman with headphones
on with an iPad in front of her.
And right next to that is an actual physical tactile graphic
that we've created to support blind students learning
how to code using Swift Playgrounds.
So I wanted to just start diving now into,
you know, what are we doing now?
You know, what does this program and what
are some these resources that we're giving people?
So you know, we truly believe that everyone can code,
including students who are blind, deaf,
someone who might have a limited physical mobility or even
sensory challenges, or autism.
And when we first introduced Swift Playgrounds,
even from day one in 2016, it had full featured support
of voiceover, Zoom, larger fonts, bold type,
and switch control.
So this shows our commitment to day one having
this just work for people.
So that's the good news.
Even if you don't have these tactile graphics,
you know, yourself or any student, any teacher
can pick this up and start learning to code.
A couple of years in, we just kind of felt like, you know,
we think there's an opportunity here to build on what's
here in this digital world.
So I think we did something pretty innovative, actually.
Is we actually worked with schools
for the blind and schools for the deaf, first of all,
got feedback to see what we think, you know,
could help them.
As well as working with Jordyn and some
of our own talented engineers and just thinking through
what would be most helpful.
So we actually worked with San Francisco
Lighthouse for the Blind and actually the RNIB
as well in the UK.
And we have actually designed tactile graphics for each level
within Swift Playgrounds.
So that's 47 within the first level.
And then another 40 plus in the second level.
So that essentially you can just have that supportive
scaffold for students who's blind.
So you can see here there's color graphics
to help with low vision.
And then there's also the tactile support with Braille
and some options there.
So these are actually, just to be aware,
these are downloadable for free from our website,
the designs themselves.
So if you or someone in your community
has access to a Braille printer, you can actually download them.
You can print all of them.
Or maybe you just get to a certain level with a student,
let's say a certain point where you go, OK.
We really do need this additional support.
And as Jordyn said, the cool thing
is it's a 3D puzzle world.
And each world has, well you'll see.
We're going to show you.
But each world actually has some orientation mobility and graph
like mapping, mental mapping like what Jordyn was saying.
So it's kind of interesting.
A student can apply some of those same concepts
of turning right, turning left, moving forward.
And voiceover helps guide us along
as well as these graphics.
So I just wanted to give you all some context.
Here's some of the tactiles, you know
what they look like a little bit more clearly.
This one's portal practice.
And you see the columns on the left and right
and the beautiful color from San Francisco Lighthouse
for the Blind.
Just able to print these is just so cool.
And then you can see the side-by-side iPad
with the graphic together.
I think that, you know, what's really impactful.
There's also Swell.
So any of you that are familiar with Braille, there's
the option to do the Swell print as well.
And again we provide those design files for free.
JORDYN CASTOR: Can I give a little more description?
JANETTE BARRIOS: Sure.
JORDYN CASTOR: Yeah.
So what the graph is kind of showing is
they sort of look like graph paper a little bit
with like the squares.
And there's different textures and different shapes
for each different artifact of the game.
And so the character is a different shape.
There's gems that you have to collect.
And so those are represented by stars.
I'm trying to think of the other ones right now.
But there's many different shapes and a key
as well to give you kind of the lay of the land as to what
represents what sort of tactile shape.
RICHARD LADNER: This is Richard, Jordyn.
And to that, so the way this would work
is the tactile graphic matches the image on the screen.
And the screen is accessible by a voiceover
so you can hear things that are synchronized
with your finger on the left--
excuse me, on the tactile graphic on the left.
What you're looking at it in sort of a multi-modal approach.
Is that what you're doing?
JORDYN CASTOR: Yeah.
That's correct.
Yep.
RICHARD LADNER: Great.
JORDYN CASTOR: So you can use the tactile graph
along with what you're hearing to solve the problem.
You could also use a tactile graph
and move characters around, move balls of Playdoh around.
You know, it's a very hands-on learning approach
so you don't have to interact with just the screen.
JANETTE BARRIOS: And we see a lot of creativity out there.
We know of teachers that have printed the 3D like versions
of a character and put it on the tactile graphic itself
as an exercise to see, how would you solve this puzzle even
before going into the digital realm
or alongside the digital realm, knowing that different students
need different things.
You know, a student might solve one of these levels
fairly quickly with using voiceover and the audio cues,
where somebody else might just need to take a little more time
to orient to the world.
That said, I think it's really important to just look at it
as not get too caught up in the graph,
you know, the tactile supports on its own.
Because it's really meant to go with the world of Swift
Playgrounds.
So that said, yeah, I think that's a great question
and a very good description from both of you.
Thank you.
So continuing on, we'd love to show you a video demo of what
is this like with voiceover on.
And I may pause it and interject Jordyn where we want to.
But let's show what this looks like.
NARRATOR: Voiceover on.
Landscape.
Instructions.
Go find the bugs and fix them.
Column one.
Row three.
Open switch at height one.
Find height one facing west.
Double tap to switch characters.
Actions available.
By playing leave animation.
Entering character picker.
Loop.
Character selection.
JANETTE BARRIOS: So something I just wanted to call out
here is some people might not even choose
to record this for you all.
But we felt it was important because, you know,
a sighted student gets to pick their character.
And maybe you like one or another, or you like the sound,
or the way it looks.
So we wanted, you know, we try to look at that whole surprise
and delight of the experience as well, not
just the solving of the puzzle.
JORDYN CASTOR: And as we're listening--
NARRATOR" Character selection.
JANETTE BARRIOS: Go ahead, Jordyn.
JORDYN CASTOR: As we're listening,
you may notice that there is different sound effects
for each portion of what the character is doing.
So as we continue through the video, when you're
listening to it, you can hear we're going
to pick the Hopper character.
But you can hear, you know, Hopper walking forward.
It kind of makes like a doo-doo-doo noise.
And you'll hear it collect the gem.
And so there is different sound effects for each thing.
And you know, there's water.
You can hear the water.
You can hear the different landscape.
And each character also has different music
that goes along with it.
So there's a lot of attention here paid
to the soundscape of the game.
And the sound effects, as well.
JANETTE BARRIOS: Yeah.
I think that's huge.
And I also want to share, we have
an option with the audio where they're, like, when you first
open the app, it will be playing music in the background.
You can actually turn off the music,
but keep on sort of the audio of the characters, which is really
helpful for sensory challenges.
Students on the spectrum, as well as Jordyn
was saying, if I'm a blind student I might like the music
or not.
But I might definitely want to hear what
the actual character is doing.
So let's keep playing this through
so you guys can experience it.
NARRATOR: Hopper.
Character selection.
Hopper.
Hopper playing [INAUDIBLE].
The world is four columns by five rows.
Hopper at height one facing west on column one.
Row three.
The goals are open switch at height one on column one.
Row three.
To repeat this description, tap outside of the World Grid.
Step through my code.
Button.
Step through my code.
Executing move forward.
Stop button.
Hopper walked to column three, row two.
Executing move forward.
Hopper walked to column three, row three.
Executing turn left.
Hopper turned left.
Now facing west.
Executing move forward.
Hopper walked to column two, row three.
Executing collect gem.
Harper collected gem at column two.
Row three.
Executing move forward.
Hopper walked to column one, row three.
Executing toggle switch.
Hopper toggled switch open.
Congratulations.
Heading.
You found a bug in the code.
All the right commands were written down,
but they were in the wrong order.
You debugged this code by rearranging the commands
until you solved the puzzle.
JANETTE BARRIOS: All right.
There you go.
And I thought what I would do is actually
take a quick pause here, Brianna,
in case there were any questions or things.
We have certainly a good amount to still get through,
but I just wanted to--
once you guys got to see the actual interaction there
and also, Jordyn, if you had any additional comments as well
after seeing that.
BRIANNA BLASER: And, folks, you can ask questions in the chat
or raise your hand, too.
JORDYN CASTOR: Yeah.
And all of this content is showing up also
if you have a Braille display connected.
All of this content is showing up on the Braille display
as well.
And in the video, the videographer
executed the step through my code.
So normally voiceover wouldn't say executing, move forward.
Executing, turn left.
You would normally hear, you know,
Hopper walks two column two, row three.
Hopper turned left.
So there's different options where you
can move for your code slowly.
You can move through your code quickly.
You can move through it at a regular pace.
So that was just one option.
For those of you who might be thinking,
that's so verbose, that that was just one option.
RICHARD LADNER: I think Venkatesh has a question.
AUDIENCE 1: Can everybody hear me?
RICHARD LADNER: Yeah, we can hear you.
AUDIENCE 1: So I was just curious.
In this example that you just showed, was there
a tactile graphic accompanying it as well?
Or was this a purely on-screen experience?
JORDYN CASTOR: There's a graphic to go with this one.
AUDIENCE 1: I see.
And is it superimposed on the screen?
JORDYN CASTOR: It's a paper.
It's a paper.
It's physical.
You get a book of tactile graphics printed.
JANETTE BARRIOS: Right now, you could
do this puzzle completely independent of the graphic,
right, the tactile?
That's just a supportive option, especially
given we didn't even have those for the first couple of years
that this app was available.
It's just more of an additional option.
And everyone does have a tactile to go with it.
AUDIENCE 1: Got it.
Got it.
Yeah, I was just trying to follow up
on the earlier question of whether it's on the screen
or beside the screen.
JANETTE BARRIOS: Right, I know.
Some people, I think the future--
we've gotten that question before.
And I think it is a little bit confusing,
because I think, in a sense, where the future is headed
is maybe people would love some type of tactile experience
overlay situation.
But as of right now, we're just taking the best practices
from the blind community of how to design really
well-done braille tactiles that are used today
alongside this digital experience.
It's an interesting concept though, right?
It's the digital and then the tactiles next to it.
Also, by the way, I completely forgot to tell you all.
This is a free app that you download
called Swift Playgrounds.
And it's an iPad app.
It's completely free.
It is made by Apple.
But I just realized that we just jumped right in and showed you
but didn't tell you how to get it.
Any other questions?
BRIANNA BLASER: Stefik asked a question.
I'm sorry, Venkatesh.
Stefik asked a question in the chat.
He said, I know Apple doesn't have currently
vibration controllers in the iPad, just the phone.
Has there been any talk of including vibration
in a new version?
And have you considered how it could
be used for accessibility?
JANETTE BARRIOS: I think that's a great question.
I would say we don't talk about future plans.
But I think haptic is certainly an area that's exciting.
And I think you see probably more activity
on things like Apple Watch, something
that you're wearing that's able to give you haptic feedback.
If you're blind and you have an Apple Watch,
you can actually get different taps to turn left or right.
And we've given developers more access
to actually be able to develop with that in mind.
So those of you that are coders and want to develop more
in this space, there's certainly more and more happening there.
So I think that's pretty much all I can say on that.
And if you guys are good, I think
I might press on here, unless there's
any other questions here.
BRIANNA BLASER: Sounds great.
JANETTE BARRIOS: Awesome-- great questions.
So I just wanted to let you know in addition to the actual app,
there's actually entire curricula
available for teachers and students that
map with these different coding concepts.
So we start within the app with basics, like commands,
and then there's actually curriculum
that helps a teacher.
And we even have student guides now
that are even accessible with voiceover.
So if you're a student and you're blind,
you can actually download the student guide
alongside your app.
And you might be one who just wants
to go without instructions.
That's fine.
But if you want some guidance and actually education,
especially given what's happening now,
I think, with remote learning, there's
just so many great resources here
I wanted to make sure you all are aware of.
And there's even a guide for coding clubs.
We've found not just in the accessibility community
but in the larger school system schools might not
be ready to dedicate an entire elective class, say, to Swift
coding.
But the clubs are really popular.
And we've seen this where you can have a smaller group.
And again, I think this works really well
in a virtual setting as well.
Maybe you might just have two students that
are super interested in this, and you
can start a little club amongst yourselves
and just have people explore.
So we actually have curriculum and ideas for some activities
to do.
And bottom line is, we want this to be
fun and engaging, flexible and accessible, yay, but also
serious code.
So we're teaching actual coding through this experience.
It's not just a game.
You actually learn to code.
And then shifting a little bit, there
was another accessible resource I
wanted to make sure you all were aware of, which
is we created some videos.
This came out of request from and conversations
with the Schools for the Deaf.
And what they expressed was what they really
needed is descriptions of the coding concept in American Sign
Language or in sign language, because it really
is another language.
So we worked with a deaf director
and two deaf programmers.
They are not Apple employees, per se,
but we found these deaf programmers
who just did such an amazing job and helped
us describe these concepts.
So if you're up for it, I can play one of these for you now.
BRIANNA BLASER: Great.
[VIDEO PLAYBACK]
- So what is a command?
A command is a specific action you want to be performed.
I'm going to give a good example, baking a cookie.
Think about the steps that are required.
You're going to need to measure the flour and sugar
and then mix the ingredients to make the dough.
Next, you'll take the cookie dough
and scoop it onto the baking sheet
and then put that baking sheet in the oven.
30 minutes later, you take it out of the oven.
Each step is a command.
Don't forget.
Eat the cookie.
[END PLAYBACK]
JANETTE BARRIOS: So it's so simple and descriptive.
And then they actually go through again,
aligning with the curriculum.
So they describe loop.
You can imagine the loop as the great next step after commands.
Functions, loops, conditional code, and those
are also embedded into the books.
So I just wanted to make sure you all
are aware those are there.
They're actually on our developer website,
and there will be a QR code later for that.
But diving on ahead, so where this also can be really
fun is with connected devices.
So once you learn how to code within Swift Playgrounds,
we have these third-party developers
who have added their own playground.
So you'll find playgrounds in the app for drones,
for robots, just all kinds of fun things.
And this is where we find it gets really exciting.
So this is an image of a young woman.
We've done a couple of workshops in person
at Schools for the Blind and Schools for the Deaf.
And we had a lot of fun introducing drones.
So after coding for just an hour or two within playgrounds,
they're able to move right into this space.
And one fun thing that we do is we don't necessarily tell them
that we're going to do this.
We just like to hand out the drones
and then give them a tactile experience
and ask them, hey, what's in front of you?
What is this?
And let them guess what it is.
And I know, Jordyn, you have some really fun anecdotes
from these experiences.
So I thought you might like to share as well.
JORDYN CASTOR: Yeah, totally.
So there's a picture on the next slide of a young boy who
is about eight or nine.
And he was so much fun to work with, oh, my goodness.
But he's super excited because the drone
is lifting off into the air.
And I'm crouched down next to him listening to the drone.
And so this is really about that surprise and delight.
A lot of blind students I know they think,
oh, they can't fly a drone.
When we were at the Youth Slam back in 2017,
there was a student who said, a couple months ago,
my dad told me that I couldn't fly a drone,
that it was impossible for a blind person.
And you guys showed me that this was possible.
So now I'm going to get a drone.
Which is so cool, because just showing
people what's possible--
and we handled the drones.
So you're like, have you ever seen one of these before?
Checking out the propellers and the wheels
and the landing glide things and all that,
and some people get it, and they're like, it's a drone!
And so you can take that drone, code everything
you want to in playgrounds.
You can make it fly.
You can make it do flips.
You can make it do turns.
You can have it carry things.
There's just so much you can do with it.
And as a blind person, you're in control
of where that drone goes.
There's a podcast by David Woodbridge,
I think, of him flying a drone around his garden
because he was able to tell the drone exactly how far to move
and exactly when to go up, when to go down.
So you are in full control of what this drone is doing.
And it is a very hands-on experience.
I feel like a lot of coding materials for beginners
are inherently pretty visual.
And it goes like drag this here and put this visual image here
and use this GUI here.
So but this is completely hands on.
And like I said before, you can make the drone do
what you want it to use.
So that's really what makes the drones
and these different connected accessories powerful.
JANETTE BARRIOS: And this is another kind of example,
just the smiles, right?
That's what you want as an educator.
You want that spark.
You want that smile and that engagement.
And that's what we really see here,
whether it's completing a puzzle or trying
one of these connected devices.
And here's a picture of Jordyn sitting at a table with four
of the MeBots.
By the way, these are all things that are sold, really,
at any retail nowadays.
Apple Stores, sure, but you can order these on Amazon,
or Best Buy will have some of these things.
So they're definitely available.
And you can connect it via Bluetooth to the iPad.
You want to use the playground that's
created by the third party, not necessarily the app.
Sometimes you can play a little bit of both, to be honest.
But sometimes those app aren't necessarily the
most accessible, like the ones that you first
turn on for the drone.
It's when they get into the playgrounds is really
where a student gets to actually code and say, do this, do
that, do this, do that.
And we work with these third parties
to bring in playgrounds that are accessible.
And so Jordyn, you're sitting here with the robots.
And I know you like to do these.
[INAUDIBLE] a little about [INAUDIBLE]..
JORDYN CASTOR: My heart is in the drones.
I love the drones.
Secretly, you can make the drones fly off your hands,
if you have your hands extremely flat.
So I have definitely found some really fun things
to show students to do with them.
But let me describe what the MeBot looks like.
It's a robot-looking thing with a square head with movable arms
and legs and just a bunch of bendable joints.
And you can make the robot dance to different music
and with different tempos.
And you can make it do crazy dances and splits
and cha-cha slides and just whatever kind of dance move
that you want it to do.
So it's really fun, because you can feel it
as you're going along.
And not only do the students get to learn things,
but I really get to learn things too.
It's just one of the most rewarding things
in the whole world helping these students learn how fun it
is to code and giving back to the community that's
given so much to me.
It took that one week of a STEM academy
to instill that confidence and that believe in myself
that I could go out there.
And I could do computer science as a career.
And seeing the excitement and just these light bulbs
going off with these students, it's just extremely humbling.
There was a student at the Youth Slam in 2017 who said,
you know, from this workshop, I want to take what I've learned
and go back and be my main school's game
designer for the School for the Blind in Kentucky.
And so I think what's really important to know
is that these workshops and these lessons on coding really,
really does change people's lives.
And you might not think that it might affect them
on a deep level, but I can tell you--
and my cat can tell you--
that it's extremely effective.
And it's life-changing.
JANETTE BARRIOS: That's awesome, Jordyn.
So yeah, I think it's just that variety too.
Like you were saying, these happen to be musical.
So a student might be more musically inclined.
Or sometimes we've done dance-offs,
with the dance-offs having one code, one dance, or another.
But it's just more options.
And what I'm going to do now is share a short video.
But I want to explain what is first.
So one of the experiences that we did
was California School for the Blind.
And they have a gym.
And so we set up the gym with the teachers
there for the drones.
We started in the library, which is what you saw earlier, just
for small lift-offs and just to get them warmed up
with the coding.
We took a break and moved over to the gym
where there could be more space.
And we discussed ahead of time.
Do we go with this idea of trying
to get them to code the drone to land
on a specific spot in the gym?
Or do we just let them do free form?
And we decided to do both.
And I think it's really important, that conversation
of, well, would a student who's maybe non-visual
be able to hit that target?
And I love that Jordyn and others
that we were working with pushed us to be like, heck, yeah,
they should hit that target.
Give them the opportunity to do that.
So what was exciting was to see--
and to be frank, some of these groups of girls or young women
just were on fire.
And there was this one group that was so determined.
They were trying different ways to code.
They were not quite hitting the target.
And the video I'm going to show you
is actually the moment when they do, which was just so cool.
So I'm going to go ahead and share that now.
[VIDEO PLAYBACK]
[CHATTER]
- Go, go, go.
[CHEERING]
[APPLAUSE]
[END PLAYBACK]
JANETTE BARRIOS: And so that was probably
a good 45 minutes it took them to figure out how to do that.
But they worked together.
They were collaborating.
They would take turns.
They would try different ways to code the drone.
And that moment was just so special.
So I thought it was a cool thing to be
able to share with you all.
JORDYN CASTOR: And the other thing
that's really awesome about playgrounds
is that collaboration aspect.
Here, we had blind students.
Some were totally blind, and some were visually impaired.
And I think that one of the coolest things
is that blind and sighted students alike
can experience the same exact thing with Swift Playgrounds
because of the accessibly-designed app
and the accessibly-designed experience.
Sorry, there was a cat walking across the screen.
I was like, don't pay attention.
It's like, [INAUDIBLE].
But so yeah, it's blind and sighted students.
Multiple groups of students can experience this together
and help each other and collaborate together.
And I think that that is just so cool, because there's so
many times in school--
I know that for me personally, I was sidelined during group
projects because people didn't think I could do any do it,
even though I told them the ways I could do it.
And so with this, everyone gets to play a part.
JANETTE BARRIOS: That's awesome.
And actually, again, a perfect segue way to a project
that you also were part of around our Worldwide Developers
Conference.
So once a year--
some of you may follow this or not,
but June is a big time for us because we have our Worldwide
Developers Conference.
This year, we did it all virtual.
It's incredible the amount of sessions that we produced.
Normally, it's in person.
It's thousands of developers from the world come together.
But we did this all online.
And what that means is you can go watch these sessions
and learn.
And any student can go watch and learn.
And these are closed captioned.
And there's one actually in particular
that Jordyn was involved with that we wanted
to share a little bit about.
So Jordyn, do you want to tell them a little about this?
JORDYN CASTOR: Yeah, yeah.
So real quickly, before I talk about the challenge,
not only are these closed captioned,
but there's also transcripts.
And if you're watching within the developer app,
you can also copy the current block of code
that the presenter is talking about to the clipboard.
And there's also transcripts of the code that follows
along right with the video.
So as a blind person, when people
are doing PowerPoints or keynotes and you're like,
man, there's code on the slide--
they're not reading it aloud--
never fear, because we solved that problem.
So definitely have access to all of the code that's
on the various keynote slides.
So in going back to the theme that I was involved in,
it's called Swan's Quest.
And on the slide, there is a picture of a turtle.
The art in this is amazing, I'm told.
But there's a picture of a turtle,
and he's walking down these steps
and giving something to the lizard.
And the goal here is to guide your turtle, the heroic turtle,
through the quest to the swan's palace.
And you complete different challenges along the way.
And so the very first challenge--
because your accessibility needs to be
built in from the ground up.
So the very first challenge is called Voices in the Dark.
And it's about guiding your turtle
through this dark, spooky cave.
And developers have to learn how to use VoiceOver and learn how
to label different elements.
So in this case, we're labeling pictures and graphics
to advance through the cave and light
the torches to guide the turtle to the swan's palace.
And from there, you progress to musical challenges.
And so you're playing music for the swan.
And we can hear the music as blind people,
but the music is also written out on a scroll.
So someone who is deaf or hard of hearing
can also make this challenge work for them.
And yeah, it was really fun teaching people
how to use VoiceOver.
I talk about VoiceOver gestures and the importance
of really labeling your elements in your app.
I feel like that's one of the biggest accessibility hurdles
that blind people face in the smartphone
and the technology area is you download this app,
and you hear nothing.
Or you hear button, button, button.
And so that was really, really fun
getting to teach the audience about VoiceOver and Swan's
Quest.
JANETTE BARRIOS: So Jordyn did an amazing job.
This slide says, what is VoiceOver?
And this is just a screenshot from the video itself.
Here is a QR code.
This will take you to the full video.
So Jordyn does a portion of it.
But the video itself explains the goal of the Swan's Quest.
Keeping in mind, this is just awesome
that we decided, every developer, this is not
[INAUDIBLE].
It's for every developer in the world.
And we're saying, we challenge you in this first quest
to learn how to use VoiceOver and solve.
And so everybody learns, right?
Every developer learns this experience.
And Jordyn just did an amazing job teaching and sharing.
[INAUDIBLE] link to the video.
JORDYN CASTOR: And there's a series of four videos.
There's four videos.
So this is the intro, Voices in the Dark.
And then the chapters continue.
And it gets progressively more difficult as you go through.
So I would really encourage you to solve it,
because it's really fun and rewarding.
JANETTE BARRIOS: So I kind of hope--
one thing I wanted to call out here
is whether you're from the beginning stages of coding
around playgrounds all the way through to something like this
and even with Xcode, we actually have some improvements
coming with macOS Big Sur to make
that more accessible as an experience as well.
So we're really just wanting everyone
to have access to coding at every level
and be able to learn and then create the stuff
and also create job opportunities.
So a couple more resources, this is our Apple Teaching Code
website.
And this QR code takes you to the portions of that website
where you can download those free tactile puzzle worlds
or link out to the videos in American Sign Language.
So I'll leave that for a minute.
So if you have your iPhone camera,
you can just hold it up, and it'll grab that link
and take you right to the website.
Any third-party apps, you can just
do that right with your iPhone camera or iPad.
And then here's, finally, just some additional resources.
So our accessibility website is a great place
to explore more of these built-in features.
If you walk away going, man, Jordyn mentioned VoiceOver.
What is that?
I need to learn more or any of our other features.
Also, to make you aware, we actually
have accessibility to apple.com.
You can email us specific feedback.
Or if you have someone in your life
that has been telling you like, oh, I
need to tell Apple they should build this feature.
Or I can't make this thing work the way I want to,
or maybe you found a bug.
This is a great place to just reach out to us.
We really do take this in.
And we have people that are trained in accessibility
actually that answer that as well as I've
put up a phone number here.
And if you do this QR code, it's actually a contact card.
So it'll save Apple's accessibility support options
for you right there into your contacts.
For Twitter, we have @AppleSupport.
And you may not realize we even have a Twitter handle now.
That one's not just for accessibility,
but we put so many great tips and tricks
out about how to take advantage of our products.
These products are so complex now.
We know that.
But we want you to learn some of these features.
And then @AppleEDU is for educators.
And we're always sharing best practices of amazing educators
out there, including coding, but also out of Create as well.
And just anything that's new from our education teams
tends to land there.
So I wanted to make sure to share those.
And with that, we are already to--
I think we have a few minutes left for questions.
So yeah, I hope you enjoyed it.
BRIANNA BLASER: Thank you so much.
This was really great to hear.
So yeah, I invite folks to ask their questions in the chat.
Or if you want to raise your hand or ask them verbally,
you are welcome to do that as well.
RICHARD LADNER: Yeah, this is Richard.
I have a quick question.
For accessibility, there is also Switch Access.
Or I call it Switch Control and Voice Control, which
is those are Apple things.
But are these playgrounds accessible for those as well?
JANETTE BARRIOS: Yes.
JORDYN CASTOR: It does work with Switch Control.
I know that.
RICHARD LADNER: I guess Voice Control is fairly new.
JANETTE BARRIOS: Yeah, Voice Control is fairly new.
I would say Switch Control for sure consistently, because it's
actually using very similar APIs as VoiceOver
in terms of how it clicks through and selects the items.
I think Voice Control is doable.
But I would say Switch Control would probably
be more reliable, at least for now,
just given the way these apps are designed.
JORDYN CASTOR: And send us bugs.
If you do it with Voice Control and it doesn't work,
let us know.
LAUREN MILNE: So I'll chime in really quick.
I'm Lauren Milne from Macalester College.
And I was actually playing around.
By the way, this is awesome.
I love the Swift Playgrounds.
But I did try it out with Voice Control.
And I think the big thing that I had a challenge
with was selecting, like if you want to drag and drop
the snippets of code.
It didn't select the code itself.
It seemed to select the whole line.
So it was selecting empty space.
So you weren't able to do the drag
commands with Voice Control.
JANETTE BARRIOS: Did you use the grid function at all?
LAUREN MILNE: No, I didn't try the grid.
That would have been smart.
I used the item function.
JANETTE BARRIOS: Yeah, no, that's
where I think the grid is really interesting in that it's really
designed for things like maps, right?
And this is probably a good example.
I just tell people.
Whenever there's things with Voice Control
that's not working necessarily with a number--
again, the way the code works in the API,
it's trying to select similar to VoiceOver
or Switch Control first.
But that grid overlay may allow you
to pick something and do more of a drag and drop or move.
So there's probably some things that
could be done, I would think, with that.
It just might take a bit more practice.
And there just may be things that
might be a little bit harder to do
or just won't work as seamlessly, say, as if you were
using a Switch or VoiceOver.
But I'm glad you tried it.
That's so cool.
And just for everybody.
JORDYN CASTOR: And you should totally email us bugs.
You should totally email us.
JANETTE BARRIOS: Yeah, Voice Control,
just for everyone on the call, was introduced last year.
It's pretty transformative in terms of you're
using your voice to actually have your device take action.
So it's different than Siri where
Siri's like an assistant that does things for you.
You're actually saying tap, swipe,
and it's actually taking specific action.
So it's particularly useful for someone
with a pretty severe physical disability,
kind of like Swift Control but just using your voice.
So that's just to give a little more context.
That's awesome, Lauren.
And like Jordyn said, you can send us bugs as well.
And when there's betas for public,
you can actually file them to Apple.
And we need more people that know accessibility
and understand accessibility and are
filing for these different scenarios,
because there are so many different people
with different setups, right?
JORDYN CASTOR: Yeah, if you download the public beta,
we're done with the public beta.
JANETTE BARRIOS: Part of the process.
JORDYN CASTOR: There's a feedback assistant
that you can file the bugs actually to us.
AUDIENCE 2: Hello.
JORDYN CASTOR: Hello.
AUDIENCE 2: Hi, yes, my name is Aboubakar.
And I don't really have a question.
But I just wanted to share some information.
So I'm a PhD student, and I'm conducting research
in accessibility of programming languages.
So recently, we conducted a study
to find out the accessible block-based programming
languages that are available and being
used by teachers to teach the students
with visual impairment.
So out of our work, we got a poster
that was accepted in ASSETS for 2020.
So I just wanted to share what we found.
We found that Swift Playgrounds was
the only language that was accessible
and being used by the teachers.
And before that, I didn't know about Swift Playgrounds.
So that's why when I got about this presentation today,
I decided to attend so that I can learn more.
So I just wanted to share that information,
that Swift Playgrounds is actually
common among teachers of students
with visual impairment.
And they find it very useful.
And though there are some challenges,
but it is very useful to these students.
And I would like to know how I can share the paper once it
is ready, because it's a poster that has been accepted
in the ASSETS for 2020.
So how I can share that to the?
RICHARD LADNER: Do you have the two-pager?
You're talking about the two page of the poster.
Is that what you're talking about?
AUDIENCE 2: Yes, yes, it's a four-page paper poster, yes.
RICHARD LADNER: Oh, OK.
You can send it to me, and I can forward it on.
AUDIENCE 2: OK.
OK, sure.
Thanks.
Yeah, so it's really a great job that you are
doing with Swift Playgrounds.
Yeah, I just wanted to share that.
JANETTE BARRIOS: That's amazing.
I really appreciate that.
It's great to have that validation.
Our engineers, as Jordyn can attest,
are often heads down and working so hard on this stuff
to have you come with a fresh perspective
and tell us that really means a lot.
And also, we're also our biggest critics.
So we know there's always things we can do better,
and we tend to focus so much on just
reaching the next bar that it does help to know and hear
that.
I will say anecdotally one of the students
from the Texas School for the Blind,
that school also has done quite a bit with this.
I remember about a year ago one of the teachers
that we had is off to Austin Community College now.
And he's taking a regular Swift Playgrounds coding program
at Austin Community College.
And I was just like, yes, because you guys know
just those individual lives, like Jordyn shared,
it's that next person that can just go on to the next level.
That's the reward.
So thank you for sharing that.
That's awesome.
BRIANNA BLASER: We have time for maybe one more question
if anybody else has a question they'd like to ask.
RICHARD LADNER: This is not a question, but I was wondering.
When you went over to the California School
for the Blind, did you meet a guy named Scott Jaffray, who
is the tech guy there?
Did you meet him?
JANETTE BARRIOS: Yeah, in fact, I should really
give them a shout out.
The team there, we collaborated with them
on all of this from the very beginning.
And that whole session was done in collaboration
with the teachers.
So really, give them a shout out and credit
for helping us set all that up and just giving us
feedback all the way along.
And some of their ideas even made it
into some of our curricula.
RICHARD LADNER: Yeah, Scott was one
of our students in a summer professional development
workshop two years ago.
JANETTE BARRIOS: That's awesome.
So by the way, he builds wooden versions of the playground.
So while we have this tactile graphics, again, what we see
is a lot of innovation with the teachers.
He has a woodshop.
And he makes smaller tactile stairs and things
of some of these levels that the students use.
So it's just so great to see that.
JORDYN CASTOR: Yeah, there's been so much innovation
surrounding it.
RICHARD LADNER: Yeah.
Well, thank you both for the presentation.
I've learned quite a bit myself.
So that was fantastic.
JORDYN CASTOR: Thank you for having us.
JANETTE BARRIOS: Our pleasure.
Yeah, anytime, and we just hope you all
share word of this stuff and appreciate your attention
and time.
I know there's a lot pulling our attention these days.
But we just want more people to know about it and more students
to be on board and more teachers and people
in the field to feel more confident to try it themselves
and get others using this stuff, because it's all doable.
People often talk about the future.
Well, hopefully, one day, a blind student
could code an app.
Well, they can right now.
JORDYN CASTOR: The future is right now.
JANETTE BARRIOS: Let's do it.
Yeah, right, Jordyn, let's do it.
Game on.
JORDYN CASTOR: Absolutely.
BRIANNA BLASER: Awesome.
That is such a great closing thought.
Thank you all for joining us.
And thanks, everybody, for your attention today.
JANETTE BARRIOS: Thanks for inviting us.
Thank you, Richard.
Thank you, Brianna.
JORDYN CASTOR: Thank you.
JANETTE BARRIOS: Thanks, everyone.
Thank you, Jordyn.