RICHARD LADNER: Richard Ladner, I'm the PI for AccessCSforAll.
And It's really my pleasure to introduce Earl Huff, who
is a fourth year PhD student at Clemson University working
under Dr. Julian Brinkley.
In fact I see Julian.
Hi there, Julian.
And he's done some remarkable work in accessibility
and especially looking at K through 12 CS education
and its accessibility, especially for blind
and visually impaired.
So Earl, go for it.
EARL HUFF: All right.
Thank you very much.
Hello, everyone.
Thank you for coming out for this talk.
Let me share my screen and get that ready to go.
All right.
This one that I want to share.
And all right.
Perfect.
All right.
So everyone should be able to see, hopefully
be able to see the screen.
All right.
So for today I'm going to talk about my ongoing research
examining the accessibility of computing education
from multiple perspectives.
Let me see if I can get this to work.
All right.
So first I'm going to give some background and motivation
behind the exploration, and then go over to three exploration
studies that I conducted-- going over their methods, findings,
and the implications.
Then I'll summarize all the findings into common themes,
and talk about what's the next step in my research.
OK.
So starting back addressing inequality in CS,
a book came out called Stuck in the Shallow End:
Education, Race, and Computing in 2008
talking about the disparities in CS
learning opportunities for students
from underrepresented populations.
And with that came exploring computer science or ECS.
So it provides a high school introduction level
CS curriculum as well as teacher professional development
to introduce computer science.
And then eight years after College Board introduced
the AP Computer Science Principles
Course, which provides a college preparatory
introduction into the field of computer science launched
in 2016.
Data suggests that there is increased participation
integrating AP Computer Science in more
schools across various states in the country, increased
participation of students from underrepresented population.
However, until now, there hasn't been much data
about the participation of students with disabilities
in computer science until a 2020 CS report where
11 states reported on participation of K-12 students
that have been served under the Individuals with Disabilities
Education Act, taking at least one computer science course.
From the 11 states just under 13% of all K-12 students
were served under IDEA.
However, just under 8% took CS.
And so my research investigated more so
of the participation of people with blind or visual impairment
represented by the image to the right of a black girl
sitting on a stool holding a white cane.
People who identify as blind or visually impaired
represent just over 2% of the US population as of 2018.
There are notable disparities in employment rate
as well as the education rates of people
with visual impairments, and particularly earning degrees
in computer science.
So just under 3% of all bachelor's degree recipients
in computer and information science are visually impaired.
And under 3% of doctorates in CS or mathematics
are earned by people with visual impairments.
And so there's been many efforts in broadening participation
to increase access to CS for people
from underrepresented populations,
including those with visual impairments and people
with disabilities in general.
So as I have on the slide, Code.org on to left,
going up top AccessComputing, as well as AccessCSforAll,
and on towards the right end is CMD-IT,
which focuses on underrepresented minorities
and people with disabilities in information technology.
Now to focus more on some of the challenges
faced by people with visual impairments
gaining access to CS or participating in CS.
There are two primary barriers.
One is on the technology side.
So programming is primarily a visual task--
things such as navigating code, debugging code, building
graphical user interfaces--
all of which tend to be challenging
according to past research.
Even so, using tools such as integrated development
environments or IDEs, these tools
that provide code editor compiling capabilities,
but they have a rather complex user interface
which can be challenging to navigate for people who
use assistive technology such as screen readers
or translating it through refreshable Braille displays.
In the workforce this often may lead
to pushing peer programmers to seek help from co-workers
for assistance with reading code bases as previous research has
suggested.
Now on the learning aspect of these challenges--
the image of a black woman sitting down
reading a book on Python--
much of the challenges reside in the lack of accessible
learning materials, so materials that may not
be in many different formats, accommodations that are not
provided in programming course, often
a lack of context in in-class instruction,
such as pointing out code on a PowerPoint slide.
And these result in students feeling
left behind in their progression compared
to their sighted peers.
And so past work has examined different ways to address some
of these problems, particularly with accessible technology
solutions that come in either development languages
and environments-- so special languages or environments
to help with building graphical user interfaces--
or using audio interfaces as part of the development
environment, or even extending existing developing
environments such as StructJumper
and AudioHighlight.
Those are plug-ins for the popular IDE Eclipse that
help with navigating code bases by creating a hierarchical tree
structure that is successful with the screen reader.
And Then there is Quorum programming language
an evidence-based language originally designed
for people with visual impairments, but now
designed to be used by people with one or more disabilities
and the accompanying software development environment Quorum
Studio.
So my interest is exploring more areas of accessibility
within curricula and tools in K-12 computing education,
notably what are the students saying
about current accessibility, what are the teachers saying
about current accessibility, strategies
that teachers are employing to instruct K-12
students with visual impairments in coding,
as well as some of the topics that K-12
students with visual impairments excel in,
and what are possibly some advantages
to their visual impairments in computing
versus their sighted counterparts.
And so as a result, I conducted an exploration
using three studies to learn more
about the current state of disability
in computing education at the K-12 level.
And so I did this through three studies
from different perspectives-- professional programmers
with visual impairments, teachers of K-12 students
with visual impairments, and then
within the context of an actual computer science course
over the course of a semester.
And so I'll start with the first exploratory study.
And this research was guided by a series of questions
asking about the challenges of visually impaired programmers
and what they faced in their workplace,
ways to improve accessibility within their line of work,
to what degree do they feel their visual impairment can
afford them certain advantages over sighted programmers, what
were their past learning experience like
or training to get to where they are now,
and what do they see as ways to make learning
to code more accessible.
And so to do that I conducted a series
of semi-structured interviews with professional programmers
with visual impairments.
11 programmers signed up for the study,
most of which were software developers--
software engineers--
with average years of experience being 10 1/2 years,
each about 34 years of age.
Interviews lasted between 25 to 40 minutes
and conducted by phone or video call.
So I'll be summarizing much of these findings,
but this study was submitted for review,
accepted, and published in the proceedings
of the International Conference of Software Maintenance
and Evolution.
So please let me know at the end of this presentation,
if you like a copy of the documents.
So the challenges that were brought up by the participants
include workplace dynamics-- so working with coworkers who do
not have much, if any, experience with visually
impaired programmers-- so using things such as Trello board
or doing pair programming can be very challenging for a lot
of the participants-- accessibility of developer
tools used at their line of work--
many prefer to use more simpler editors or even
a command line interface for, let's say,
the Python developers--
and then lastly, programming tasks
such as code navigation, debugging, user interface
design.
Some of these findings which are reinforced by prior work
conducted.
Suggestions for improvements in the workplace
would be to use more accessible tools
or improve current accessibility, a number
of quality of life improvements such as video tutorials
for using these tools from the perspective of a screen reader
user--
while previous research has shown
that there are many videos produced
by visually impaired persons, whether some of them
involve tutorials on using software development tools is
currently unknown--
and lastly, developing tools with input
of visually impaired programmers.
Many of the participants believe that these current tools
were designed from the perspective
that the developer is someone with full sight.
And as a result, they believe that much of these tools
are not designed to work well with screen readers.
So a more user-focused approach to design
of such developer tools in the future would be necessary.
Now, in talking about their past education and training
experience, the three main themes that came up were
inaccessible learning materials, which often force participants
to look outside of classroom instruction--
so online video tutorials, websites,
looking for accessible PDF versions of textbooks
as an example--
the tools that were used in the classroom-- so the editors
that the professor would use may not work well with students,
and often have to find some sort of alternative--
and then in class instruction with a lack of context.
So if code was being introduced on a PowerPoint slide
but the professor does not indicate,
let's say, a line number of the code,
then it makes it very difficult for that student
to identify where in the code base
to look for this line of code perhaps.
So some suggestions to improve CS education
from these professional programmers
would be the accessibility learning
materials, particularly various formats of textbooks
with alt text for the images--
so a lot of the textbooks didn't have alt text descriptions
for images, so the screen reader would have a challenging time
to describe them--
as well as simpler editors or tools to use on education
or even s command line interface --
so having alternatives to perhaps the traditional tools
that are used in classrooms for students with visual
impairments.
And then we asked about perceived advantages
of visual impairments.
The most notable things that came up
was memorization of code structure and shortcuts,
without sight there's the lack of distractions
from visual stimuli, which gives them more focus, greater
attention to detail, and also being
able to retain more information about the code structure.
In some cases they believe that there are some tasks
that they're able to do better than their sighted
of co-workers, such as spotting errors in the code base.
So to summarize this study, going back to the research
questions, the challenges involved in the workplace
include workplace dynamics, developer source accessibility,
and challenging programming tasks.
Ways to improve this would be to use simpler IDEs
or provide ones with simpler user
interface, a number of quality life improvements
like video tutorials to use tools,
and then for future developer tools,
including input from programmers with visual impairments.
For research question three, the advantages
include better memorization, understanding
of code structure, greater focus, attention to detail.
Question four involves education,
so some of the challenges in learning
was learning materials and tools being inaccessible,
lack of context during instruction.
And then ways to address that would it
be providing different accessible formats,
other materials, and considering more accessible alternatives
to the tools used in classrooms.
And so for this study, this primarily
shows kind of like a base.
Because many of these programmers who
talked about issues learning, many of these tools
may have been improved from the time
that they had been using them for their educational purposes.
And so for exploratory study two,
we look at more recent attempts at using these developer tools
to teach the CS.
And so this study involved interviewing teachers of K-12
visually impaired students.
So the questions that guide this research talk
about the challenges teachers face
when instructing blind or visually impaired
students to write code.
Challenges they see in students in learning to write code,
and how the teachers adapt their curriculum and teaching
style to serve the needs of their students.
And so much like exploratory study one,
I use semi-structured interviews with the teachers.
Many of them have on average about 10 years
experience teaching students with visual impairments.
They taught between 3rd and 12th grade.
The interviews lasted between 20 and 60 minutes
and were conducted by phone or video call.
So again, I will be summarizing much of my findings.
But this was also reviewed and accepted
for publication at the 2021 SIGCSE Technical Symposium.
So if you like a copy of that document
you can let me towards the end of this presentation.
So some of the themes that came up
included curriculum-- the topics and tools.
So the majority of the curricula that was used
came from CodeHS, the Computer Science AP Principles,
and its variant desk can be used with Quorum Studio.
Many of the topics taught include programming,
but in additionally, how to use a computer,
assistive technology and navigation--
because many of the students come
with various levels of proficiency
in using a computer or some technologies.
The languages that were used to teach CS
included Quorum, Swift app development, Java, Python,
HTML.
And the different tools that were used
included Quorum Studio, Swift Playgrounds when using Swift,
and more simpler editors such as Notepad, Word Pad,
and web-based editors available, such as the ones on CodeHS.
Some of the assistive technology-- screen readers,
Braille displays, tactile maps, and magnifications were used.
In terms of accommodations provided
and concerns from the teacher's perspective,
most of the accommodations came in the way of modifying
the existing curricula.
For example, one teacher does role play with the curriculum
and assignments to determine how accessible they
are for their students.
So they would go through some of the lectures
and the assignments and determine
if, from the student's perspective,
they would be able to understand it.
Another teacher creates different formats
of their materials.
So they have instructional guides
with keyboard commands and code snippets for the students.
But they convert most of those materials
to Braille for the blind students.
And then one teacher that used to do robotics programming
was switched to audio programming
because blind students, while they could do the robot
programming, they wouldn't be able to see the robot's actions
when it performed.
So they fell out of love with it very quickly.
Some of the concerns that were brought out from the teachers
included much of what was echoed from the first study,
a lack of accessible learning materials,
students not having precursor skills--
so proficiency in using a computer or assistive
technologies, hence why they included it in their curriculum
for their courses--
and then the current editors, and the complex
interfaces that they possess.
And so some of the teachers professed
that many of these students are losing interest in CS.
And thus, they are dropping out of CS
courses because of some of these challenges and concerns
that are being brought up.
In terms of topics they found straightforward or easy to do
or maybe more challenging to understand,
some of these topics include conditionals, repetition,
logic statements, and topics involving
discrete points of information, such as tables as an example.
And more difficult topics include anything
dealing with graphics--
visual elements, especially if they lack any description--
code navigation, code placement--
so figuring out where the code belongs in the code base,
in terms of line numbers--
debugging, and then languages with particular coding
conventions,
So if you're familiar with the camel case notation, where
you would have a lowercase letter for the first name
of the function or variable and then uppercase other words,
so in some cases, the students were having trouble
figuring out capitalization of the words at different points.
And then asking about what they perceived as advantages,
visual impairments for their students, some of which
were echoed in the first study included greater attention
to detail, better concentration and memorization
of code structure, and then, interestingly, one teacher
brought up that they noticed their blind students were
better at logical structuring than their low vision
and sighted students.
So there is even differences in abilities
for visual impairments depending on the degree of vision
loss for the students.
And then we're asking about areas for improvement
for both the students and teachers
in computing education.
On the students end the most echoed
concern was editors with a more simplistic user interface--
so less menus to go through, simplification of key command
and keyboard shortcuts.
And then for the teachers themselves, they
would like to have access to more accessible learning
materials or ways to create them themselves a little easier.
For the institutions, they would like
better investment or resources.
Many of the teachers talked about using outdated devices,
computers, or not having the necessary software licenses
for certain computer software which limited their options
to teach students programming-- and so they wanted greater
investment from the institutions on those fronts--
and then additional training for teaching certain topics
to blind, visually impaired students--
so this goes back to the concerns about the more
challenging topics brought up that students were struggling
with--
and then some pre-learning material
as well for the students to read over prior to the lecture.
And so to summarize study two, so the first question teachers
find that certain curricula are lacking accessible
learning materials.
The lack of institutional support
also limits what they can do with their students
in providing accommodations, then lack of options
for accessible tools--
so editors and other hardware--
and then needed training for teaching programming
to students with visual impairments.
I should note that for many of the teachers, some of them
do not come from a computer science background.
Some of them were learning how to teach programming
to their students.
And so they would like additional resources
for training and learning how to teach computer science
and programming to their students.
For question two, accessible tools and difficult topics
as well as lack of learning materials
was what they perceived as the biggest areas of concern
for their students.
And then question three, teachers are often modifying
existing curricula or they may have used the AP CS Principles
course with Quorum, which has proven to be more
accessible for their students.
And so now going to the final exploratory study
I looked at the context of actual teaching programming
to students within a course.
And so, this study was a remote observation
of what it was like to [AUDIO OUT] instruct blind
and vision impaired students.
Programming indicates what level.
And so the two questions guiding this research
talked about ways blind or visually impaired students
learn how to program and what ways do teachers instruct
their students in programming.
So the original plan for the study
was to do an in-person observation
at a school or schools to learn more about the process that
goes on in teaching.
However, because the COVID-19 pandemic,
that was not an option.
So a remote method was utilized to carry out this research.
And so I utilized a diary study.
So a diary study is a form of field study
that allows the researcher to collect data
remotely by making the participants the data
collectors.
So the participants are provided diaries or questions
to answer in which they report back
or submit them to the researchers at different times,
depending on the kind of diary study you are running--
so there is event and time based.
This study was event based in which students
had to submit a diary after each lecture or submission
of a homework assignment.
And for teachers, they submit diaries after each lecture.
This was a four week diary study taking place
during the run of a programming course
where one teacher and two students were involved.
The questions involved in each diary
were different but the same in certain instances.
So for student diaries, questions
asked about the topics they learned
from the lecture, the topics they found
were comprehensible or challenging,
the programming languages and tools that were used,
their perception of difficulty of the assignments,
and their overall thoughts about the lecture and/or assignments.
For the teacher diaries, topics that were taught,
assignments were used, how did they present their lecture
materials, the language and tools used, their perceptions
of their student's ability to complete assignments,
and overall their thoughts on a student's understanding
of the topics and the lecture.
So this course was focused more on web development.
They use the curriculum from Code.org--
using language mark-up such as HTML and CSS.
And they were using the online code editor
from Code.org WebLab, which I'll talk about
later on in the presentation.
So now, I'll give a brief recount of the week
by week diaries for teachers and students.
This work has yet to be submitted yet.
So for week one, it was about introduction
to HTML and debugging.
Students were tasked with changing
a heading element, which is an H1 surrounded
by angle brackets.
So they want to change an elements heading size as well
as locating an error in the provided code.
So from a teacher diary, they expressed
students were getting used to editor and the website
from Code.org.
They were concerned about the fatigue
students would have encountered from the debugging
because they were having issues finding
the error within the code.
On the student side, one student has a color deficiency
or is colorblind to red, which unfortunately
most errors in code bases do turn out to be red.
So it became a little more challenging
for them to locate it.
Another student noted the small and thin text,
which also increased the challenge
in locating the error.
But nonetheless they were able to complete the assignment
as instructed.
So for week two they dealt with ordered and unordered lists.
And so this was one of the easier topics
that they were able to pick up.
Teacher noticed that they didn't have
much issue with the assignment.
But they noticed that students were
having trouble with writing code from scratch
for the first time.
They were learning how to do code placement.
And it was echoed in the student diaries as well.
So overall they were still able to complete the assignments
and move forward with the next week's project, which
was building their own HTML web page.
So diaries were not submitted from the students,
but the teachers submitted one basically recalling
how they perceived the student's handle the assignment,
to do it.
So I'll get that towards the end of this study.
So week four proved to be the most challenging topic
for the students, which was cascading style sheets, or CSS.
So the debugging aspect of the assignment
was pretty easy for the students.
However, when it came to creating the style
sheet from scratch and linking it to the HTML page,
the students had trouble, mainly because of the styling rules
syntax, primarily the semicolons that you
add at the end of each styling rule.
And so it caused a lot of confusion for the students
as they had noted in their diaries.
Some students also mentioned they
would like to have more separation of the code working
area from the rest of the user interface, which
I'll get into towards the end of this presentation.
And so now I'll touch on the quantitative aspect
of the diaries.
So the students rated different aspects of the course
from 1 to 5--
1 being challenging, 5 being easy.
Now week three was the project so they didn't
submit an entry for this.
So for week 1, 2, and 4 they submitted their ratings.
These are averages that you see on the screen--
so lecture format, assignment format,
language, tool and editor, overall class,
and overall assignment.
So you can see week two proved to be the easiest topic.
But when we get to week four you start see the numbers drop
down, meaning it was more challenging because it
was the CSS topic.
So they struggled more with using CSS.
Now on the teacher side they also had a quantitative rating
system.
So they focused more on the ratings being from 1 to 5--
so 1 being easy, 5 being more challenging.
So for weeks one and two things, for the most part,
were fairly easy for the students.
But when you got to week four, with CSS,
you notice the numbers went up to indicate that it's
more challenging to learn CSS.
And so to summarize this study, students
were able to achieve their learning outcomes despite some
of the hiccups that they came across
such as the small and thin text, having to write code
from scratch for the first time, and then some debugging issues
that arose.
The teacher felt that students were
able to complete or demonstrate their comprehension of topics,
and additionally, learning that online instruction proved
to be potentially another avenue for accessible
education for students with visual impairments.
And so now I'm going to summarize the findings
from all three studies.
This was broken down into three main themes.
So the first I'll touch on is curriculum and materials.
So overall, in class instruction can
suffer or be problematic for the students of visual impairments
due to lack of learning materials, the lack of context
provided for in class instruction,
many CS curricula, as it stands, are not
designed to accommodate the needs of the students
and perhaps many professors are not
equipped to also accommodate students with their needs
as well.
Certain CS topics have proved to be
a challenge, such as continuous information or even
visual elements.
Teachers are often left to modify or create
more materials to accommodate their student needs.
But some aspects may be overcome given the advantages discovered
from their visual impairment.
Second one is developer tools, as noted
by the studies and past research, a number of editors
have issues with the interfacing which screen readers are often
the biggest issue for visually impaired programmers.
They often prefer simpler editors or even
a command line interface to not deal with any menus.
Our current tools are developed under the impression
that developers are sighted users.
And so a more user-centered approach
needs to be taken to incorporate programmers
with visual impairments to build more accessible and more usable
tools.
And then lastly, the perceived advantages of visual
impairments-- greater attention to detail,
having greater focus, the lack of visual stimuli
also helps and leads to better memorisation,
understand of code structure, as well as
performing certain tasks better than sighted programmers.
So with that said, the question becomes, well what's next?
What's next in line?
So one area we can look at, and this is part of my dissertation
research, is online learning.
So common among some of the studies
was online learning, especially given
how the pandemic has forced more distance learning utilizing
certain sites like Code.org or CodeHS
for their own online built editors.
Online learning can provide a number of advantages such
as availability learning materials,
and given the web-based nature of some of these materials they
may be more screen reader accessible,
it also eliminates the need for physical locations--
but that may also be a disadvantage for students that
do need more in class instruction--
also allowing for a synchronous learning, independent learning
as well.
But there are disadvantages such as presuming the accessibility
of certain web sites or tools--
for certain learning management platforms or even some
of the sites like Code.org, questioning their accessibility
for screen readers--
also having to use the internet and a web browser
requires a certain level of proficiency when
using a computer and assistive technology--
as mentioned in study two, a lot of them
to teach computer proficiency as well as proficiency
of assistive technologies--
and then of course, requirement access to the internet.
Just because we're in the 21st century
doesn't mean that everyone has access to the internet,
as been noted from the year 2020 when everything was switching
to remote learning, there were issues with students
not being able to access online materials because
of lack of internet access back at home.
So I'm just running a couple of examples
of some online learning tools that are known.
So the graphic shown is of Code.org's WebLab,
which can be used for teaching online and also using
their online editor, as you see.
So towards the bottom portion of the image
there is a code editor with some HTML code
that you can then run and see the results on the screen next
to it when you run the code.
So a lot of the courses on web development
or in other languages can utilize this editor.
The same thing for CodeHS.
They also have their own editor.
So as seen here, they're teaching JavaScript
using a function in the code editor, println Hello, world,
which is used to print out the phrase "Hello,
world", which is very common when you're introducing
programming to new students.
So on the right side, there is an section
where students can click on the run code button.
And then it will spit out the results of whatever
was put in the editor.
And so this influences the idea of how can we
improve the accessibility of online learning in CS
and even online learning in general.
So that is where my dissertation research comes in.
So I'm proposing on examining the features necessary to make
online learning accessible for blind and visually impaired
learners.
Using a user-centered design process
I intend to design and implement features for online learning
that are coming from the stakeholders.
So user-centered design is a philosophy
where you take an input from your stakeholders, your product
end users, and incorporate their input and feedback
into the design and implementation of a product
or system.
And so from my research, I intend
to do a series of design activities
with teachers of blind and visually impaired learners
as well as blind and visually impaired learners themselves,
to learn more about the necessary features
to incorporate into an online learning
platform that would make the experience more
accessible and useful.
And then using these features within this high fidelity
prototype of an online learning platform,
I will evaluate it and study for its accessibility
and its usability.
And so what I expect to come from this research
is to better inform educators, instructional designers,
accessibility researchers, on designing
these accessible and usable experiences to online
learning for learners with visual impairments,
demonstrating the usefulness of user-centered design
to improve accessibility in online education.
So as mentioned from study one where
programmers we're talking about developer tools
need a more user focused approach to developing tools,
user-centered design is such an avenue.
And then to uncover features that are
desirable for both learners and teachers in online learning
platforms.
And so with that I end my presentation
and am ready to take questions.
Thank you for listening.
RICHARD LADNER: Thank you, Earl.
That was fantastic.
And so if you want to ask a question, you can unmute
or you can put a question in the chat.
So I have a question for you to start with, Earl.
So when you did your studies with the teachers,
were the teachers in a mainstream setting or were they
in schools for the blind and visually impaired,
or who were the teachers?
EARL HUFF: Great question.
So it was a combination.
So a few of them were in mainstream schools,
but most of them were in schools for the blind or visually
impaired.
So there was a couple of them that were at a public school.
And one was a worker for the district.
RICHARD LADNER: Were they professional TVIs,
teachers of visually impaired or were
they regular mainstream teachers, general education
teachers?
EARL HUFF: So for the mainstream teachers
they were more mainstream.
So they were learning more so to teach programming
to the students as well.
But the ones at the blind and visually impaired institutions,
they were more professional TVIs.
RICHARD LADNER: Great.
Brianna, can you check the chat?
BRIANNA BLASER: Yeah, Meenakshi asked a question
about block based programming since they're so commonly used
in K-12 computer science education
and are so highly visual.
Did you get any feedback about that?
EARL HUFF: There was some feedback about block
based when they were using--
well, they were using mostly apps.
So Swift was kind of what they were using.
One teacher was using Swift.
For the most part they found it useful for their students,
although there were some issues that they've
come across in working with it.
But that was the only instance that I've
come across where they were looking at block based
programming in that instance.
Most of them were using more web focused,
more like HTML, Python, Java, and Quorum primarily.
But I do believe I did look in past research
about the usefulness of block based
programming and the work that's being done in that realm.
So I hope that answers your question.
ANDREAS STEFIK: I have a question.
But before I say it, let me just say, Earl, this is awesome.
Like seriously, like Brianna and I were talking the entire time
on a chat about--
every time you had a result, we're like, wow,
I didn't realize that.
We've worked in this space for a while,
so the fact that you showed me some stuff that's new
too is just awesome.
I really appreciate that you and your advisor
have kind of picked this up.
It's very needed.
And I just want to say thank you for doing such a great job.
So that thing said first, I do have a question,
and that is oftentimes as tool designers
it's really easy for us to be kind of technical focused
or sort of like get our knee into the grindstone
about some hard problem.
But if you had to look at across your studies
and sort of say what should the priorities be
for us technical people that are sort of designing
these tools-- like if you had to pick which thing should we try
to improve the most, which thing would have the biggest
bang for your buck-- like for example
you mentioned debugging or online stuff--
but which one do you think are causing the most problems
that we should try to solve?
If you had to guess.
EARL HUFF: Thanks for the question.
Thanks for a comment and thanks for the question.
So I would definitely say from the studies more
of the aesthetics, the user interface--
so navigating across the different menus getting
to the code editor.
That was often the biggest complaint
was students were having trouble using their screen
readers to get through some of the interface.
So looking more as a simplistic form or a minimalistic approach
perhaps to some of these editors will greatly
improve how close they get to the editor.
So the example used from CodeHS and WebLab
was kind of like in the right ballpark
in terms of a more minimalistic design for editors
in that case.
And so, a lot of that definitely helps
when we take the time to kind of learn from the end users
about what they really need to have a better
experience with these tools.
ANDREAS STEFIK: That's great.
Thanks.
RICHARD LADNER: Any other questions for Earl?
And I want to echo what Stefik just said.
You know I learned a lot from your talked too.
And getting that close to so many teachers and variety
of teachers and learning what their issues are
is really, really important.
So I appreciate the techniques used also--
the diary study, the interviews, the semi-structured interviews.
You're really using some good methodology
to find the answers to your research questions.
And the research questions were important questions.
So I appreciate that.
I should mention that a month ago, two months ago--
I can't remember-- we had our workshop.
I guess it was a couple of months ago at Microsoft.
Of course it was a virtual one.
And Earl was on a small group with us.
And we developed a paper about this and also
other disabilities as well.
And so we'll follow up with email
to everyone with the link to that.
And thank you, Earl, for being on that team and producing.
Our article was the longest of all the articles.
[LAUGHTER]
EARL HUFF: It was.
I did take a look at other papers,
and yes, ours was very, very lengthy.
But we did touch on a lot of different points
in that paper too.
So big shout out to everyone on the team who
contributed to that.
RICHARD LADNER: Yeah, Brianna put a chat a link to that.
But we'll put it also in the email to everybody,
because we have your emails.
And we will invite you to be on our mailing
list for the AccessCSforAll so that you
can pick up on things that we're doing in the future as well.
BRIANNA BLASER: And somebody asked
if it was possible to get the recording of this talk.
And so we make sure that we get good captions of the recordings
and then post them on our website.
So I will email everybody when that is available.
It normally takes a little while to get them up online.
Somebody also asked, Earl, whether your results
have been published.
And I know you mentioned that one
will be published at SIGCSE.
You have a paper at SIGCSE.
Are there any other places folks should look?
EARL HUFF: So yes, the one at SIGCSE
should be coming out fairly soon.
And also the one for study 1, the professional programmers
interview, that has already been published in the International
Conference for Software Maintenance and Evolution,
that's an IEEE conference.
So I can definitely provide Dr. Blaser with links to those.
If you want to get those out to them I can provide them.
BRIANNA BLASER: Yeah, that'll be awesome.
If you email them to me I can include them
when I email folks.
ANDREAS STEFIK: I'd like to read those too because I actually
haven't read them yet.
I knew that you were working on this,
but I haven't talked to you since November.
And I couldn't remember exactly where you were at
in the publishing process.
So I'm looking forward to reading those as well.
I was going to say too, your dissertation work sounds pretty
much, I don't know, fantastic.
And so as you are getting results in
and stuff like that, please, please let us
know what you're doing.
Because we do a lot of stuff with online tools.
And I imagine we know nothing.
And so we'd be happy to play follow the leader
and learn from whatever you're doing.
So please feel free to keep in touch on that.
EARL HUFF: Well, funny enough, I'll be defending my proposal
in about two weeks.
ANDREAS STEFIK: OK, so more than likely Julian
is going to fail you, and then we'll
never hear from you again.
That right Julian?
Is that how it works?
I think it is.
JULIAN BRINKLEY: No, I'm super supportive of Earl.
He's a very diligent worker and a very smart guy.
So I'm sure he'll be fine.
ANDREAS STEFIK: This does not surprise me at all.
So he seems like he's a pretty motivated bloke, so anyway.
BRIANNA BLASER: Any remaining questions?
I have to say too, I'm pretty impressed that you pivoted
in the pandemic to figure out how
to still get the research done.
Because I know I've talked to a lot of grad students,
in particular, who have struggled
with how to keep on top of their research
and not extend their stay in graduate school, so to speak.
EARL HUFF: Thank you.
Yes, I have to also give credit to my advisor.
He helped me out with that one aspect too-- kind
of working out how to work a more remote method to deal
with the observation study.
So it was definitely a pivot.
BRIANNA BLASER: Awesome, well thank
you everybody for joining us.
Thank you, Earl, for taking the time to put this together
and share with our audience here.
And we'll be in touch over email.
RICHARD LADNER: I was going to mention one more thing.
That AccessCSforAll is putting together
a series called Accessible Computer Science Education
Teacher to Teacher.
So we've recruited four teachers to give presentations,
90 minute webinars.
And there'll be a question and answer as well
during those webinars.
So these will be--
and the first one is going to be on March 24th
by a teacher of blind and visually impaired
who has a lot of experience.
And we're trying to get as many teachers as possible who
are in the mainstream to join this and learn
from the expertise of this teacher.
It'll be in the late afternoon on the West Coast
and early evening on the East Coast
to accommodate teacher's schedules.
So you'll all hear more about that in the future.
Thank you very much, Earl.
That was fantastic.
I don't know what that noise was behind you but that was a--
[LAUGHTER]
EARL HUFF: Oh, sorry about that.
So unexpectedly they're trimming trees just outside
of my apartment.
I like, uh, I didn't think you heard it.
BRIANNA BLASER: No worries, you can't always
control those things.
RICHARD LADNER: Yeah.
EARL HUFF: Nothing you can do, so.
RICHARD LADNER: Thanks very much, everybody, for coming.
BRIANNA BLASER: Thank you.
ANDREAS STEFIK: Thank you.