Slide to unlock

Today’s touch-screen world is leaving behind an entire segment of the population: those with motor impairments. iSchool Ph.D. candidate Martez Mott wants to change that.

Scroll Down

We live in a touch-screen world. One where tiny computers full of information live in our pockets. Where boarding passes are printed at touch-screen kiosks and groceries are paid for at touch-screen card readers. A world where extending a finger and making contact with a particular point gets us what we want.

Well, except for the millions of us across the world who experience motor impairments — the millions of us who are being left behind.

The many
definitions
of “touch”

“I’m pretty sure if you went around and asked, ‘Hey, what do you think a touch is?’, you might get different answers, but it would probably converge around something that looks like this,” says Martez Mott, raising his pointer finger before tapping it on the table in front of him. “For the participants in my studies, that was not the case at all.”

Mott is a Ph.D. candidate in the UW’s Information School and an active member of the philanthropy-funded MAD (Mobile + Accessible Design) Lab and the DUB (Design. Use. Build.) Group, where his research centers on making touch screens accessible for people who live with motor impairments, from cerebral palsy to muscular dystrophy to Parkinson’s disease.

“Touch screens were developed for a certain type of use,” says Mott, who received financial support through the University of Washington Graduate Opportunity Award. It’s simple: If you want to interact with something on your device, you have to be able to extend a finger and touch it cleanly and accurately. “That’s it,” he says. “Everything else is relegated to some other type of functionality — if you touch your phone with two fingers, it thinks you’re trying to zoom.”

For those who live with cerebral palsy, however, touch might look like patting the screen with the back of the hand, or dragging a palm or a fist. “When that happens, the screen picks up a big blobby area,” says Mott, “and the system has no idea what to do with it.”

Yet.

For those who live with cerebral palsy, touch might look like patting the screen with the back of the hand, or dragging a palm or a fist.

How we touch

What happens if you can’t touch your device with a single finger?

ipad screen showing user touching with finger ipad

When a single finger makes contact with the screen, the device registers the center of that point as the user’s intended touch point.

ipad screen showing user touching with whole hand ipad

When the back or side of the hand makes multiple points of contact with the screen, the device can’t register which point is the user’s intended touch point.

Martez Mott

Say hello to smart
touch

Calibration software for touch-screen accessibility

Imagine a system that adapts to touch — whatever that might look like. Enter: Smart Touch. The project is a work in progress, constantly being tested and tuned by Mott and a small team of researchers, including his advisor, Associate Professor Jacob Wobbrock.

The objective? To get touch-screen access to a level where anybody can pick up a device and, with a simple calibration procedure, be able to use it right out of the box without needing additional specialized technologies.

Think of it as the counterpart to automated speech recognition, where the device asks you to repeat a few phrases so it can pick up on your unique speech patterns. In Mott’s model, the device asks you to touch a couple targets — a balloon or a bull’s-eye, for example — so it can pick up on your unique touch patterns and adjust how it responds.

And it’s all guided by Mott’s participants, who give him real-life, real-time feedback.

Meet Martez Mott

  • Hometown: Detroit
  • Bachelor of Science in Computer Science, Bowling Green State University, ‘10
  • Master of Science in Computer Science, Bowling Green State University, ‘12
  • Ph.D. candidate in Information Science, University of Washington
  • University of Washington Graduate Opportunity Award recipient
  • Post-grad plans: Teaching, preferably at the iSchool. “I’m working toward my Ph.D. because I love the research, but I also want to empower others the way my mentors have empowered me.”
The device asks you to touch a couple targets — a balloon or a bull’s-eye, for example — so it can pick up on your unique touch patterns and adjust how it responds.

You can make inclusive technology possible

Mott is just one of many UW students and researchers working to make digital innovations accessible for everyone. By giving to the iSchool’s Human Computer Interaction Research Fund, you can fuel life-changing tech solutions for people of all abilities.

Making smart touch a reality

With real-life users

Two years ago, at the start of the project, Mott lugged a huge, table-sized Microsoft Surface into Provail: a local center that helps people with disabilities pursue the lives they want to live. Among the nonprofit’s many offerings? Computer classes — and lots of motivated learners willing to participate in his study.

There, Mott worked with eight volunteers to make Smart Touch a reality, beginning by asking them to hit targets on the screen. “I had a wide range of participants, so I saw a wide range of strategies used to complete the tasks,” he says. The first person would prop one hand on the tablet for support while dragging the other hand across the screen. The next person would interact with the device in an entirely different way. This, says Mott, is why inclusive innovation — actually looping the software’s target population into the design process — is so important.

“When people are motivated to use a technology, they’ll employ a lot of different strategies in order for that to become possible. It was difficult to get an understanding of what ‘touch’ looks like,” he says. “People used the software in ways I couldn’t have predicted, which resulted in a lot of failure — and fixing — for me.”

Martez Mott works on Smart Touch with Provail participant Ken Frye

Martez Mott works on Smart Touch with Provail participant Ken Frye

A DJ with a dream

Ken Frye, a Provail participant in Mott’s study who has cerebral palsy, has been in the radio industry for over 40 years, with an impressive career that includes gigs at Seattle’s very own KEXP and Centralia’s Live 95.1. Today, he hosts his own radio show "Ken’s Ten," which features music from the ’20s to today “because all music can and should live together” — but that doesn’t come without its challenges. Watch to learn how Smart Touch could transform the way Frye navigates the world.

Touch screens, everywhere

Failure is part of the learning process, and now — “after a lot of learning,” says Mott, laughing — he has a system up and running. And as he creates newer versions of Smart Touch over the next few years, retooling algorithms to improve accuracy and creating better designs for performance gains, the iterations will only get better.

“The hope is that you could buy a device, turn on an accessibility setting, and then go through a calibration procedure for better touch performance so you can use your phone or tablet however you like,” says Mott, who’s slated to wrap up his dissertation by 2019.

“This research isn’t just a theoretical contribution; it’s a practical one,” he says. “Hearing my participants tell me how important and impactful this work can be in their daily lives is the most rewarding thing. It’s what drives me.”

Ken Frye using a touch device

Ken Frye interacts with a touch-screen device

And as far as rolling it into the market? That’s a few years away, but Mott has a vision — starting with buy-in from the tech industry. “I would love to see every touch-enabled device — public or private — be accessible, with this technology integrated into every operating system, quietly running in the background. I want to see a check-out kiosk at the grocery store embedded with intelligence that can infer people’s behaviors based on their unique touch,” he says. “We can do it. We just haven’t done it yet.”

Ken Frye chats with KEXP DJ Cheryl Waters

Ken Frye chats with KEXP DJ Cheryl Waters

Spotify

Ken Frye’s 10 favorite songs

Listen

(Requires Spotify player)

My hope for technology is that somebody like me can be clearly understood. Having a device I could use would mean freedom.

Ken Frye

What you care about
can change the world

The University of Washington is undertaking its most ambitious campaign ever: Be Boundless – For Washington, For the World. You can be part of Mott’s project and help iSchool students like him make technology accessible for everyone at the MAD (Mobile + Accessible Design) Lab by making a gift to the Human Computer Interaction Research Fund.

© 2017 University of Washington | Seattle, WA