Designing Accessible Cyberlearning: Current State and Pathway Forward
Editors: Sheryl Burgstahler, Terrill Thompson
Contributors: Shiri Azenkot, Sheryl Burgstahler, Jill Castek, Dan Comden, Kevin Crane, Lyla Crawford, Bria Davis, Prasun Dewan, Shari Gardner, Tracy Hammond, Mike Jones, Fengfeng Ke, Aaron Kline, Richard Ladner, Katrina Martin, Lorna Quandt, Hadi Rangin, Meaghan Roper, Raymond Rose, Sofia Tancredi, Ana Thompson, Terrill Thompson, Emily Moore, Louise Wilkinson, Bingran Wang, and Elizabeth Woolner
Suggested reference: Burgstahler, S., & Thompson, T. (Eds). (2019). Designing accessible cyberlearning: Current state and pathway forward. Seattle: University of Washington. Retrieved from https://www.washington.edu/doit/designing-accessible-cyberlearning
Burgstahler, S., & Thompson, T. (Eds). (2019). Guidelines for cyberlearning researchers. Seattle: University of Washington. Retrieved from https://www.washington.edu/doit/guidelines-cyberlearning-researchers
Burgstahler, S., & Thompson, T. (Eds). (2019). Accessible cyberlearning: A community report of the current state and recommendations for the future. Retrieved from https://circlcenter.org/events/synthesis-design-workshops
Given rapid increases in the quantity and variety of formal and informal learning practices and research that make use of digital technologies, it is important to ensure that all potential students can benefit from technological innovations at the intersection of learning theory and emerging technologies—referred to as “cyberlearning.” The goal is simple: All cyberlearning opportunities should be accessible to, usable by, and inclusive of everyone, including instructors and students with disabilities. This paper emerged from a review of relevant literature and ongoing consultation from attendees at an onsite capacity building institute and members of an online community (AccessCyberlearning 2.0, 2019). It serves to inform the design of current and future learning technologies and pedagogy to be inclusive of everyone. To make progress toward this goal, it is recommended that instructors who use digital tools in their courses employ inclusive teaching practices, that faculty members teach about accessibility in computing courses, that researchers address accessibility issues in all stages of their work, and that funding agencies require their funded cyberlearning projects to do the same.
In the following paragraphs, the authors of this paper discuss issues that support the need for cyberlearning opportunities that are accessible to students with disabilities. Topics include the demand for science, technology, engineering, and mathematics (STEM) innovators, the ubiquity of technologies for learning, legal mandates, and the role of cyberlearning in the marginalization of individuals with disabilities. The paper is a product of the AccessCyberlearning 2.0 project described in the About the AccessCyberlearning Projects section at the end of this paper.
Meeting the demand for STEM innovators
The US must meet the demand for STEM professionals to remain competitive and grow the economy. Attracting more individuals from underrepresented groups (e.g., those who are racial/ethnic minorities, women, or have disabilities) to these fields can help meet this demand.
According to NSF’s 2019 Women, Minorities, and Persons with Disabilities in Science and Engineering report, “Women, persons with disabilities, and underrepresented minority groups—blacks or African Americans, Hispanics or Latinos, and American Indians or Alaska Natives—are underrepresented in science and engineering (S&E). That is, their representation in S&E education and S&E employment is smaller than their representation in the U.S. population.” Although some progress has been made over the past twenty years, these groups remain underrepresented in STEM as a whole, and are particularly underrepresented in some engineering specialties, mathematics, computer science, and the physical sciences. Most of the gains in diversifying S&E have been for white women and Asian men and women (NSF, 2017).
“…Lacking diversity on an engineering team, we limit the set of solutions that will be considered and we may not find the best, the elegant solution.” (Wulf, 2000, p. 595) STEM research and practice can benefit from being informed by the unique experiences and expertise of individuals with disabilities. However, increasing the participation of people with disabilities is not just a matter of quantity, but of quality as well. Engaging more individuals with disabilities is consistent with NSF’s mandate to support “the best ideas from the most capable researchers and educators” (NSF, 2006, p. 3). As stated by NSF Director France Cordova, “Science is too important and rewarding to be exclusive, and too vital to the nation's future to leave anyone out” (2017).
Many people from underrepresented groups, including those with disabilities, lack adequate mentorships, role models, and motivation instruction. Some consider STEM boring, solitary, and void of compelling real-world applications (Jacobs & Simpkins, 2006; Thomas & Allen, 2006; Yardi & Bruckman, 2007). However, success stories of individuals with disabilities in STEM fields (DO-IT, 1993-2018; Stern & Woods, 2001) demonstrate that opportunities do exist for people with disabilities who are interested in STEM, develop academic and self-determination skills, and successfully overcome barriers imposed by inaccessible facilities, curricula, and IT; inadequate academic supports; and lack of encouragement.
Ubiquity of technologies for learning
In the past few decades, digital content and tools have become ubiquitous in all facets of our lives, including within opportunities for learning. For example, there have been dramatic increases in the variety of technologies used in online learning, the number of online courses offered, and the number of students enrolled in these courses (Digital Learning Compass, 2017; Seaman, Allen, & Seaman, 2018; Lederman, 2018). Unfettered access to information technology (IT), digital resources, and collaborative platforms create opportunities to use and produce a vast range of materials that can be shared, revised, and remixed and allow us to connect with individuals and groups both locally and globally in order to contribute to an ever-expanding knowledge base. Designers of learning environments, instructors, and administrators, as evidenced by more than 200 cyberlearning projects funded by NSF (n.d.), continue to find innovative ways to harness the capabilities of current and emerging information technologies to enhance learning opportunities for all students.
One motivation to promote digital learning that is fully inclusive of individuals with disabilities is to comply with civil rights laws. For both K-12 and postsecondary institutions relevant legislation includes Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 1990 and its 2008 Amendments. Together these laws mandate that no otherwise qualified person with a disability shall, solely by reason of his or her disability, be excluded from the participation in, be denied the benefits of, or be subjected to discrimination in public programs. They are interpreted to require access to educational opportunities, including those that make use of IT, to eligible students with disabilities (Burgstahler, 2017a; Office of Civil Rights, 2016). In addition, K-12 schools must comply with the Individuals with Disabilities Education Act (IDEA), which ensures a free appropriate public education to eligible children with disabilities as well as special education and related services (Individuals with Disabilities Act, 2004). Such education and services increasingly involve digital tools and resources.
Each year, the US Department of Education Office for Civil Rights receives hundreds of complaints regarding the inaccessibility of IT used within educational entities (Office of Civil Rights, 2016). Resolving these conflicts as well as lawsuits regarding inaccessible IT are costly for institutions. The resolutions make clear that the use of inaccessible videos, websites, documents, online learning activities, and other IT resources and tools violates the civil rights of individuals with disabilities (EDUCAUSE, 2015; Office of Civil Rights, 2016). Taking steps to ensure that accessibility is addressed within cyberlearning research and practice has the potential to reduce legal risk to educational entities who wish to offer current and emerging digital learning opportunities. In addition, future cyberlearning professionals with accessibility skills are valuable to employers who wish to develop accessible products and opportunities that comply with legal mandates and address the needs of a growing aging population. In 2015, several prominent technology companies joined with universities to form a collaboration called Teach Access, specifically to address the urgent need for students in higher education to attain fundamental knowledge and skills related to accessible technology design (Teach Access, n.d.).
The role of cyberlearning in the marginalization of individuals with disabilities
Some people with disabilities face challenges in pursuing STEM fields because learning opportunities are not accessible to them. Many educators and researchers are simply unaware of accessibility barriers and solutions or how their practices and research can be informed with what is known about diversity, and, more specifically, disability. Disability-related issues cut across linguistic, cultural, gender-defined, socio-economic, and other groups. People with disabilities who are women, who are racial/ethnic minorities, and for whom English is not their first language face multiple challenges to pursuing STEM (AAAS, 2001; Gil-Kashiwabara, Hogans, et al., 2007; Leake, et al., 2006). Intersectionalities such as these remind us that even a subgroup of people with disabilities whose members have a similar disability are not a homogenous group, no one member represents the needs and experiences of the whole group, and every other underrepresented group includes people with disabilities. Individuals with disabilities can only take advantage of cyberlearning opportunities if they have access to IT tools and content and are encouraged to participate.
Questions addressed in this paper
The authors of this paper address the following four questions, which additionally serve to organize the content of this paper.
- What challenges do individuals with disabilities face in using cyberlearning tools?
- What inclusive design frameworks exist to guide cyberlearning research and practice?
- What are recommendations for cyberlearning researchers that can ensure that digital learning opportunities are accessible to, usable by, and inclusive of students and instructors with disabilities?
- What are the roles of other stakeholder groups, including instructors and funding agencies, in ensuring that current and future cyberlearning opportunities are inclusive of individuals with disabilities?
Some of the challenges in using IT faced by students and instructors with disabilities are summarized in the following section.
Cyberlearning Challenges Faced by Individuals with Disabilities
Today people interact with technology using thousands of devices. These devices include assistive technologies (ATs) that allow individuals with disabilities to engage with mainstream IT. People who access IT include:
- individuals who are blind and use audible (e.g., screen readers that read digital content using synthesized speech) and/or tactile output (e.g., a refreshable braille device);
- individuals with learning disabilities such as dyslexia who use text to speech (TTS) technologies that read aloud digital text while visually highlighting each word;
- individuals with low vision who enlarge default fonts or use screen magnification software that allows them to zoom into the screen;
- individuals with fine motor impairments who use assistive technologies such as speech recognition, head pointers, mouth sticks, or eye-gaze tracking systems;
- individuals in a noisy or noise-free environment or who are deaf or hard of hearing and therefore depend on captions or transcripts to access audio content; and
- individuals who use mobile smartphones, tablets, or other devices, which have a variety of screen sizes, as well as gestures or other user interfaces for interacting with their devices and accessing content.
This list describes only a few of the hundreds of assistive technologies (ATs) and custom configurations that allow individuals with disabilities to effectively use IT. However, full engagement by this group requires that mainstream IT be designed in accordance with accessibility standards. For example, two technologies already in widespread use to deliver educational content are digital documents and videos. Although both can be created in ways that are accessible to individuals with disabilities, many documents and videos are not.
Digital documents, whether delivered in HTML, PDF, Microsoft Word, or other formats, must be created in particular ways in order to be accessible, especially to individuals using screen readers. For example, it is critical for readers to know whether the text they are reading is a heading, subheading, paragraph, list item, table, footnote, or other structural element. This information is usually communicated visually to sighted readers, but readers who are blind depend on the document being coded in such a way that their screen readers can inform them of the structure. Without such coding, documents are presented to the reader as an unstructured linear stream of text, which can be extremely difficult to parse.
Video content also presents challenges for certain groups of users. Audio content contained within a video is inaccessible to people who are deaf, and to some who are hard-of-hearing, unless the video is captioned. In addition, content that is shown visually, but not described in the program audio, is inaccessible to individuals who are blind. The standard solution to resolve this problem is audio description, a supplemental audio track that describes essential visual content. Even if a video has captions and audio description, it can be inaccessible to many users if it is delivered using an inaccessible media player (e.g., one that cannot be operated without using a mouse).
As the complexity of technology increases, so does the potential for introducing barriers for individuals with disabilities. Emerging cyberlearning technologies provide promising opportunities for inclusive learning by offering multimodal representation of information (e.g., embodied interactions or 3D/4D modeling), highly interactive and/or immersive learning environments (e.g., virtual reality and augmented reality), and data science-driven adaptive or personalized learning. However, when designing new technologies, careful thought must be given not only to the narrowly-defined intended beneficiaries, but also to individuals for whom the new technologies might erect new barriers.
Principles for the Design of Inclusive Cyberlearning Technology and Practices
NSF’s Cyberlearning and Future Learning Technologies program promotes the use of current and new technologies and instructional practices to enhance learning. Although cyberlearning offers exciting opportunities for most students, many technological innovations and teaching practices create unintentional barriers to some students with disabilities. Most traditional efforts to address these challenges follow what has been called a “medical” or “deficit” view of disability, where the primary focus is on the medical diagnosis and resulting functional limitations of an individual and how this person can be accommodated in order to gain some access to information and otherwise engage in established activities (Loewen & Pollard, 2010; Moriarty, 2007). In contrast, “social,” “social-political,” and similar views of disability (DePoy & Gibson, 2008a, b; Gabel & Peters, 2010) consider variations in sensory, physical, learning, and other abilities a natural part of the human experience like other aspects of diversity that include gender, gender identity, race, and ethnicity. Those with this view suggest that more attention should be devoted to designing products and environments that are accessible to everyone to limit the need for accommodations.
The next section of the paper presents principles, guidelines, and practices that can be used to guide the development and use of inclusive cyberlearning technology and instructional practices. It unveils a design approach that values diversity, equity, and inclusion; that promotes best practices without lowering standards or limiting innovation; and that benefits people with disabilities, along with many other people as well.
Inclusive design frameworks, including universal design
One approach for addressing the needs of a diverse audience that includes individuals with disabilities is to apply proactive design practices that include accessible design, user-centered design, ability-based design (Wobbrock, et al., 2011), design for user empowerment (Ladner, 2015), and universal design (UD). Although all offer contributions to the field, UD was selected as the proactive design framework on which to elaborate in this paper because it has led to the most comprehensive and established principles and practices that can guide the design of inclusive cyberlearning technology and pedagogy. The basic framework is underpinned by the seven principles of UD and the three principles of Universal Design for Learning (UDL) as described in the paragraphs that follow.
UD requires that a broad spectrum of abilities and other characteristics of potential users be considered when developing products and environments, rather than simply designing for the average user and relying on accommodations alone when a product or environment is not reasonably accessible to an individual. UD is defined by the Center for Universal Design (n.d.) as “the design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design.” Principles for the UD of any product or environment include the following:
- Equitable use: The design is useful and marketable to people with diverse abilities.
- Flexibility in use: The design accommodates a wide range of individual preferences and abilities.
- Simple and intuitive use: Use of the design is easy to understand, regardless of the user’s experience, knowledge, language skills, or current concentration level.
- Perceptible information: The design communicates necessary information effectively to the user, regardless of ambient conditions or the user’s sensory abilities.
- Tolerance for error: The design minimizes hazards and the adverse consequences of accidental or unintended actions.
- Low physical effort: The design can be used efficiently, comfortably, and with a minimum of fatigue.
- Size and space for approach and use: Appropriate size and space is provided for approach, reach, manipulation, and use regardless of the user’s body size, posture, or mobility. (Story, Mueller, & Mace, 1998, pp. 34–35)
These principles, originally applied to the design of architecture and commercial products, have also been applied to the design of IT hardware and software, later to instruction, and even later to student services in higher education (Burgstahler, 2015a). A universally designed space or product, including a cyberlearning environment, is accessible to, usable by, and inclusive of everyone, including those with disabilities, as illustrated in Figure 1.
In this model, “accessible” is focused on issues that pertain specifically to individuals with disabilities. According to the Office of Civil Rights and Department of Justice, with respect to technology
“accessible” means a person with a disability is afforded the opportunity to acquire the same information, engage in the same interactions, and enjoy the same services as a person without a disability in an equally effective and equally integrated manner, with substantially equivalent ease of use. The person with a disability must be able to obtain the information as fully, equally and independently as a person without a disability. (2013.)
“Usable” means that participants can effectively engage with a product or in an environment according to the purpose for which it is intended. “Inclusive” dictates that individuals of all abilities can use the same product or engage in the same environment because of its flexible design.
UD challenges society to construct a world in which everyone can fully participate. Consistent with social justice and similar views of disability, UD addresses a range of diversity issues that include disability, but also those related to learning style, native language, gender, race, ethnicity, and other characteristics. A sidewalk curb cut is an application of UD. Originally created to address barriers erected by the typical design of sidewalks to wheelchair users, the ubiquitous availability of curb cuts today benefits many other people, including parents pushing baby strollers and delivery personnel with rolling carts.
Many UD-inspired frameworks have emerged to specifically address instructional applications. Each is based upon a common finding in educational research that learners are highly variable with respect to their abilities and responses to instruction. The most common UD-inspired framework applied in K-12 settings is called Universal Design for Learning (UDL). Developed by the Center for Applied Special Technology (CAST), UDL promotes offering students multiple means of engagement, representation, and action and expression throughout the design of all learning activities, as represented below.
- Engagement: For purposeful, motivated learners, stimulate interest and motivation for learning.
- Representation: For resourceful, knowledgeable learners, present information and content in different ways.
- Action and expression: For strategic, goal-directed learners, differentiate the ways that students can express what they know (Center for Applied Special Technology, 2018).
Applying the combination of UD and UDL principles is particularly suitable for addressing all physical, technological, and pedagogical aspects of cyberlearning in order to ensure that students are offered multiple ways to gain knowledge, demonstrate knowledge, and interact. Examples of UD and UDL practices relevant to cyberlearning applications include making interactions effective for all students, arranging instructional spaces to maximize inclusion and comfort, ensuring that everyone can use equipment and materials, selecting flexible curriculum, providing multiple ways to gain knowledge, delivering instructions clearly and in multiple ways, using large visual and tactile aids, providing materials in accessible formats, setting clear expectations, providing regular feedback and corrective opportunities, and providing multiple ways to demonstrate knowledge (Burgstahler, 2015b).
Although the need is minimized with this approach, reasonable accommodations may in some cases still be necessary to ensure full access and engagement to a particular student when the universally designed offering does not. For example, a student with a learning disability engaging in a universally designed online course may require extra time on an examination as determined by a special education teacher or a postsecondary disability services office.
A UD approach can be viewed as an attitude, a goal, and a process that values diversity, equity, and inclusion; that promotes best practices without lowering standards or limiting innovation; that is proactive; that benefits many people; and that minimizes the need for accommodations. Unfortunately, UD and UDL principles are not routinely applied in the design of cyberlearning technology and pedagogy.
Principles specifically relevant to the design of cyberlearning technology
Many specific barriers to digital tools and content faced by individuals with disabilities today have well-documented solutions. These include those articulated by the Web Content Accessibility Guidelines (WCAG), originally published in 1999 by the World Wide Web Consortium (W3C) and most recently updated to WCAG 2.1 (2018). The Guidelines dictate that all information and user interface components follow the following four guiding principles:
- Perceivable: Users must be able to perceive the content, regardless of the device or configuration they’re using.
- Operable: Users must be able to operate the controls, buttons, sliders, menus, etc., regardless of the device they’re using.
- Understandable: Users must be able to understand the content and interface; and
- Robust: Content must be coded in compliance with relevant coding standards in order to ensure its accurately and meaningfully interpreted by devices, browsers, and assistive technologies.
WCAG details how to apply these four principles and how to measure the success criteria to determine when a website or application is compliant. Accessible practices include captioning and audio describing videos, providing alternative text to describe content in images, using headings at appropriate levels within documents and websites, making link text descriptive, and ensuring that technology can be operated with a keyboard alone. While the Guidelines were developed to apply to web-based technologies, the principles, guidelines, and success criteria can also be applied to digital media, software, and other technologies (W3C, 2013).
As a complement to WCAG, the International Organization for Standardization (ISO) has published ISO 14289, also known as PDF/UA, which provides specific accessibility standards for PDF (ISO, 2014). PDF/UA complements WCAG 2.0, but also contains some specific requirements that are unique to PDF. For example, a PDF file must be a “tagged” PDF (which includes an underlying tag structure for semantically identifying headings, lists, tables, etc.). Also, security settings must allow assistive technology access to the content (some PDF security settings do not); and fonts must be embedded, and text mapped to Unicode (otherwise assistive technologies may be unable to parse the text).
Another relevant set of guidelines is the IMS Guidelines for Developing Accessible Learning Applications (IMS Global Learning Consortium, 2004). These guidelines are relatively old, and it is unclear whether there are plans for updates. However, they nevertheless continue to offer practical suggestions for the design of accessible online learning that researchers and practitioners may find useful:
- Allow for customization based on user preference.
- Provide equivalent access to auditory and visual content based on user preference.
- Provide compatibility with assistive technologies and include complete keyboard access.
- Provide context and orientation information.
- Follow IMS specifications and other relevant specifications, standards, and/or guidelines.
- Consider the use of Extensible Markup Language (XML).
Each of these IMS guidelines is supported by UD, UDL and/or WCAG principles.
The next section of the paper explores how research in the learning sciences can be integrated with UD, UDL, and WCAG principles in the design of inclusive cyberlearning opportunities.
Applying UD, UDL, and WCAG principles and practices supported by research in the learning sciences
Ideally, cyberlearning instructors adopt practices supported by research in how people with a wide range of characteristics learn. There are two problems with this approach. First, most relevant research does not specifically address the effectiveness of practices with respect to students with disabilities and from other underrepresented groups. Even in the hundreds of research articles presented as evidence to support UDL practices (CAST, 2018), most results reported do not address disability-related issues in their research design, selection of participants, data collection, analysis of results, and conclusions. Consequently, the practices supported by these studies can only be considered “promising” rather than “proven” for students with disabilities because they have not been fully tested with this population. Studies that specifically address the efficacy of UD with respect to instructional practices are not plentiful (Roberts, Satkyjgykyjova, & Park, 2015). Researchers and practitioners led by the research committee of the Universal Design for Learning–Implementation and Research Network (UDL-IRN) came to the same conclusion, stating that, despite the growing popularity of UDL, defining UDL practices and reporting their outcomes “remain elusive” (Smith, et al., 2019). The group identified issues that need addressing, which included (a) operationalizing and applying UDL; (b) instruments for the measurement of UDL; (c) teacher education/professional development in UDL; and (d) UDL tools, technologies, and resources.
Another limitation in applying the vast body of research about learning and instruction to cyberlearning is that most of it does not specifically test pedagogy within digital learning environments. Perhaps a reasonable approach for creating inclusive cyberlearning pedagogy at this time is for cyberlearning designers and instructors to select general evidence-based instructional practices and then apply relevant UD, UDL, and WCAG principles to each practice. The process is presented in Figure 2.
This process can be applied by cyberlearning practitioners and researchers to any research-based principles and practices, such as those reported in the book How Learning Works: Seven Research-Based Principles for Smart Teaching. The authors of this book present the following seven research-based principles of learning that can be used to guide the development of instructional practices:
- Students’ prior knowledge can help or hinder learning.
- How students organize knowledge influences how they learn and apply what they know.
- Students’ motivation determines, directs, and sustains what they do to learn.
- To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned.
- Goal-directed practice coupled with targeted feedback enhances the quality of students’ learning.
- Students’ current level of development interacts with the social, emotional, and intellectual climate of the course to impact learning.
- To become self-directed learners, students must learn to monitor and adjust their approaches to learning. (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010, pp. 4-6)
For each principle, the authors of this book recommend best practices to maximize the learning of students. Following the model presented in Figure 2, cyberlearning researchers and practitioners can tailor these best practices to digital learning applications before applying UD, UDL, and WCAG principles to ensure accessibility, usability, and inclusion for diverse audiences in environments that are both on-site and online. For example, one best practice supported by the first principle is for a cyberlearning instructor to determine prerequisite knowledge and skills for an activity, then share with students the terms they need to know and knowledge and skills they need to have before embarking on the activity. The instructor could also present website links where students could remediate each prerequisite where they consider themselves insufficient to fill in the gaps on their own. All instruction and resources shared would be designed to be inclusive of all potential students through the application of established UD, UDL, and WCAG principles, guidelines, and practices.
This approach makes clear that instructors do not need to abandon their favorite teaching and learning philosophies and theories—e.g., learner-centered instruction (Mino, 2004), cooperative learning, constructivist approaches, differentiated instruction (Hall, Strangman, & Meyer, 2003; Santangelo & Tomlinson 2009)—in order to integrate UD, UDL, and WCAG principles into the design of cyberlearning research and practice. Once a specific instructional practice is selected, such as a “flipped” classroom, the three sets of principles and related guidelines and practices can be applied to that strategy (e.g., to ensure that all students can effectively participate and learn when the strategy is applied). For example, if the practice requires small group collaboration, the instructor could provide multiple ways for students to interact with each other (e.g., online bulletin board, email, conference software), require that each group use methods that are accessible to all members of the group, and have groups assign roles that ensure that all members contribute to the effort.
As demonstrated by these examples, UD, UDL, and WCAG principles complement, rather than replace, principles of good digital learning design in the same way that applying UD complements other accepted practices for the design of a specific product—applying UD to an otherwise excellently-design of a microwave oven makes an excellently-designed oven usable by more people, whereas applying UD to a poorly-designed microwave oven makes a poorly-designed oven usable by more people. Applying the combination of UD, UDL, and WCAG principles to cyberlearning designs allows for user choice instead of limitations. Such customization has the potential to benefit many students in addition to those with disabilities.
Besides applying UD, UDL, and WCAG principles and practices to the design of mainstream products, there is a need to develop cyberlearning products that address the unique needs of specific subgroups of individuals with disabilities. For example, for some students with learning disabilities building basic reading and writing is difficult. Students on the autism spectrum may require special assistance in order to understand facial expressions or to engage in group work. Students who are blind may need to learn braille and/or how to use screen readers; students who are deaf may need to learn sign language. However, UD, UDL, and WCAG principles should still be applied in these unique situations because a student on the autism spectrum, for example, may also experience low vision or a hearing impairment. Such intersectionality lends further support for the application of UD and UDL in all cyberlearning applications.
Fourteen principles for inclusive cyberlearning design
Development of cyberlearning products and pedagogy that are engaging and effective for all learners requires that developers assume that potential students and instructors will have a wide variety of abilities; commit to inclusive learning practices; understand challenges individuals with disabilities often face; and apply principles to avoid the creation of barriers to some individuals. As presented earlier in this section of the paper, fourteen well established definitions, principles, guidelines, and practices can be used to guide the design of inclusive digital learning technology and pedagogy—the seven principles of UD of the National Center on Universal Design, the three principles of UDL established by CAST, and the four principles of the WCAG established by the World Wide Web Consortium. Even as cyberlearning research pushes the boundaries of current practices, practitioners and researchers can trust that these basic principles, listed below, will stand the test of time.
- UD1: Equitable use
- UD2: Flexibility in use
- UD3: Simple and intuitive use
- UD4: Perceptible information
- UD5: Tolerance for error
- UD6: Low physical effort
- UD7: Size and space for approach and use
- UDL1: Multiple means of engagement
- UDL2: Multiple means of representation
- UDL3: Multiple means of action and expression
- WCAG1: Perceivable
- WCAG2: Operable
- WCAG3: Understandable
- WCAG4: Robust
Employing practices supported by the combination of UD, UDL, and WCAG principles to cyberlearning technology and pedagogy ensures that
- students have multiple ways to gain and demonstrate knowledge;
- each practice is accessible, usable, and inclusive;
- practices do not lower standards or limit innovation; and
- the need for accommodations for individual students with disabilities is minimized.
Unfortunately, studies have repeatedly revealed that most technologies used in education at all levels fail to employ even basic accessibility practices. Inaccessible technologies include websites (Bray, Flowers, Smith, & Algozzine, 2003; Opitz, Savenye, & Rowland, 2003; Wells & Barron, 2006; Bray et. al, 2007; Krach & Jelenic, 2009; Kimmons, 2017), e-books (Mune & Agee, 2016), computer-based assessment (Kamei-Hannan, 2008), maker tools (Seo, 2019), and a range of instructional technologies (Wisdom et al, 2007; Shaheen & Watulak, 2019). Addressing this problem may require widespread educational efforts to ensure key stakeholders are aware of needs, legal issues, and best practices for inclusive design, and that they are sufficiently motivated to devote the extra time and resources required to ensure accessibility.
In the next section a case study reveals past and current practices for a common cyberlearning application, online learning.
An Example: Historical Perspective and Guidelines for Inclusive Online Learning
One cyberlearning topic with a long history is the delivery of online instruction—including fully online courses as well as blended/hybrid courses where some instruction is offered online and some on-site. As presented in the introduction of this paper, the past few decades have witnessed dramatic increases in the variety of technologies available for teaching online, the number of online courses offered, and the number of students enrolled in these courses. Together, these three trends, along with heightened awareness of ethical and legal considerations, make accessibility issues increasingly complex and important to address in order to ensure that people with disabilities are not left out when online opportunities are offered.
Rose and Meyer (2002) emphasize the importance of providing access to learning for all students through the application of UDL principles to instructional design since “barriers to learning are not... inherent in the capacities of the learner, but arise in learners’ interactions with inflexible educational materials” (p. iv). In online learning, access barriers can occur with the technology used, such as with functions of a learning management system (LMS) or conferencing system, as well as the methods and curriculum selected and developed by instructors, such as content presented within videos and PDF files. Typical LMSs make it possible to create courses accessible to individuals with disabilities; therefore, the accessibility of a course for the most part depends on the choices designers and instructors make in selecting IT and creating digital materials (e.g., documents, videos, collaborative tools outside of the LMS), and pedagogical and assessment methods (e.g., options for gaining and demonstrating knowledge).
It has often been reported that many online learning opportunities unintentionally erect barriers to individuals with disabilities (Behling, 2017; Burgstahler, 2011, 2006, 2007; Coombs, 2010; Fichten et al., 2009; Keeler & Horney, 2007; Kinash, Crichton, & Kim-Rupnow, 2004; Kim-Rupnow, Dowrick, & Burke, 2001; Massengale & Vasquez, 2016; Schmetzke, 2001; Kent, 2015; Pavithran, 2017; University of California, Davis, n.d.; Straumsheim, 2017). Students with disabilities report frustration that some courses use inaccessible documents and uncaptioned videos and do not effectively facilitate interactions between students and faculty that are accessible to everyone (Gladhart, 2010). Almost half of the respondents with disabilities in one study said that they “perceived their disability to have a negative impact on their ability to succeed in online courses” and close to 70% of these students had not disclosed their disabilities to their online instructors (Roberts, Crittenden, & Crittenden, 2011). Female students with learning disabilities taking online courses have reported their learning environments to be less supportive and less satisfactory than females without learning disabilities; they also reported that academic services did not fully address their accommodation needs and were less satisfied than other students with the courses overall (Heiman, 2008).
Participants in the AccessCyberlearning 2.0 CBI (2019) engaged with a panel of five individuals with disabilities related to mobility, sight, learning, and social interactions. According to these panelists, each with experience as a student or instructor in online learning, relatively simple technology and instructional design practices can contribute to their positive experiences in online courses. Their recommendations include those that follow.
- Offer multiple ways to gain knowledge, such as through a video paired with printed materials.
- Provide all materials that are accessible to students with disabilities at the same time they are provided to other students.
- Caption videos to benefit a wide variety of students, including English language learners, those in noisy (e.g., airports) or noiseless (e.g., libraries, buses) environments, and individuals who want to search content, as well as people with hearing and learning disabilities.
- Design videos to include audio content for visual elements of a video whenever possible (e.g., have the credits and other information at the end of a video spoken by the narrator) to maximize access for individuals who are blind or otherwise cannot see the screen. Consider adding audio description to describe other key elements of the content presented visually.
- Provide text descriptions for all visuals.
- Use accessibility designed documents (e.g., PDFs, PowerPoint slides).
- Engage with students in multiple ways.
- In online discussions, to help students, especially those with learning and communications challenges, provide a specific focus to each discussion question, provide guidance in how to answer the question, engage in and guide the discussion, and summarize the group of responses.
- For any on-site components of a course, make sure the facility can be easily maneuvered with a wheelchair. (AccessCyberlearning 2.0, 2019, p. 19-20)
Online instructors and institutions tend to employ an accommodations-only framework rather than one that is proactive such as UD (Barnard-Brak & Sulak, 2010; Kim-Rupnow, Dowrick, & Burke, 2001; Kinash, Crichton, & Rupnow, 2004; Lightfoot & Gibson, 2005; Roscorla, 2016; Fisher & Parsi, 2018). Common accommodations for students with disabilities in online environments are remediating inaccessible documents (e.g., PDFs) and captioning videos.
Strategies for accessible online learning
Practitioners can apply UD, UDL, and WCAG principles to the overall design of the online course (e.g., choosing which LMS features to employ) as well as specific course products (e.g., providing text alternatives to describe the content in graphic images and placing captions on videos) and communication methods (e.g., ensuring that discussions are accessible to students with all types of abilities). UD can also be built into an assignment to make it accessible to everyone. For example, if an instructor chooses working in small groups as a teaching strategy for a specific assignment, instead of individual groups being told what communication tool to use, each group could be required to select a communication method through which everyone can actively participate in group meetings and tasks. For example, in an assignment in an online course, an instructor might assign students to work in small groups to complete a project and answer specific questions to report their work to others in the class. The instructor could emphasize that the tool they select for communication must be usable by everyone in group. This approach places accessibility in the hands of the student group. In making such an assignment, the instructor may discover that asynchronous communication works particularly well for individuals with a variety of abilities that may impact their ability to use synchronous tools at all or to communicate quickly enough within that environment.
There is no shortage of guidance for online course instructors and designers with the desire to make their offerings accessible to individuals with disabilities. Many specific strategies for making online learning accessible to students with disabilities are reported in the literature (e.g., Chambers, Varoglu, & Kasinskaite-Buddeberg, 2016; Cifuentes, Janney, Guerra, & Weir, 2016; Coombs, 2010; Fichten et al., 2009; Dell, Dell, & Blackwell, 2015; Fidalgo & Thormann, 2017; Keeler & Horney, 2007; Pearson & Koppi, 2002; Rao, Edelen-Smith, & Wailehua, 2015; Seale, 2014; Lederman, 2017; Moorefield-Lang, Copeland, & Haynes, 2016; Robinson & Wizer, 2016; Rogers-Shaw, Carr-Chellman, & Choi, 2018; Smith, Lewis, & Moore, 2017; Thomson, Fichten, Havel, Budd, & Asuncion, 2015; Vie, 2018). Relatively easy practices are reported in a publication of twenty tips for faculty members who wish to get started in making their online course using current technology accessible to students with disabilities. It was based on a literature review as well as direct experiences of the author in teaching online. Suggested first steps include those that follow.
For course web pages, documents, images, and videos, follow this advice:
- Use clear, consistent layouts and organization schemes for presenting content.
- Structure headings (using style features built into the Learning Management System and applications such as Microsoft Word and PowerPoint, PDFs, etc.) and use built-in layouts (e.g., for PowerPoint slides).
- Use descriptive wording for hyperlink text (e.g., “DO-IT Knowledge Base” rather than “click here”).
- Minimize the use of PDFs, especially when presented as an image; make sure the text is accessible by testing to see if you can copy and paste it).
- Always offer a text-based alternative as well.
Provide concise alternative text descriptions of content presented within images.
- Use large, bold fonts on uncluttered pages with plain backgrounds.
- Use color combinations that are high contrast and can be read by those who are colorblind.
- Make sure all content and navigation is accessible using the keyboard alone.
- Caption or transcribe video and audio content.
With respect to instructional methods, use these recommendations:
- Assume students come from a range of linguistic and cultural backgrounds and have a wide range of technology skills. Educators need to provide options to tap into learners’ experiences, including those from homes and communities, and create rich experiences for gaining the technology skills needed for course participation.
- Present content in multiple ways (e.g., in a combination of text, video, audio, and/or image format).
- Address a wide range of language skills as you write content (e.g., spell acronyms, define terms, avoid or define jargon).
- Make instructions and expectations clear for activities, projects, and assigned reading.
- Make examples and assignments relevant to learners with a wide variety of interests and backgrounds.
- Offer outlines and other scaffolding tools to help students learn
- Provide adequate opportunities for practice.
- Allow adequate time for activities, projects, and tests (e.g., give details of project assignments in the syllabus so that students can start working on them early).
- Provide feedback on project parts and offer corrective opportunities.
- Provide options for communicating and collaborating that are accessible to individuals with a variety of disabilities.
- Provide options for demonstrating learning (e.g., different types of test items, portfolios, presentations, discussions). (Burgstahler, 2018, p. 2-3)
Besides guidelines for a specific course, there are also inclusive strategies that can be applied to distance learning programs as a whole. One set of guidelines was created by a project funded by the Office of Postsecondary Education at the U.S. Department of Education and led by the DO-IT Center Universal Design in Higher Education (UDHE) initiative collaborators on sixteen campuses engaged in creating a list of characteristics of a universally designed online learning program. Subcategories are organized around key stakeholder groups—students and potential students, distance learning designers, distance learning instructors, and program evaluators. Participants concluded that “employing universal design principles as Internet-based distance learning courses are created can bring us closer to making learning accessible to anyone, anywhere, at any time” (Burgstahler, Corrigan, & McCarter, 2005).
Status regarding the accessibility of online courses
Most articles, books, and web resources about online learning do not discuss access barriers for people with disabilities nor do they include design strategies that can reduce these barriers. For example, a large study that analyzed the performance gap between online learning college student groups as defined by grade point average, age, gender, and race, neglected to report the relative performance of students with disabilities (Xu & Jaggars, 2014). It is no surprise that, in one study, 80% of online learning faculty surveyed had not considered the needs of students with disabilities in developing their courses and less than 12% had at least “partially” considered the needs of students with disabilities in developing their courses (Bissonnette, 2006). Unfortunately, many diversity efforts as well as research projects that explore the outcomes of digital learning for various underrepresented groups do not address disability (e.g., Shallish, 2017).
In relation to this, when overall guidelines for instructional designers and instructors are provided, they often do not address accessibility issues, and, when they do, emphasis is on compliance with legal mandates or after-the-fact accommodations through disability support services. This is true with the popular National Standards for Quality Online Teaching (2019), which were developed and continue to be maintained by a loosely connected organization of K-12 stakeholders. Included within Standard A: Professional Responsibilities is the wording “The online teacher can explain his or her responsibilities in carrying out local or national law or mandates related to accessibility.” Standard F: Diverse Instruction takes a reactive, accommodations approach with such phrases as “the online teacher communicates with appropriate school staff regarding specific accommodations…,” identifies “learners who need additional support services,” “creates alternative formats of course materials, if needed,” and “recommends assistive technologies where appropriate” (p. 21). There is no mention of disability or accessibility in the benchmarks under Standard B: Digital Pedagogy, Standard C: Community Building, Standard D: Learner Engagement, Standard E: Digital Citizenship, Standard G: Assessment and measurement, Standard H: Instructional Design.
Many instructors do not feel confident that they have the knowledge and support to accommodate students with disabilities in their courses that make use of IT (Roberts, Park, Brown, & Cook, 2011), and, not surprisingly, a low percentage of online faculty are aware of techniques for ensuring the accessibility of their courses (Gladhart, 2010). The combined results of three studies support the need for accessibility training for online learning personnel that includes a summary of access challenges for people with disabilities, legal issues, UD guidelines, and online resources (Burgstahler, 2007). Outcomes of a study reported by the Association of Higher Education and Disability also reveal the need for more training on disability issues for researchers and practitioners to improve the climate for students with disabilities. It is noted that many practitioners believe that disability service providers should address all disability issues through accommodations, even though creating a climate that is welcoming and accessible to individuals with disabilities requires the engagement of instructors and online learning designers (Harbour & Greenberg, 2017; Kim, & Aquino, 2017). If accessibility is considered at all in the design of innovations, a typical pattern is to first innovate, then try to fix the accessibility limitations that resulted from the innovation. Retrofitting inaccessible products is often costly and impractical.
Clearly, to increase the accessibility of future online courses, researchers, instructors, and instructional designers must address accessibility issues with respect to the design and delivery of courses. Using established principles, guidelines, and practices, researchers should consider these issues and engage individuals with disabilities in all stages of research efforts.
Examples of Current Research in the Design of Cyberlearning Technology and Pedagogy
Two lines of research make important contributions to the cyberlearning field: (1) creating mainstream digital learning products and pedagogy that are designed to be accessible to all students, including those with a wide range of disabilities, and (2) building products and pedagogy that benefit individuals with specific disabilities. Many examples of the first research category come from applications to online learning, as discussed in the previous section of this paper. The second research category is explored below in looking at the state of current research and technology and how accessibility and disability topics are brought into consideration during that research.
Specific disability cyberlearning technology research
Cyberlearning research that has focused on improving education for targeted groups of students who have disabilities includes the following current work reported by researchers at the AccessCyberlearning 2.0 CBI (2019).
- Aaron Kline at Stanford University leads The Autism Glass Project, which uses expression recognition to provide synchronous feedback to students on the autism spectrum regarding the facial expressions and emotions of people with whom they are interacting. It stores the interactions so that the viewer can review them later to enhance learning.
- Prasun Dewan and fellow researchers at the University of North Carolina at Chapel Hill lead Cyber Support, exploring difficulty detection and difficulty amelioration science, which applies machine intelligence to reduce barriers to learning. Using instructor intervention or automatic recommendations, Cyber Support is designed to automatically identify and assist the learner, and has the potential to benefit both learners and teaching assistants with disabilities, particularly those who have vision impairments.
- Shiri Azenkot and her team at Cornell University are harnessing the potential of 3D printing to create accessible tactile and audio learning experiences for students with vision impairments. This project has developed a toolkit that includes Markit to tag 3D models, and Talkit, which senses the models and, combined with a camera, provides audible feedback and descriptions that identify the tagged components of 3D models as students interact with them.
- Sofia Tancredi at the University of California Berkeley and San Francisco State University leads research into the sensory regulation needs of diverse learners to identify sensory parameters affecting cyberlearning outcomes. Her current project focuses on incorporating amplified sensory inputs such as ankle weights into movement-based math learning for students with low neurological threshold for sensory input (e.g., many individuals with Attention Deficit Hyperactivity Disorder (ADHD), Autism Spectrum Disorder (ASD), and certain mental, emotional, and genetic disorders).
- Mike Jones and his team at Brigham Young University are conducting research in which deaf and hard of hearing students use eyewear that presents a live video feed of a sign language interpreter in a see-through head-mounted display. Students can see the superimposed signing wherever they look in the learning environment. This allows students to see the signed narration without requiring that they look away from the instructor, demonstration, or other visuals.
- Lorna Quandt and her team at Gallaudet University are engaged in a research project called SAIL: Signing Avatars and Immersive Learning, which puts signing avatars in virtual reality environments to create an immersive experience for learning American Sign Language. The project uses high-quality motion capture recordings, gesture recognition, and an interactive gaming engine.
Some of these examples reveal how embodied learning theories have spurred interest in the field of cyberlearning. These theories acknowledge that, although technology has the potential to move discourse and education into digital spaces, there still remains a need to directly engage body movement and the senses in learning processes. The convergence of embodied learning experiences with new technologies such as augmented reality and 3D printing holds promise for increasing accessibility in educational environments as well as creating improved learning tools for all students. At the Embodied Design Research Laboratory at the University of California Berkeley, researchers are investigating how students with sensory processing differences may be differentially affected by the growing trend of embodied learning practices, such as interactive devices used to teach STEM concepts through multiple modalities.
Consistent with the UD approach, many specialized functions such as those described above have the potential to be integrated within mainstream products, just as many features within mainstream technology today originated as assistive technologies for users with disabilities. For example, smartphones include features to enlarge text, customize colors and contrast, predict words as users type in order to minimize the amount of typing required, accept users’ commands by voice, and deliver content using synthesized speech. Technologies such as texting and video chatting were originally created as assistive technologies for people who are deaf but have now become mainstream tools used by all (Ladner, 2016).
In addition to contributing to the development of specialized technologies, it is equally important for cyberlearning practitioners and researchers to consider the wide range of unique human characteristics of potential users of their tools and practices and embrace UD, UDL, and WCAG principles in all of their work. Otherwise, innovations will continue to be innovations for a few, rather than innovations for everyone. A number of cyberlearning research projects are working at the intersection of mainstream educational technology and accessibility. Together, these projects suggest that the next generation of educational tools have the potential to make great strides in creating technology that is not only accessible to those with disabilities but more usable by and inclusive of other individuals as well. By applying inclusive design practices in all phases of research, the cyberlearning community can ensure that individuals with disabilities are not restricted from benefiting from these emerging technologies. Proactive, inclusive designs predict and pre-empt barriers that would otherwise be created.
The Cyberlearning Community Report of the Center for Innovation Research in CyberLearning (CIRCL), describes three fundamental questions around which the cyberlearning community is generally united (Roschelle, Martin, Ahn, & Schank, 2017). Included in Table 1 is each question paired with related accessibility considerations proposed by the authors of this paper.
Table 1 Cyberlearning research questions and accessibility considerations
How can students use their bodies and minds to learn what will be important in the 21st century, such as collaboration, scientific argumentation, mathematical reasoning, computational thinking, creative expression, design thinking, and civic engagement?
Researchers should address this question with all potential students in mind, not just “average” or “typical” students. They should consider the rich variety of ways that individuals, including those with disabilities, use their bodies and minds to interact with the world and engage in the activities listed.
What advances in computation and technology are needed to design, develop, and analyze innovative learning experiences?
Designers of advances in computation and technology should address the needs of individuals with disabilities, including those who use assistive technologies, from the start and throughout development and assessment steps.
How can learning with technology expand access, equity, and depth of learning across diverse people, institutions, and settings?
Researchers and practitioners should consider how to ensure that current and emerging technologies and pedagogy systematically apply UD, UDL, and WCAG principles in order to avoid the creation of barriers for some students and instructors, in particular those with disabilities.
The Cyberlearning Community Report includes illustrative cyberlearning designs and methods summarized in the following two sections along with commentary from the authors of this paper regarding accessibility implications.
Cyberlearning technology designs
The following content illustrates trends in cyberlearning technology designs included and described on page two of the Cyberlearning Community Report along with comments regarding accessibility considerations for each genre provided by the authors of this paper.
Definition and examples: Community mapping includes “mobile, geospatial tools for learning in context at the scale of a neighborhood, community, or city” such as alerting students to learning opportunities along their route (e.g., Cities of Learning) and geocaches, augmented reality walking tours, mapping of important spaces and obstacles in the community (e.g., Mobile City Science).
Accessibility implications: This cyberlearning genre has the potential to expand access to a broad audience as students can map salient features for more independent navigation, for example by considering the unique needs of wheelchair users for accessible pathways, slopes, terrain surfaces, parking spaces, and entrances to buildings and other spaces.
However, to reach its UD potential, researchers and designers need to avoid envisioning a map as a purely visual medium by exploring questions such as, “What is the primary function of the map for both visual and non-visual users?” and “How can the map interface be designed in a way that provides the optimum experience for all users, including those who are non-visual and those who are unable to use a mouse?”
Definition and examples: Expressive construction design considers “computing as a creative literacy, focusing on students’ expressiveness, ability to represent STEM ideas, and sharing of emerging understandings” such as using digital manipulatives and a digital pen to explain reasoning (e.g., INK-12), sketching and animation to model scientific phenomena (e.g., eSiMSAM, DataSketch), collaborative circuit design for a digital underwater environment using blocks and a screen representation (e.g., Oztoc), lowered barrier of entry tools (e.g., Scratch for coding and Makescape for engineering), journalistic activities (e.g., STEM Literacy through Infographics), and makerspaces/fabrication (e.g., FabLearn).
Accessibility implications: This genre has the potential to apply UD, UDL, and WCAG principles, as it has, at its core, a respect for enabling students to have flexibility in how they express their ideas. It embraces the richness of the wide variety of technologies available with which to build things, including a variety of modalities and user interfaces (e.g., gesturing, touching, speaking, sketching, plus traditional modalities such as typing and using a mouse). However, flexibility is key for both technology and pedagogy design and practice for this genre to meet its potential for inclusion. If an instructor limits expressive activities to particular tools, it is likely that the tools selected will be inaccessible to some students. For example, many activities for developing basic computer programming skills and expressing computing literacy are built around Scratch, a block-based graphical programming language developed at the Massachusetts Institute of Technology’s Media Lab (n.d.). Scratch, however, is only accessible to students who have eyesight and can effectively operate a mouse. For these tools to be universally accessible, multimodal and differently-accessible interfaces of equivalent quality will need to be systematically built into tools like Scratch.
An important implication of expressive construction is that not only must all students have means of expression, but these means must also be mutually intelligible across students with different profiles in order for students to be able to engage in the critical process of discourse. For example, a student who is blind must be able to represent and communicate with a sighted peer about what they are learning (Abrahamson, Flood, Siu, & Miele, 2019). This calls for the inclusion of multiple modalities (e.g., tactile, visual, auditory) in learning designs. Contextualized within the larger challenge of balancing engagement and the production of functional products with making progress on rigorous learning goals, it is also critical to ensure that students with disabilities have equitable, ambitious, and appropriate learning goals.
Digital Performance Spaces
Definition and examples: Digital performance spaces explore “Immersive, participatory, social investigations of simulated scientific phenomena that appear to be occupying the entire space of the classroom,” such as participatory simulations. For example, these spaces can see how different interactions in the classroom affect spread of imagined disease and embedded phenomena or allow for moving through the classroom to take seismographic measurements at different distances from a simulated epicenter (e.g. RoomQuake).
Accessibility implications: This genre affords individuals the opportunity to access information and learn content through multiple modalities. Whether an individual may have full or a limited range of mobility and/or speech, individuals can collectively interact within digital performance spaces. For example, through systems such as interactive Science through Technology Enhanced Play (iSTEP), individuals have the flexibility of manipulating an augmented simulation through full body movement, gestures, holding props, and or being placed in the space as a prop (Enyedy, Wagnister, Burke, & Danish, n.d.). Motion tracking technology within this type of digital performance space, grants agency to all users by providing them with a range of opportunities to receive and collaboratively distribute knowledge of how a system or phenomena works with their peers. It is important to note that the simulations within these spaces can be tailored by facilitators in ways that support the inclusion of all participating individuals. For instance, in the iSTEP system, simulated objects can have characteristics such as the movement speed, object quantity, and background colors adjusted in ways that can be suitable for individuals with sensory processing differences. Overall, instructors who aim to incorporate learning activities within digital performance spaces must be mindful of the technology settings when designing these activities. While the technology affords a range of possibilities for individuals to interact with a simulation, instructors should concurrently attend to what features of the activities should or should not be foregrounded so that all users can be ensured access to learning the content at hand.
Virtual Peers and Coaches
Definition and examples: Virtual peers and coaches are agents “that use verbal and nonverbal communication to establish rapport with a student and thereby support engagement in explaining STEM concepts,” such as speech and gesture responsive digital avatars through peers (e.g., Alex, Adriana) or teachers (e.g., Invention Coach).
Accessibility implications: Products in this cyberlearning category have the potential to expand access through scalable feedback individualized in both content and delivery. Examples such as signing avatars show promise in improving learning experiences (see Lorna Quandt’s project under Examples of Current Research in the Design of Cyberlearning Technology and Pedagogy). However, honing the adaptation of virtual agents to diverse learners is critical to their accessibility. It will be important to train virtual agents through interactions with individuals with diverse multimodal communication styles (facial expression, gesture, movement, gaze) in order for virtual agents’ interactions to accurately attune to all students. It will also be important to systematically build in multiple means of avatar expression so as to ensure their full communicative and rapport efficacy is not concentrated in one modality, such as visually presented gestures, actions, and expressions that would limit efficacy for students who are blind or visually impaired. Integration of virtual agents with other interactive media such as expressive construction could potentially support this kind of expansion.
Remote Scientific Labs
Definition and examples: In remote scientific labs, students “control real scientific equipment at a distance, learning about science with authenticity and support,” such as video interaction with remote scientific apparatus and construction of computer simulations through desktop computers or mobile devices (e.g., iLabStudio).
Accessibility implications: Remote labs hold the potential to make diverse scientific manipulations more accessible by providing a proximal, intentionally designed interface for interacting with remote tools. A critical factor is the accessible design of all mobile and computer applications developed for this purpose.
Touch Screen Interfaces
Definition and examples: Touch screen interfaces expand “collaborative learning via multitouch interfaces on tabletop, tablet and mobile computers,” such as controlling tablets through a variety of manipulations rather than using a mouse and improved monitoring of student learning.
Accessibility implications: The expanded gestural control repertoire on tablets expands affordances for participation, so long as multiple options are systematically built into the activities and designs students engage with. The increase in data available to teachers to monitor and review students’ learning processes can also help to ensure all students are fully accessing learning experiences in real time and support teacher professional development in UD and UDL over time.
Cyberlearning technology methods
The following content illustrates trends in cyberlearning technology methods included in the Cyberlearning Community Report along with comments regarding accessibility considerations for each method provided by the authors of this paper.
Definition and examples: Multimodal analysis integrates “multiple streams of data, such as audio, video, eye gaze, sensors, and clickstream data.”
Accessibility implications: Common metrics include gaze patterns, heart rate, accelerometry, movement detection, galvanic skin response, gesture, and expression. The potential to expand access includes personalized, real-time feedback during activities; the ability to flag students who may be struggling; and the ability to learn about student experience in real time. However, to reach this potential, multimodal analytics need to be tested on individuals with diverse characteristics in order to capture differences in expression. Potential access challenges, such as image recognition software failing to recognize people in a wheelchair, engagement identification software misinterpreting gaze patterns of individuals with ASD or movements of individuals with ADHD, and gesture recognition software failing to recognize American Sign Language, will all require further research.
Analytics for assessment
Definition and examples: Analytics for assessment measures “student learning as they use games and other online experiences to inform teachers and increase learning across different types of experiences,” such as IXL and Khan Academy.
Accessibility implications: Products in this genre have the potential to expand access because of the capabilities for real-time differentiation, tailored scaffolding, tailored practice, and real-time feedback. However, assessments are often in only one format that may not accurately represent the abilities and knowledge of learners with diverse communication skills and other characteristics.
User- and community-centered design
Definition and examples: User- and community-centered design engages “users and community members in the design process to make learning tools more attractive, useful and effective.”
Accessibility implications: This genre has the potential to expand access through the inclusion of the voices of users with disabilities. However, there is potential for these products to misrepresent some demographics.
Summary of Current State of Cyberlearning Access
Considering what has been covered in this paper thus far, it is clear that
- Disability is not always addressed as a diversity issue, even in cyberlearning projects focused on broadening participation.
- Many current and emerging cyberlearning technologies and pedagogy do not comply with civil rights legislation under which educational institutions are covered entities, as evidenced by the current annual rate of more than 500 complaints about inaccessible IT filed with the US Department of Education Office for Civil Rights.
- Although definitions, principles and guidelines, and practices exist to guide cyberlearning researchers, developers, and instructors about accessible design of digital learning tools and pedagogy, proactive inclusive approaches are not widely applied.
- To achieve systemic change toward more accessible, usable, and inclusive digital learning opportunities, multiple stakeholder groups must be engaged (e.g., researchers, instructors, instructional designers, computing faculty, IT companies, funding agencies).
Recommendations for Cyberlearning Research and Practice
Routinely applying UD, UDL, and WCAG principles in cyberlearning research and practice requires no less than a paradigm shift from design for typical students to design for all students. This paper was created to provide cross-agency guidance for improving the quality, coherence, and pace of knowledge development in STEM education for six levels of research.
Recommendations for each level of research
NSF’s Common Guidelines for Education Research and Development (2013), a report developed from a collaboration between NSF and the Institute of Education Sciences within the U.S. Department of Education, identifies six types of research, described on page nine of the report. The paradigm shift to more inclusive practices needs to take place within each type of research, as explained below.
Definition: Foundational research “provides the fundamental knowledge that may contribute to improved learning and other relevant education outcomes.”
Recommendation: In these studies researchers should explore how fundamental knowledge may contribute to different education outcomes for students with disabilities.
Early-stage or exploratory research
Definition: Early stage or exploratory research “examines relationships among important constructs in education and learning to establish logical connections that may provide the basis for future interventions or strategies to improve education outcomes.”
Recommendation: Conduct research designs that provide the basis for future interventions or strategies that benefit students with a wide range of abilities.
Design and development research
Definition: Design and development research “develops solutions to achieve a goal related to education/learning, such as student engagement or mastery of a set of skills.”
Recommendations: Ensure that the goal in education/learning applies to students with a wide range of disabilities. Apply UD, UDL, and WCAG principles in the design of project products.
Definition: Efficacy research “allows for testing a strategy or intervention under ‘ideal’” circumstances.”
Recommendations: Apply UD, UDL, and WCAG principles in the design of project products. Be sure that the research includes testers with disabilities.
Definition: Effectiveness research examines the “effectiveness of a strategy or intervention under circumstances that would typically prevail in the target context.”
Recommendations: Apply UD, UDL, and WCAG principles in the design of project products. Include students with disabilities in the testing. If they are not available in the setting selected (such as in a specific offering of an online course), use multi model research to conduct usability tests with students who have disabilities.
Definition: Scale-up research “examines effectiveness in a wide range of populations, contexts, and circumstances.”
Recommendations: Include individuals with disabilities within the population for which the research is conducted. Ensure individuals with disabilities are included in other contexts or circumstances explored.
The following section of this paper provides specific guidelines for cyberlearning researchers as they work to making cyberlearning more inclusive.
Recommendations for cyberlearning projects
The Cyberlearning Community Report: The State of Cyberlearning and the Future of Learning with Technology published by the NSF-funded CIRCL, includes examples of cyberlearning research (Roschelle, Martin, Ahn, & Schank, 2017). Consistent with other literature, there is little indication in the document that accessibility of learning environments to instructors or students with disabilities is routinely explored; twice it is mentioned that cyberlearning technologies have the potential to extend educational opportunities to students or investigators with “disabilities” but without further explanation.
Two lines of research can make important contributions to the cyberlearning field: (1) building products and pedagogy that benefit individuals with specific disabilities and (2) creating mainstream cyberlearning products and pedagogy that are accessible to all students, including those with a wide range of disabilities. Examples of cyberlearning research in each of these categories were shared by participants at the AccessCyberlearning 2.0 CBI (2019). Examples in the first category include research projects focusing on specialized technologies designed to enhance education for targeted groups of students such as students on the autism spectrum, students with vision impairments, students who are deaf, and students with a low neurological threshold for sensory input. Most examples of research in the second category are related to design practices that make an online learning course more inclusive of students and instructors with a wide range of abilities with respect to seeing, hearing, moving, or learning.
Additional research is needed in both areas. In particular, cyberlearning researchers are encouraged to pursue research in both learning science and technology design that specifically addresses the unique needs of individuals who have a wide variety of abilities and other diverse human characteristics. Research is needed to solve accessibility problems that exist with current technologies and to harness emerging technologies into solving accessibility challenges. Cyberlearning researchers are also encouraged to evaluate accessibility implications of current and emerging cyberlearning technologies, e.g., those described in the Cyberlearning Community Report (Roschelle, Martin, Ahn, & Schank, 2017). Researchers must determine how each design might be inaccessible to students with a wide variety of disabilities and explore what additional research, design, or development is needed in order to address accessibility problems that are revealed. Furthermore, research is needed that helps to reduce the cost of accessibility implementation, thereby making implementation more feasible on a large scale (e.g., by automating the creation of accurate alternative text for images and captions for videos).
In addition to identifying specific areas of research focus, recommendations for cyberlearning researchers emerged from a literature review, current practices, and presentations and discussions in the AccessCyberlearning 2.0 capacity building institute (2019). Participants in the project agreed that cyberlearning opportunities should be made available to all potential students, including those with disabilities. However, tensions arose with respect to how much effort toward accessible design researchers should be required to make at different stages of prototype development and how researchers can gain the expertise they need to do so. Participants considered whether new design principles are needed, but came to the conclusion that it’s less about new principles and more about ensuring that the technologies of the future are designed with today's principles and best practices in mind. They also acknowledged a need to apply accessibility principles in developing guidelines and practices for specific cyberlearning types. Most agreed that getting more researchers to routinely consider accessibility in their work will be difficult. The recommendations listed below were supported by many of the project collaborators. They are organized into two groups: Those that can be implemented immediately in existing projects and products, and those that can be implemented in the future.
Recommendations for immediate actions
Immediately, cyberlearning researchers should:
- become familiar with the UD, UDL, and WCAG principles and established guidelines and practices they support as they apply to the design of inclusive cyberlearning tools and pedagogy;
- explore how cyberlearning practices supported by the science of learning can be overlaid with UDL, UDL, and WCAG principles to make them inclusive of individuals with disabilities;
- invite someone with IT accessibility knowledge to be a member of their research teams;
- ensure project staff are trained on basic accessibility principles and standards-compliant coding practices;
- establish internal policies and guidelines for accessibility within their projects, and, if relevant, their departments or institutions;
- consider a broad range of learning styles and disability types during the earliest phases of conceiving and designing a project;
- analyze the experiences of participants with different types of disabilities along with other demographic groups when reporting research results; and
- when reporting limitations of their studies, include accessibility limitations.
Recommendations for future actions
In the future–both in near-term (i.e., 1-3 years) and longer term (3-5 years) timeframes–cyberlearning researchers should develop and promote practices related to the following suggestions that fully embrace disability-related considerations into their research workflows. They should:
- implement an agile, iterative design process that involves users with a wide variety of disabilities and other human characteristics in all phases of the research and design process;
- actively participate in collaboration and communication among academia and industry on issues pertaining to the accessible design of cyberlearning;
- contribute to the development and sharing of guidelines for accessible design of cyberlearning tools and pedagogy;
- avoid being deterred by cost. Low-cost interventions can provide great benefits to users. On the other hand, early designs of innovative technologies may be quite expensive, but long term, if widely adopted and/or if adjustments are made to the design, the cost per user can drop significantly;
- articulate the broad characteristics of potential users in the design of a tool or pedagogy being developed or studied, and specify how the characteristics of various groups of individuals with disabilities will be addressed in the research design;
- resist generalizing all people who share a specific disability when designing technology to improve access for a population—for example, designing something to improve access for people who are blind should consider that not all people who identify as “blind” have the same vision capabilities or personal preferences for learning. Technology design should allow the user to customize their experience;
- if instructor guidelines will be created as part of the study, share information about accessibility issues for students with disabilities, including how some activities/products developed in the project may not be accessible to certain groups (e.g., students who are blind) along with suggested accommodations that might be provided (e.g., working with a sighted person); and
- integrate accessibility recommendations into existing project management practices to keep them on the forefront, rather than being an afterthought. Doing so will allow accessibility to be addressed as an integrated part of the project.
The next section summarizes roles various stakeholders can play in making these efforts a reality.
Roles of Stakeholder Groups
In order for cyberlearning to be effective for all students, including those with disabilities, a wide variety of stakeholders need to be involved in addition to researchers. It is recommended that members of each stakeholder group work to
- understand their own role and actively work to fulfill it.
- understand the roles of other stakeholder groups, consider how the roles of various stakeholders intersect, and seek ways to engage.
- encourage members of other stakeholder groups to follow these same recommendations.
Implementing widespread change involves many stakeholders. The ADKAR model of change offers a useful framework for considering issues from the perspective of each stakeholder group as they collaborate toward effecting positive change (Hiatt, 2006):
- Awareness of need to change: Why should this stakeholder change to more inclusive cyberlearning practices and/or research?
- Desire to participate and support change: How can we motivate them to change?
- Knowledge of how to change: What do they need to know and what is the best way for them to gain this knowledge?
- Ability to implement change on a day-to-day basis: What tools and support do they need?
- Reinforcement to keep change in place: What support and resources might encourage them to continue to work toward more inclusive cyberlearning practices and/or research?
Roles of various stakeholder groups in supporting positive change are suggested in the paragraphs that follow, along with possible supports that can help the group realize change. Guidance for stakeholders who feel unprepared to fulfill their roles can be found in the Resources section of this paper. For each group change leaders can flesh out the ADKAR model as a starting point for tailoring awareness-building activities, professional development, and resources to their needs.
The availability of inclusive cyberlearning technology and practices will increase if cyberlearning researchers routinely include individuals with disabilities and accessibility considerations within every phase of their design, development, and evaluation processes, as described throughout this paper.
People with disabilities and their allies
Students who report accessibility barriers for themselves and/or others with respect to a cyberlearning activity can encourage instructors to design more accessibility, especially when they not only register a concern, but suggest a solution (like placing a 3D printer on an adjustable-height table). For the greater good, parents, caregivers, and other allies of people with disabilities can encourage educational entities to employ practices that promote a paradigm shift to more inclusive practices.
Cyberlearning designers and instructors
Cyberlearning practitioners can apply practices supported by the science of learning overlaid with UD, UDL, and WCAG principles and related guidelines and practices throughout their course development workflows. They can ensure that all tools used for online learning are tested and verified for accessibility and work around inaccessible features, perhaps in collaboration with disability service personnel through accommodations for specific students with disabilities. Their goal should be to make sure students with disabilities get access equal to that of their peers with respect to the syllabus, course materials, discussions, videos, and online resources and documents. Besides employing proactive design strategies, instructors can make themselves available for one-on-one consultations by offering multiple communication methods from which a student can choose. They can begin gaining knowledge in this area by reading the Resources section of this paper, the AccessCyberlearning Guidelines publication (AccessCyberlearning, 2019), other resources linked from the AccessCyberlearning website (n.d.), and published articles (Burgstahler, 2018; Scott, Temple, & Marshall, 2015). Practitioners can encourage those who create guidelines for online course design (e.g., The National Standards for Quality Online Courses, Online Teaching, and Online Programs) to recommend that accessible design principles be applied to reduce the need for accommodations in their courses.
Instructors of K-12 and postsecondary computing courses
The creation of more accessible cyberlearning tools of the future can be promoted when those who teach computer science and other IT-related courses include in their curriculum topics regarding IT access issues for individuals with disabilities, barriers that are erected, and how they can be avoided through standards-compliant and other inclusive design practices. Specifically, they can include accessible/universal/inclusive design, WCAG principles and guidelines, and similar topics in their courses (e.g., teaching web accessibility principles in a course that teaches about web design in general). Although a minority of computing faculty include these topics in their courses (Shinohara, Kawas, Ko, & Ladner, 2018), there is no longer a shortage of ideas for doing so (e.g., Cohen, Fairley, Gerry, Lima, 2005; Ludi, Huenerfauth, Hanson, & Rajendra Palan, 2018; Petrie & Edwards, 2006; Putnam, Dahman, Rose, Cheng, & Bradford, 2016; Rosmaita, 2006; Shinohara, Bennett, Pratt, & Wobbrock, 2018; Waller, Hanson, & Sloan, 2009; Wang, 2012; Welch, 1995).
Makerspace, engineering lab, and other learning facility managers
Individuals in positions to design and manage learning spaces can take into consideration the wide variety of characteristics potential users of the space might have as they make design decisions. They can begin to gain knowledge in this area by consulting the publication Making a Makerspace? Guidelines for Accessibility and Universal Design (AccessEngineering, 2015).
Disability services staff
Disability support staff should be responsive to requests for accommodations from students with disabilities and provide reasonable accommodations that give them access to a course or other cyberlearning activities in a timely manner. In their interactions with faculty and designers, they can encourage the application of inclusive technologies and teaching practices.
Educational administrators and other leaders
Educational leaders at all levels who value diversity, equity, and inclusion can ensure that policies and processes in all areas of the organization reflect this view by proactively addressing potential access issues for individuals with disabilities. Specifically, they can implement policies, guidelines, and procedures related to accessibility, and can ensure that sufficient staffing, training, and resources are available to support these efforts. Diversity, including accessibility issues, could also be included in onboarding training for new employees. A university accessibility policy is one example of a crucial incentive for all stakeholders at the institution to learn the specific requirements, project scope and priorities for implementing accessibility practices. Guided by a high-level policy, each organization within the institution could be encouraged to develop their own guidelines that are applicable to their own stage of accessibility implementation.
Personnel who procure cyberlearning products
To ensure that institutions meet their legal and ethical obligations for access, procurement administrators can put processes in place for those who procure IT and related products and services to ensure that cyberlearning technologies are tested and verified for accessibility prior to purchase and deployment. Procurement leaders can begin to gain knowledge in this area by reading the Guidelines document and visiting the AccessCyberlearning website.
To develop expertise within IT companies, company leaders can include accessibility knowledge and skills among the qualifications in job announcements, and demonstrate a preference for hiring people who meet these qualifications. They can also offer training opportunities to current staff, and encourage colleges and universities to teach about accessibility in computing courses.
Agencies that fund cyberlearning projects can require those projects to develop technology and pedagogy that is accessible to individuals with disabilities. Funded projects could be required to share in project reports and published articles the participation and experiences of individuals with disabilities, design prototypes and project products that comply with WCAG, apply UD principles to on-site and online environments, and incorporate UDL practices in learning activities. In cases where innovative designs are in early stages of development and there are not enough resources to make the designs fully accessible, projects should, within their reports and published articles, recommend future research that would address accessibility issues.
As a wide range of digital technologies become more abundant in formal and informal learning opportunities, the need to make technological innovations and pedagogical practices more inclusive of potential students and instructors with disabilities is critical. This paper reveals key information for creating accessible cyberlearning.
- US civil rights legislation requires that students with disabilities have access to educational opportunities, including opportunities that make use of IT.
- There is little evidence that cyberlearning technology and pedagogy research and practice routinely address access issues for individuals with disabilities.
- Established principles, guidelines, and practices currently exist to guide the development and use of accessible, usable, and inclusive cyberlearning technology and pedagogy.
- Some accessible design practices that focus on disabilities benefit other groups as well.
- To achieve systemic change toward more inclusive cyberlearning, it is important that researchers, instructors and course designers, computing faculty, IT companies, funding agencies, and other stakeholder groups be engaged.
- Stakeholders need training and resources tailored to their particular roles in ensuring that future cyberlearning innovations are more inclusive of individuals with disabilities.
As a wide range of digital technologies become more abundant in formal and informal learning, the need to make these technological innovations as well as pedagogical practices accessible to, usable by, and inclusive of potential students and instructors with disabilities is critical. Further, this paper provides illustrations of how UD practices benefit many students besides those with disabilities. Thus, inclusive practices are arguably a best practice in the delivery of instruction. This paper serves to inform the use of current digital learning technology and pedagogy as well as the design of the next generation of digital learning environments.
The ideal state for future cyberlearning research is that researchers routinely include individuals with disabilities and accessibility considerations within every phase of research design, development, and evaluation processes. Reaching this goal requires a paradigm shift from designing for some to designing for everyone.
About AccessCyberlearning Projects
This paper, as well as the summary document Accessible Cyberlearning: A Community Report of the Current State and Recommendations for the Future (Burgstahler & Thompson, 2019a) and a short Guidelines for Cyberlearning Researchers publication (Burgstahler & Thompson, 2019b), are products of the AccessCyberlearning 2.0 project. The purpose of this project is to inform the design of the next generation of cyberlearning environments for STEM. AccessCyberlearning 2.0 is funded by the NSF’s Cyberlearning and Future Learning Technologies program of the Division of Information and Intelligent Systems (#DRL-1824540), which supports projects that work within the space defined by the intersection of digital technology and learning. These projects integrate the capabilities of current and emerging technologies with advances in what is known about how people learn, advance innovation and research in these areas, and promote widespread use of new genres. AccessCyberlearning 2.0 serves to ensure that “widespread use” includes individuals with disabilities.
AccessCyberlearning 2.0 is led by the DO-IT (Disabilities, Opportunities, Internetworking, and Technology) Center at the University of Washington. Project activities that led to the creation of project publications engaged staff and collaborators in a two and one-half day on-site capacity building institute (CBI) in January of 2019 and within an ongoing online community (AccessCyberlearning 2.0, n.d.).
In an earlier AccessCyberlearning project, a CBI (AccessCyberlearning, 2016) and online community were conducted to help NSF-funded cyberlearning projects make their activities and products accessible to and inclusive of individuals with disabilities; it culminated in the publication Equal Access: How to Broaden Participation in Cyberlearning Projects to Ensure Access to People with Disabilities (Burgstahler, 2017b). AccessCyberlearning 2.0 addresses questions that emerged from this earlier work to further identify digital accessibility problems and solutions and make recommendations for future cyberlearning environments that are fully accessible to, usable by, and inclusive of students and instructors with disabilities. Together, these AccessCyberlearning projects align with NSF’s Innovation Cycle of Educational Practice and Research presented in Figure 3 (Jamieson & Lohmann, 2009).
The first AccessCyberlearning project engaged in work at the “Educational Practice” and “Questions and Ideas” levels of the Cycle. AccessCyberlearning 2.0 further identified “Questions and Ideas,” and created this comprehensive paper, a summary document (Burgstahler & Thompson, 2019a), digital learning guidelines (Burgstahler & Thompson, 2019b), and related products to provide an overview of educational research and share answers and insights that can in turn guide future educational practice and research to ensure that individuals with disabilities fully benefit from current and future digital learning opportunities.
There is no shortage of resources that can provide knowledge related to the accessible design of cyberlearning technology and pedagogy. Listed below are good places to start.
Center for Universal Design in Education
Disabilities, Opportunities, Internetworking, and Technology (DO-IT)
Guidelines for Developing Accessible Learning Applications
IT Accessibility Risk Statements and Evidence https://library.educause.edu/resources/2015/7/it-accessibility-risk-statements-and-evidence
Universal Design for Learning udlguidelines.cast.org/?utm_medium=web&utm_campaign=none&utm_source=cast-about-udl
Web Content Accessibility Guidelines (WCAG) 2.1
Abrahamson, D., Flood, V. J., Miele, J. A., & Siu, Y.-T. (2019). Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students. In E. de Freitas, N. Sinclair, & L. Healy (Eds.) Mathematical bodies: Material ecologies of teaching and learning [SI]. ZDM Mathematics Education, 51(2), 291-303. https://doi.org/10.1007/s11858-018-0998-1
AccessCyberlearning 2.0. (2019). Capacity building institute. Seattle: DO-IT, University of Washington. Retrieved from https://www.washington.edu/doit/accesscyberlearning-20-capacity-building-institute-2019
AccessCyberlearning. (2016). Proceedings from the 2016 AccessCyberlearning Capacity Building Institute. Seattle: DO-IT, University of Washington. Retrieved from https://www.washington.edu/doit/accesscyberlearning-capacity-building-institute-2016
AccessEngineering. (2015). Making a makerspace? Guidelines for accessibility and universal design. Retrieved from https://www.washington.edu/doit/making-makerspace-guidelines-accessibili...
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. John Wiley & Sons, Inc.: San Francisco, CA.
American Association for the Advancement of Science. (2001). In pursuit of a diverse science, technology, engineering, and mathematics workforce. Washington, DC: Author.
Americans with Disabilities Act of 1990. 42 U.S.C.A. § 12101 et seq.
Americans with Disabilities Act Amendments Act of 2008. 42 U.S.C.A. § 12102 note.
Barnard-Brak, L., & Sulak, T. N. (2010). Online versus face-to-face accommodations among college students with disabilities. American Journal of Distance Education, 24, 81-91.
Behling, K. (2017). Accessibility considerations for hybrid courses. New Directions for Teaching and Learning, 149, 89-101. https://onlinelibrary.wiley.com/doi/full/10.1002/tl.20230
Bissonnette, L. (2006). Meeting the evolving education needs of faculty in providing access for university students with disabilities (Doctoral dissertation, Concordia University, Montreal). Retrieved from http://files.eric.ed.gov/fulltext/EJ1002144.pdf
Bray, M., Pugalee, D., Flowers, C. P., & Algozzine, B. (2007). Accessibility of middle schools’ web sites for students with disabilities. The Clearing House, 80(4), 169–178.
Bray, M., Flowers, C. P., Smith, S., & Algozzine, R. F. (2003). Accessibility of elementary schools’ web sites for students with disabilities. Education, 123(4), 815–830.
Burgstahler, S. (2018). Twenty tips for teaching an accessible online course [Brochure]. Seattle, WA: DO-IT, University of Washington. Retrieved from https://www.washington.edu/doit/20-tips-teaching-accessible-online-course
Burgstahler, S. (2017a). ADA compliance for online course design. Educause Review. Retrieved from https://er.educause.edu/articles/2017/1/ada-compliance-for-online-course-design
Burgstahler, S. (2017b). Equal access: How to broaden participation in cyberlearning projects to ensure access to people with disabilities. Seattle: University of Washington. Retrieved from https://www.washington.edu/doit/equal-access-how-broaden-participation-c...
Burgstahler, S. (Ed). (2015a). Universal design in higher education: From principles to practice. Cambridge: Harvard Education Press.
Burgstahler, S. (2015b). Universal design of instruction: From principles to practice. In S. Burgstahler (Ed.), Universal design in higher education: From principles to practice (pp. 31-64). Boston: Harvard Education Press.
Burgstahler, S. (2011). Universal design: Implications for computing education. ACM Transactions on Computing Education, 11(3).
Burgstahler, S. (2007). Accessibility training for distance learning personnel. Access Technology Higher Education Network (ATHEN) E-Journal, 2.
Burgstahler, S. (2006). The development of accessibility indicators for distance learning programs. Research in Learning Technology, 14(1), 79-102.
Burgstahler, S., Corrigan, B., & McCarter, J. (2005). Applying universal instructional design to course websites by using course evaluations. Collected Essays on Learning and Teaching, 5.
Burgstahler, S., & Thompson, T. (Eds). (2019a). Accessible cyberlearning: A community report of the current state and recommendations for the future. Seattle: University of Washington. Retrieved from https://circlcenter.org/events/synthesis-design-workshops
Burgstahler, S., & Thompson, T. (Eds). (2019b). Guidelines for cyberlearning researchers. Seattle: University of Washington. Retrieved from https://www.washington.edu/doit/guidelines-cyberlearning-researchers
Center for Applied Special Technology (CAST). (2018). Universal design for learning guidelines version 2.2. Retrieved from http://udlguidelines.cast.org/more/research-evidence
Center for Universal Design. (n.d.). History of universal design. Retrieved from https://projects.ncsu.edu/design/cud/about_ud/udhistory.htm
Chambers, D., Varoglu, Z., & Kasinskaite-Buddeberg, I. (2016). Learning for all: Guidelines on the inclusion of learners with disabilities in open and distance learning. Paris, France: United National Educational, Scientific, and Cultural Organization.
Cifuentes, L., Janney, A., Guerra, L., & Weir, J. (2016). A working model for complying with accessibility guidelines for online learning. TechTrends: Linking Research and Practice to Improve Learning, 60(6).
Cohen, R.F., Fairley, A.V., Gerry, D., Lima, G.R.: Accessibility in introductory computer science. In proceedings of the 36th SIGCSE Technical Symposium on Computer Science Education (pp. 17–21). New York: ACM Press. http://doi.org/10.1145/1047344.1047367
Coombs, N. (2010). Making online teaching accessible: Inclusive course design for students with disabilities. San Francisco, CA: Jossey-Bass.
Cordova, F. A. (2017). Remarks. Arlington, VA: National Science Foundation. Retrieved from https://www.nsf.gov/news/speeches/cordova/17/fc170106_includes.jsp
Dell, C. A., Dell, T. F., & Blackwell, T. L. (2015). Applying universal design for learning in online courses: Pedagogical and practical considerations. Journal of Educators Online, 13(2), 166-192.
DePoy, E., & Gibson, S. (2008a). Disability studies: Origins, current conflict, and resolution. Review of Disability Studies, 4(4), 33–40.
DePoy, E., & Gibson, S. (2008b). Healing the disjuncture: Social work disability practice. In K. M. Sowers & C. N. Dulmus (Series Eds.), & B. W. White (Vol. Ed.), Comprehensive handbook of social work and social welfare: Volume 1, The profession of social work (pp. 267–282). Hoboken, NJ: Wiley.
Digital Learning Compass. (2017). Distance education enrollment report 2017. Retrieved from http://onlinelearningsurvey.com/reports/digtiallearningcompassenrollment2017info.pdf
EDUCAUSE. (2015). IT accessibility risk statements and evidence. Washington DC: Author. Retrieved from https://library.educause.edu/resources/2015/7/it-accessibility-risk-stat...
Enyedy, N., Wagnister, C., Burke, J., & Danish, J. (n.d.). DIP: Collaborative research: Interactive Science Through Technology Enhanced Play (iSTEP). Center for Innovative Research in Cyberlearning. Retrieved from https://circlcenter.org/dip-collaborative-research-istep/
Fidalgo, P., & Thormann, J. (2017). Reaching students in online courses using alternative formats. International Review of Research in Open and Distributed Learning, 18(2). http://dx.doi.org/10.19173/irrodl.v18i2.2601
Fichten, C. S., Ferraro, V., Asuncion, J. V., Chwojka, C., Barile, M., Nguyen, M. N., Klomp, R., & Wolforth, J. (2009). Disabilities and e-Learning problems and solutions: An exploratory study. Educational Technology & Society, 12(4), 241–256.
Fisher, J. F., & Parsi, A. (2018). Why true equity in learning depends on proactive, not reactive, design. Getting Smart. Retrieved from https://www.gettingsmart.com/2018/02/why-true-equity-in-learning-depends-on-proactive-not-reactive-design/
Gabel, S., & Peters, S. (2010). Presage of a paradigm shift: Beyond the social model of disability toward resistance theories of disability. Disability & Society, 19(6), 585–600.
Gil-Kashiwabara, E., Hogansen, J. M., Geenan, S., Powers, K., & Powers, L. E. (2007). Improving transition outcomes for marginalized youth. Career Development for Exceptional Individuals, 30(2), 80–91.
Gladhart, M. A. (2010). Determining faculty needs for delivering accessible electronically delivered instruction in higher education. Journal of Postsecondary Education and Disability, 22(3), 185-196.
Hall, T., Strangman, N., & Meyer, A. (2003). Differentiated instruction and implications for UDL implementation. Wakefield, MA: National Center for Accessing the General Curriculum.
Harbour, W. S., & Greenberg, D. (2017, July). Campus climate and students with disabilities. NCCSD Research Brief, 1(2). Huntersville, NC: National Center for College Students with Disabilities, Association on Higher Education and Disability.
Heiman, T. (2008). Females with learning disabilities taking on-line courses: Perceptions of the learning environment, coping and well-being. Journal of Postsecondary Education and Disability, 21(1), 4-13.
Hiatt, J. (2006). ADKAR: a model for change in business, government, and our community. Prosci.
IMS Global Learning Consortium. (2004). IMS guidelines for developing accessible learning applications. Retrieved from https://www.imsglobal.org/accessibility/accessiblevers
Individuals With Disabilities Education Act, 20 U.S.C. § 1400 (2004).
International Standards Organization (2014). ISO 14289-1:2014: Document management applications - Electronic document file format enhancement for accessibility - Part 1: Use of ISO 32000-1 (PDF/UA-1). Retrieved from https://www.iso.org/standard/64599.html
Jacobs, J. E., & Simpkins, S. D. (Eds.) (2006). Leaks in the pipeline to math, science, and technology careers New directions for child development, No. 110. San Francisco: Jossey-Bass.
Jamieson, L. H., & Lohmann, J. R. (2009). Creating a culture for scholarly and systematic innovation in engineering education: Ensuring US engineering has the right people with the right talent for a global society. Washington, DC: American Society for Engineering Education.
Kamei-Hannan, C. (2008). Examining the accessibility of a computerized adapted test using assistive technology. Journal of Visual Impairment & Blindness, 102(5), 261–271.
Keeler, C. G., & Horney, M. (2007). Online course designs: Are special needs being met? The American Journal of Distance Education, 21(2), 61-75.
Kent, M (2015). Disability and elearning: Opportunities and barriers. Disability Studies Quarterly, 25(1).
Kim-Rupnow, W. S., Dowrick, P. W., & Burke, L.S. (2001). Implications for improving access and outcomes for individuals with disabilities in post-secondary distance education. The American Journal of Distance Education, 18(1), 51-62.
Kim, E., & Aquino, K. C. (2017). Disability as diversity in higher education. New York, NY: Routledge.
Kimmons, R. (2017). Open to all? Nationwide evaluation of high-priority web accessibility considerations among higher education websites. Journal of Computing in Higher Education, 29(3), 434–450.
Kinash, S., Crichton, S. & Kim-Rupnow, W. S. (2004). A review of 2000-2003 literature at the intersection of online learning and disability. American Journal of Distance Education, 18(1), 5-19.
Krach, S., & Jelenic, M. (2009). The other technological divide: K-12 web accessibility. Journal of Special Education Technology, 24(2), 31–37.
Ladner, R. (2016). People with disabilities in computing. In proceedings from AccessComputing's Building Capacity to Increase the Participation of People with Disabilities in Computing (pp. 13-14). Seattle, WA: AccessComputing.
Ladner, R. E. (2015). Design for user empowerment. Interactions, 22(2), 24–29. https://doi.org/10.1145/2723869
Leake, D., Burgstahler, S., Rickerson, N., Applequist, K., Izzo, M., Arai, M., & Picklesimer, T. (2006). Literature synthesis of key issues in supporting culturally and linguistically diverse students with disabilities to succeed in postsecondary education. Journal of Postsecondary Education and Disability, 18(2), 149–165.
Lederman, D. (2018). Online education ascends. Inside Higher Ed. Retrieved from https://www.insidehighered.com/digital-learning/article/2018/11/07/new-data-online-enrollments-grow-and-share-overall-enrollment
Lederman, D. (2017). Understanding the faculty role in digital accessibility. Inside Higher Ed. Retrieved from https://www.insidehighered.com/digital-learning/article/2017/03/15/digital-accessibility-experts-discuss-how-they-approach-faculty
Lightfoot, E., & Gibson, P. (2005). Universal instructional design: A new framework for accommodating students in social work courses. Journal of Social Work, 41(2), 269-277.
Loewen, G., & Pollard, W. (2010). The social justice perspective. Journal of Postsecondary Education and Disability, 23(1), 5–18.
Ludi, S., Huenerfauth, M., Hanson, V., Rajendra Palan, N., & Garcia, P. (2018, February). Teaching inclusive thinking to undergraduate students in computing programs. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (pp. 717-722). New York: ACM Press. http://doi.org/10.1145/3159450.3159512
Massengale, L. R., & Vasquez, E., III. (2016). Assessing accessibility: How accessible are online courses for students with disabilities? Journal of the Scholarship of Teaching and Learning, 16(1), 69-79.
Mino, J. (2004). Planning for inclusion: Using universal instructional design to create a learner-centered community college classroom. Equity & Excellence in Education, 37(2), 154–160.
MIT Media Lab. (n.d.). Scratch. Retrieved from https://scratch.mit.edu/
Moorefield-Lang, H., Copeland, C. A., & Haynes, A. (2016). Accessing abilities: Creating innovative accessible online learning environments and putting quality into practice. Education for Information, 32(1), 27-33.
Moriarty, M. A. (2007). Inclusive pedagogy: Teaching methodologies to reach diverse learners in science instruction. Equity and Excellence in Education, 40(3), 252–265.
Mune, C. & Agee, A. (2016). Are e-books for everyone? An evaluation of academic e-book platforms' accessibility features. Journal of Electronic Resources Librarianship, 28(3), 172-182.
National Science Foundation. (n.d.). Advanced Search Results: Cyberlearning for Work at the Human-Technology Frontier awards. Retrieved from https://www.nsf.gov/awardsearch/advancedSearchResult?ProgEleCode=8020&BooleanRef=All
National Science Foundation. (2017). Committee on Equal Opportunities in Science and Engineering (CEOSE) 2015-2016 biennial report to Congress. Washington, D.C.: U.S. Government Printing Office. Retrieved from https://www.nsf.gov/od/oia/activities/ceose/index.jsp
National Science Foundation. (2013). Common guidelines for education research and development. Washington, DC: U.S. Government Printing Office. Retrieved from https://www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf
National Science Foundation. (2006). Investing in America's future: Strategic Plan FY 2006-2011. Arlington, VA: Author. Retrieved from https://www.nsf.gov/pubs/2006/nsf0648/NSF-06-48.pdf
National Standards for Quality Online Teaching. (2019). National standards for quality online teaching. Quality Matters and Virtual Learning Leadership Alliance. Retrieved from https://www.nsqol.org/the-standards/quality-online-teaching/
Office for Civil Rights. (2016). Securing equal educational opportunity: Report to the president and secretary of education (p. 32). Washington, DC: U.S. Department of Education. Retrieved from https://www2.ed.gov/about/reports/annual/ocr/report-to-president-and-sec...
Office of Civil Rights. (2013). South Carolina Technical College System OCR compliance review No. 11-11-6002. Washington, DC: U.S. Department of Education. Retrieved from https://www2.ed.gov/about/offices/list/ocr/docs/investigations/11116002-b.html
Opitz, C., Savenye, W., & Rowland, C. (2003). Accessibility of state department of education home pages and special education pages. Journal of Special Education Technology, 18(1), 17–28.
Pavithran, S. D. (2017). Expert consensus on barriers to college and university online education for students with blindness and low vision (Doctoral dissertation). Utah State University, Logan, Utah.
Pearson, E. J., & Koppi, T. (2002). Inclusion and online learning opportunities: Designing for accessibility. Association for Learning Technology Journal, 10(2), 17–28.
Petrie, H., & Edwards, A. (2006). Inclusive design and assistive technology as part of the HCI curriculum. In proceedings of HCI Educators Workshop’2006 (pp. 23-24). Limerick, Ireland. Retrieved from http://bit.ly/2XMnm21
Putnam, C., Dahman, M., Rose, E., Cheng, J., & Bradford, G. (2016b). Best practices for teaching accessibility in university classrooms: cultivating awareness, understanding, and appreciation for diverse users. ACM Transactions on Accessible Computing (TACCESS), 8(4), 1–26.
Rao, K., Edelen-Smith, P., & Wailehua, C. (2015). Universal design for online courses: Applying principles to pedagogy. Open Learning, 30(1), 35–52.
Roberts, J. B., Crittenden, L. A., & Crittenden, J. C. (2011). Students with disabilities and online learning: A cross-institutional study of perceived satisfaction with accessibility compliance and services. Internet and Higher Education, 14(4), 242–250.
Roberts, K. D., Park, H. J., Brown, S., & Cook, B. (2011). Universal design for instruction in postsecondary education: A systematic review of empirically based articles. Journal of Postsecondary Education and Disabilities Journal, 24(1), 4–18.
Roberts, K. D., Satkyjgykyjova, M., & Park, H-J. (2015). Universal design for instruction in postsecondary education: A literature review of empirically based articles. In Universal design in higher education: From principles to practice (pp. 65-80). Boston: Harvard Education Press.
Robinson, D. E., & Wizer, D. R. (2016). Universal design for learning and the Quality Matters guidelines for the design and implementation of online learning events. Journal of Technology in Teaching and Learning, 12(1), 17-32.
Rogers-Shaw, C., Carr-Chellman, D. J., & Choi, J. (2018). Universal design for learning: Guidelines for accessible online instruction. Adult Learning, 29(1), 20-31.
Roschelle, J., Martin, W., Ahn, J., Schank, P. (Eds.) (2017). Cyberlearning community report: The state of cyberlearning and the future of learning with technology. Menlo Park CA: SRI International. Retrieved from http://circlcenter.org/wp-content/uploads/2017/07/CyberlearningCommunityReport2017.pdf
Roscorla, T. (2016, January 28). The struggle to make online courses accessible in higher ed. Center for Digital Education. Retrieved from https://www.govtech.com/education/higher-ed/The-Struggle-to-Make-Online-Courses-Accessible-in-Higher-Ed.html
Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Alexandria, VA: Association for Supervision and Curriculum Development.
Rosmaita, B. J. (2006, October). Accessibility now!: Teaching accessible computing at the introductory level. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 277-278). New York: ACM Press. http://doi.org/10.1145/1168987.1169053
Santangelo, T., & Tomlinson, C. A. (2009). The application of differentiated instruction in postsecondary environments: Benefits, challenges, and future directions. International Journal of Teaching and Learning in Higher Education, 20(3), 307–323.
Scott, L. A., Temple, P., & Marshall, D. (2015). UDL in Online College Coursework: Insights of Infusion and Educator Preparedness. Online Learning, 19(5), 99-119. Retrieved from https://files.eric.ed.gov/fulltext/EJ1085764.pdf
Schmetzke, A. (2001). Online distance education: Anytime, anywhere but not for everyone. Information Technology and Disability, 7(2). Retrieved from http://itd.athenpro.org/volume7/number2/axel.html
Seale, J. (2014). E-learning and disability in higher education: Accessibility theory and practice. 2nd Edition. New York: Routledge.
Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase: Tracking distance education in the United States. Babson Park, MA: Babson Survey Research Group. Retrieved from http://onlinelearningsurvey.com/reports/gradeincrease.pdf
Section 504 of the Rehabilitation Act of 1973, as amended. 29 U.S.C. § 794.
Seo, J. (2019). Is the maker movement inclusive of ANYONE?: Three accessibility considerations to invite blind makers to the making world. Tech Trends, 1-7. https://doi.org/10.1007/s11528-019-00377-3
Shaheen, N. & Watulak, S. L. (2019). Bringing disability into the discussion: Examining technology accessibility as an equity concern in the field of instructional technology. Journal of Research on Technology in Education, 1-15. https://doi.org/10.1080/15391523.2019.1566037
Shallish, L. (2017). A different diversity? Challenging the exclusion of disability studies from higher education research and practice. In E. Kim & K. C. Aquino (Eds.), Disability as diversity in higher education: Policies and practices to enhance student success (19-30). New York, NY: Routledge.
Shinohara, K., Kawas, S., Ko, A. J., & Ladner, R. E. (2018, February). Who teaches accessibility?: A survey of US computing faculty. In proceedings of the 49th ACM Technical Symposium on Computer Science Education (pp. 197-202). New York: ACM Press. http://doi.org/10.1145/3159450.3159484
Shinohara, K., Bennett, C. L., Pratt, W., & Wobbrock, J. O. (2018). Tenets for social accessibility: Towards humanizing disabled people in design. ACM Transactions on Accessible Computing (TACCESS), 11(1), 6. http://doi.org/10.1145/3178855
Smith, S. J., Rao, K., Lowrey, K. A., Gardner, J. E., Moore, E., Coy, K., Marino, M., and Wojcik, B. (2019). Recommendations for a national research agenda in UDL: Outcomes from the UDL-IRN preconference on research. Journal of Disability Policy Studies Research Brief. Retrieved from https://journals.sagepub.com/doi/full/10.1177/1044207319826219
Smith, T. L., Lewis, C. & Moore, E. B. (2017). Description strategies to make an interactive science simulation accessible. The Journal on Technology and Persons with Disabilities, 590, 225-238.
Stern, V., & Woods, M. (2001). Roadmaps and rampways. Washington, DC: American Association for the Advancement of Science.
Story, M. F., Mueller, J. L., & Mace, R. L. (1998). The principles of universal design and their application. In M. F. Story, M. L. Mueller, & R. L. Mace. The universal design file: Designing for people of all ages and abilities, pp. 32–36. Raleigh, NC: Center for Universal Design. Retrieved from http://www.ncsu.edu/ncsu/design/cud/pubs_p/docs/udffile/chap_3.pdf
Straumsheim, C. (2017, May 17). Glacial progress on digital accessibility. Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2017/05/18/data-show-small-improveme...
Teach Access. (n.d.). Teach Access. Retrieved from http://teachaccess.org/
Thomas, T., & Allen, A. (2006). Gender differences in students’ perceptions of information technology as a career. Journal of Information Technology Education, 5, 165–178.
Thomson, R., Fichten, C. S., Havel, A., Budd, J., & Asuncion, J. (2015). Blending universal design, e-learning, and information and communication technologies. In S. Burgstahler (Ed.), Universal Design in Higher Education (2nd ed.) (275–284). Cambridge, MA: Harvard Education Press.
University of California, Davis. (n.d.) Accessibility for online courses. Canvas. Retrieved from https://canvas.ucdavis.edu/courses/34528/pages/accessibility-for-online-courses
Vie, S. (2018). Effective social media use in online writing classes through universal design for learning (UDL) principles. Computers and Composition, 49, 61-70.
Waller, A., Hanson, V., & Sloan, D. (2009). Including accessibility within and beyond undergraduate computing courses. Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility, 155-162.
Waller, A., Hanson, V. L., & Sloan, D. (2009, October). Including accessibility within and beyond undergraduate computing courses. In proceedings of the 11th international ACM SIGACCESS Conference on Computers and Accessibility (pp. 155-162). New York: ACM Press. Retrieved from http://sonify.psych.gatech.edu/~walkerb/classes/studio/
Wang, Y. D. (2012, October). A holistic and pragmatic approach to teaching web accessibility in an undergraduate web design course. In Proceedings of the 13th Annual Conference on Information Technology Education (pp. 55-60). New York: ACM Press. http://doi.org/10.1145/2380552.2380568
Welch, P. (Ed.). (1995). Strategies for teaching universal design. Berkeley, CA & Boston, MA: Adaptive Environment & MIG Communications.
Wells, J. A., & Barron, A. E. (2006). School web sites: Are they accessible to all? Journal of Special Education Technology, 21(3), 23–30.
Wisdom, J. P., White, N., Goldsmith, K., Bielavitz, S., Rees, A., & Davis, C. (2007). Systems limitations hamper integration of accessible information technology in northwest U.S. K-12 schools. Educational Technology & Society, 10(3), 222–232.
Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, H., & Froehlich, J. (2011). Ability-based design: Concept, principles and examples. ACM Transactions on Accessible Computing, 3(3). http://doi.org/10.1145/1952383.1952384
World Wide Web Consortium. (2013). Guidance on Applying WCAG 2.0 to Non-Web Information and Communications Technologies. Retrieved from https://www.w3.org/TR/wcag2ict/
World Wide Web Consortium. (2018). Web content accessibility guidelines, 2.1. Retrieved from http://www.w3.org/TR/wcag21/
Wulf, W. A. (2000). How shall we satisfy the long-term educational needs of engineers? Proceedings of the IEEE: Institute of Electrical and Electronics Engineers, 88(4), 593–596.
Xu, D., & Jaggars, S. S. (2014). Performance gaps between online and face-to-face courses: Differences across types of students and academic subject areas. Journal of Higher Education, 85(5), 633-659.
Yardi, S. & Bruckman, A. (2007). What is computing?: Bridging the gap between teenagers' perceptions and graduate students' experiences. In proceedings of the Third International Workshop on Computing Education Research (pp. 39–50). Atlanta, GA: ACM Press.
This material is based upon activities and research funded by the NSF under Grant #DRL-182540 and #IIS-1550477. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the CBI presenters, attendees, and publication authors and do not necessarily reflect the views of the NSF.
Copyright © 2019 University of Washington. Permission is granted to copy these materials for educational, noncommercial purposes provided the source is acknowledged.