In selecting its practices, DO-IT has been guided by white papers and committee reports, evidence-based practices of other programs, and published research.
Critical Junctures Model
Grounded in research and practice, the Critical Junctures Model was created as a useful tool for organizing evidence-based practices and promising interventions for high school, college, and graduate students with disabilities. As new projects unfold, project staff modify and enhance the Model in light of new evidence.
Published White Paper and Committee Reports
Project interventions for students with disabilities incorporate the five conditions for nurturing healthy, successful children identified by America's Promise - The Alliance for Youth. These five conditions are:
- Caring adults
- Safe places and constructive use of time
- A healthy start and healthy development
- Effective education for developing marketable skills and lifelong learning
- Opportunities to make a difference through helping others
Reference: America's Promise - The Alliance for Youth. (2006). Every child, every promise: A report on our nation's young people. Alexandra, VA: Author.
Through project activities, participants in DO-IT projects sponsored by the National Science Foundation (NSF) are encouraged to learn science (attraction), choose to keep learning mathematics and science (retention), graduate (persistence), and continue into STEM careers - steps encouraged by the NSF Committee on Equal Opportunities in Science and Engineering.
Reference: Committee on Equal Opportunities in Science and Engineering (CEOSE). (2004). Broadening participation in America's science and engineering workforce. The 1994-2003 Decennial and 2004 Biennial Reports to Congress.
Experiences of Successful Projects that Serve Other Populations
DO-IT staff members consult reports regarding projects that serve to increase the participation of underrepresented groups (e.g., women and underrepresented minorities) in college and career programs, such as those in the challenging fields of science, technology, engineering, and mathematics (STEM). For example, DO-IT's AccessSTEM projects have been particularly influenced by the Louis Stokes Alliances for Minority Participation (LSAMP) Program, which was established in 1991 by the NSF to increase the quality and quantity of racial/ethnic minority students who successfully complete baccalaureate degrees and who continue on to graduate studies in STEM. The Urban Institute was commissioned to conduct a comprehensive review of the literature on strategies and approaches that increase the participation and persistence of underrepresented minorities in STEM, describe the LSAMP model, and evaluate project components. The evaluators found, as whole, the LSAMP Program to be successful in reaching its objectives.
The Urban Institute determined that, although the specific activities of individual projects varied greatly, the overall strategies and approaches employed in LSAMP projects constitute an identifiable program model of an integrated set of approaches and strategies that is grounded in research and theory. Leaders in LSAMP projects reported that specific activities were chosen based on "Alliance members' own experiences of what works, effective strategies identified in the research literature, and current or past efforts of partner institutions." The program evaluators concluded that program success may be dependent on the presence of a comprehensive collection of integrated strategies. The strategies and approaches commonly used by LSAMP projects are clustered into four categories:Student-centered, pre-college development:
- Summer programs
- High school outreach
- Career awareness
- Peer study groups
- Skills-building seminars
- Learning centers
- Academic advising
- Summer academic enrichment
- Research experiences
- Conference attendance
- Career awareness
- Graduate school admissions support
- Graduate summer bridge programs
- Training on teaching
- Training regarding diversity sensitivity
- Course reform
- New course development
- Curriculum material sharing
- Distance learning courses
- Changes in institutional/ departmental policies and practices
Researchers found that three student-centered program components stood out as having significant positive relationships with desired outcomes in LSAMP - research with faculty, internships, and summer bridge programs. It was also found that LSAMP students who pursued graduate degrees and who completed graduate degrees tended, on average, to participate in more activities and spend more time involved in the program. The common perception of project staff, professors, institutional administrators and students is that LSAMP participation leads to increased student interest, commitment, and confidence in STEM; provides valuable academic support and professional development; fosters supportive relationships with faculty and peers; and facilitates academic progress and preparation for graduate studies.
Reference: Clewell, B. C., de Cohen, C. C., Tsui, L., Deterding, N. (2006). Revitalizing the nation's talent pool in STEM. The Urban Institute.
In an earlier study of LSAMP, "keys to success" were identified as summer bridge programs, research experiences, mentoring, drop-in centers, caring staff, and alliance structure.
Reference: Sharp, L., Kleiner, B., & Frechtling, J. (2000). A description and analysis of best practice findings of programs promoting participation of underrepresented undergraduate students in science, mathematics, engineering, and technology fields. (Report No. NSF 01-31). Arlington, VA: National Science Foundation.
Although the research and practice foundation for LSAMP relates to interventions for increasing the participation of underrepresented racial/ethnic minorities in STEM, the experiences of the successful LSAMP programs help DO-IT staff identify promising interventions to employ with students who have disabilities. For example, in AccessSTEM they considered all of the LSAMP strategies and approaches, adopted most, adapted some (e.g., mentoring), and found others to be less appropriate (e.g., learning centers because of the small size of the group of students with disabilities).
Like LSAMP, AccessSTEM fosters connections within an institution (e.g., between peers, between students and faculty, and involvement in institutionally sponsored activities and events), but also facilitates connections outside of the institution (e.g., between precollege and postsecondary partners, between community college and university partners, between students and employers, and between colleagues at conferences), is significantly engaged in resource-sharing and co-sponsoring events with other stakeholder units, and uses "graduates" of previous programs as a rich resource for mentors, speakers, employees, and volunteers. As was found with successful LSAMP programs, AccessSTEM has benefited from the accumulated expertise and experiences that staff members bring from other NSF projects previously and currently funded through DO-IT. Important differences between the campus populations served by LSAMP and AccessSTEM include the smaller size of the population of students with disabilities when compared to racial/ethnic minorities on partner campuses making some group activities more difficult to pull together and the additional accessibility issues related to STEM programs and student services that must be addressed in interventions for participants with disabilities.
Published Research and Practice
Activities implemented in DO-IT include those that:
- have evidence of effectiveness as reported in research/evaluation literature or
- are promising interventions reported by educators and practitioners through publications in the literature, conference presentations, phone conferences, meetings, and personal correspondence.
Resources linked to the following topic areas provide guidance to DO-IT staff regarding the selection and implementation of evidence-based practices and promising interventions for student participants.
- College preparation, transition, self-determination
- The efficacy of comprehensive programs over isolated efforts
- Broadening participation of students with disabilities in STEM
- Mentoring, peer support, and learning communities
- Research experiences, internships and other work-based learning
- Serving veteran students with disabilities
Project staff use What Works Clearinghouse (WWC) standards to assess the rigor of research evidence regarding the effectiveness of interventions being considered in order to make informed decisions about project practices. Unfortunately, the most relevant studies reported in the literature do not often have random samples and well-matched control groups or other characteristics that would result in a high evidence rating by the U.S. Department of Education's Institute of Education Sciences WWC. As stated on the WWC website, a low level of evidence rating for a particular practice does not indicate that the recommendation is any less important than other recommendations with a strong or moderate rating. Rather, it suggests that the WWC panel cannot point to a body of research that demonstrates its effects on student achievement. In some cases, this means that the recommended practice would be difficult to study in a rigorous, experimental fashion (e.g., ethical issues regarding not delivering service to a control group, small sample size, limited funds available for evaluation of a practice). In other cases, a low rating means that there is weak or conflicting evidence of effectiveness.
DO-IT staff also consult WWC's Practice Guides as they have been developed. Some references in Practice Guides have been helpful to DO-IT projects, including:
- Helping Students Navigate the Path to College: What High Schools Can Do
- Structuring Out-of-School Time to Improve Academic Achievement
- Encouraging Girls in Math and Science
Project staff also review the Doing What Works (DWW) for research-based education practices and practical tools to improve instruction. Although reported studies in the WWC collection and other literature often focus on populations not identical to potential participants in DO-IT projects, they have provided these projects with evidence that is used to select and refine activities that can be further tested.
DO-IT staff engage in regular phone conferences with projects that serve to broaden the participation of underrepresented groups, particularly in STEM fields, where they share and learn of other evidence-based practices and promising interventions. Even though some of these practices have not been extensively tested, it is determined that they hold promise for achieving successful results based on evaluation results, the opinion of experienced practitioner stakeholders, and feedback from students with disabilities. Some promising practices are published in the DO-IT Knowledge Base.