About the College Readiness Mathematics Test

In 2007-2009, the Office of Educational Assessment (OEA) worked with a collaborative group of Washington state teachers and faculty to create a new college readiness mathematics test. The new test would be aligned with the recently developed College Readiness Mathematics Standards (CRMS), and would be available to high school and postsecondary students throughout Washington state. This project was sponsored by the Transition Mathematics Project (TMP).

Background

Over the past several years, there have been an increasing number of conversations about the level of mathematics preparation among high school students both nationally and within Washington state. Connections between K-12 and post-secondary educational sectors are key to the educational process, and Washington's Transition Mathematics Project (TMP) has taken a lead role in shaping state-wide efforts to improve these linkages. In June 2004, TMP initiated a collaborative project to determine standards for college readiness in mathematics to be applied throughout the state, and the final College Readiness Mathematics Standards (CRMS) were released in March 2007. Two studies assessed the alignment of the Math Placement Tests (administered by the Academic Placement Testing Program, or APTP) with the CRMS. Achieve, Inc. aligned a single form of the Intermediate MPT (MPT-I) with the standards, mapping each item to only a single standard. OEA expanded the study to include three versions of both the Intermediate and Advanced tests, mapping items to multiple standards, as appropriate. Based on this work and in response to specific legislation, OEA collaborated with TMP to propose modification of the Intermediate Math Placement Test (MPT-I) used by Washington state public baccalaureate institutions to bring it into alignment with the newly defined CRMS. The revised test would be used at two- and four-year post-secondary institutions throughout the state to provide a standard definition of college readiness in mathematics.

College Readiness Mathematics Standards (CRMS)

Achieve, Inc. alignment study

OEA alignment study (212K PDF)

Second Substitute House Bill 1906 (see Sections 10 and 11)

OEA College Readiness Mathematics Test (CRMT) Development Plan (86K PDF)

Timeline

timeline

Project Events

Meeting
Date and Time: May 11, 2007, 9:00 am - 5:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Develop test specifications for CRMT 12th-grade test (CR-2) and create sample items to "operationalize" CRMT components.
Resources: Agenda (18K PDF), Slides (310K PPT), Minutes (26K PDF)

Meeting
Date and Time: June 22, 2007, 9:00 am - 5:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Continue item development and begin planning for pre-test.
Resources: Agenda (26K PDF), Minutes (23K PDF)

Prepare for Autumn 2007 item pretest
Participants of the May and June working meetings determined that the existing Intermediate Math Placement Test (MPT-I) should be retained in its current form to optimize placement into precalculus courses, and a new General Math Placement Test (MPT-G) should be created for placement into general entry-level college math courses other than precalculus. The new test would contain items assessing each of the five CRMT Content Standards in contrast to the heavier weighting given to algebra and functions by the MPT-I. Both tests would generate a college readiness score. Drawing on sample items created during the working meetings, OEA staff created a pool of potential MPT-G items and formed them into five 30-item tests (one for each Standard). Each test consisted of six benchmark items taken from the Intermediate MPT (MPT-I), Version F and 24 content items. A second version of each test was created by reversing the order of the items, in case students were unable to finish in the time allotted. The resulting ten test versions were then administered during an item pretest during autumn 2007.

Autumn 2007 item pretest
The item pretest was carried out between September 25 and October 8, 2007. Nine institutions participated and we tested a total of 1566 students in 38 entry-level math courses (23 precalculus and 15 general math).
Participating institutions : Eastern Washington University, Pierce College, Spokane CC, Spokane Falls CC, Seattle Central CC, UW-Bothell, UW-Seattle, Whatcom CC, and Yakima CC.

Prepare for Winter 2008 instrument pilot-test
Extensive statistical analyses were carried out on the results of the item pretest, including internal consistency analyses of each test version, and assessment of distractor effectiveness, item reliability, and item difficulty relative to MPT-I benchmark items. These analyses identified items that performed well, and those that should be modified, or eliminated. Analyses also indicated that very few items had 4 (or even 3) functional distractors. This, together with the possibility that students would be able to complete more items if the number of response alternatives was reduced, suggested that the format of MPT items should be changed from 5-options (used until now for all MPT tests) to either 4-options (used by OSPI) or 3-options (suggested by testing literature as preferable). Selecting best-performing items from the item pretest, OEA staff created two parallel forms of the new MPT-G and two types of each form (one with three options for each item and one with four options). Two versions of each type were created by reversing the order of the items. The resulting eight test versions were then administered during an instrument pilot test during winter 2008.

Meeting and conference call
Date: December 10, 2007
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11 th Ave. NE
Purpose: Review analyses of item pretest and discuss plans for instrument pilot test.
Resources: Agenda (23K PDF)

Winter 2008 instrument pilot test
The instrument pilot test was carried out during the first three weeks of January, 2008. Five institutions participated and we tested a total of 1566 students in 38 entry-level math courses ( 23 precalculus and 15 general math).
Participating institutions : Eastern Washington University, Spokane Falls CC, UW-Seattle, Walla Walla CC, and Whatcom CC.

Meeting
Date and Time: April 7, 2008, 9:00 am - 3:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Review results of instrument pilot test and discuss implementation issues.
Resources: Agenda (23K PDF), Minutes (51K PDF)

Finalization of the General Math Placement Test
Based on results of the item pretest and instrument pilot, we developed two parallel versions of the MPT-G. This is described in OEA Report 08-01 (462K pdf). 

Formalize testing procedures to expand testing program
To provide statewide student access, Math Placement Tests are administered at 1) college Testing Centers throughout the year and 2) independent Testing Sites each May and June.  Students at any institution (4-year, 2-year, secondary) may sit for the tests, however, we may need to establish additional testing dates and locations to optimize access for high school and community and technical college students. We have formalized APTP Testing Center and Testing Site manuals, along with other procedural documents, to facilitate this process.

Survey of CTC test administrators
OEA staff polled test administrators at community and technical colleges via phone interviews and an online survey in May and June. Questions focused on testing resources of the institutions and usefulness of the MPT for placement at the CTCs. Although it was initially anticipated that combining placement and college readiness testing in a single instrument would reduce the testing load on students, survey results (OEA Report 08-02) (154K pdf) suggest this would not be the case at two-year schools. The vast majority of students at these institutions require placement into courses below college level whereas college readiness testing necessarily focuses on college level courses.

Prepare for Spring statewide Math Placement testing
Although we had initially planned to introduce the new MPT-G during the APTP annual statewide testing in May and June of 2008 for the use of students entering two-year schools, discussions among the CTCs regarding testing procedures were still ongoing. Additionally, development of the MPT-G in 3-option item format introduced the need reformat existing MPT-I and MPT-A tests.  For these reasons, we determined to use this testing season to introduce what would be the ongoing APTP test renewal process (see diagram below).

Ongoing Academic Placement Testing Program (APTP) Test Renewal Cycle
test-renewal cycle

Since the inception of the APTP, new versions of the Intermediate and Advanced tests have been introduced periodically so that the specific test content would not become widely known. We have formalized this process by testing new items during each spring statewide testing. New items are interspersed in each test and the testing period has been expanded from 60 minutes to 75 minutes.  New items do not contribute to the student test score, but are analyzed for inclusion in new test versions.

For the spring 2008 testing season, OEA staff created five versions of the MPT-I by interspersing 3-option Version G items throughout the primary MPT-I (Version F).  Another five MPT-I versions were created using 4-option items.  Similarly, OEA staff created ten versions of the MPT-A by interspersing 3-option and 4-option Version H items into the MPT-A (Version G). 

Spring 2008 statewide Math Placement testing
The annual APTP spring testing is conducted in May and June at over 50 testing sites around the state. In 2008 we administered approximately 1500 Intermediate and 1000 Advanced tests.  Implementation of the expanded testing time went smoothly with no apparent disruptions.

Meeting
Date and Time: Tuesday, September 16, 2008, 9:00 am - 5:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Review results of statewide testing and test administrator survey.  Discuss methodology of setting college-readiness cutoff scores and expansion of APTP Governing Committee.
Resources: Agenda (19K pdf), Minutes (27K pdf)

Winter and Spring 2009 instrument pilot
In fall, 2008, the APTP program distributed the new General Math Placement Test (MPT-G) and revised (three-option) versions of the Intermediate and Advanced tests (MPT-I and MPT-A, respectively) to testing centers at public four-year institutions. The new tests replaced existing Intermediate and Advanced tests for use in ongoing placement into math courses. Institutions provided subsequent end-of-course grades as part of a nine-month pilot test to inform standards setting. The MPT-G and MPT-I also were administered at high schools and two-year institutions during winter and spring, 2009, and schools provided end-of-course grades for math courses in which students were currently enrolled. Results of the pilot are reported in OEA Report 09-03 (214K pdf).

Meeting
Date and Time: Wednesday, August 19, 2009, 1:30-3:30 pm
Location: Conducted online using Elluminate conferencing tool
Purpose: Review results of pilot test and set initial college readiness cut score.
Resources: Agenda (23K pdf), Slides (189K ppt), Minutes (30K pdf)

Implementation
Participants of the August 19 Elluminate session defined the college readiness cutscore as a raw score of 17 on the MPT-G based on the results of the 2008-09 pilot test and current placement cutscores. SBCTC and COP representatives subsequently obtained approval of the recommended college readiness cutscore from their respective systems groups. The MPT-G is now available to all high school and college students through the Academic Placement Testing Program (APTP).

Public two-year institutions. Beginning in the 2009-2010 academic year, all community and technical colleges have agreed to allow students who score at the MPT-G college readiness cutscore or higher to place into Math 107; additional courses are left to individual college decisions. Students who meet this cutscore as juniors will need to take senior math with a grade of C or better to qualify for this placement. (See Instruction Commission Agreement, 12K pdf.)

Public four-year institutions. Starting with the autumn 2010 admissions cycle, eleventh-grade students who score at or above the college readiness cutscore will not be required to take remedial math at four year institutions if they meet the HECB minimum admissions requirements and enroll in a public baccalaureate institution the summer or fall following their senior year. (See Provosts Agreement, 17K pdf.)