About the College Readiness Mathematics Test
The Office of Educational Assessment (OEA) is working with a collaborative group of Washington state teachers and faculty to align the Intermediate Math Placement Test (MPT) with the recently developed College Readiness Mathematics Standards (CRMS), and to create a college readiness mathematics test to be taken by Washington state 11th-grade students. This project is sponsored by the Transition Mathematics Project (TMP).
Background
Over the past several years, there have been an increasing number of conversations about the level of mathematics preparation among high school students both nationally and within Washington state. Connections between K-12 and post-secondary educational sectors are key to the educational process, and Washington's Transition Mathematics Project (TMP) has taken a lead role in shaping state-wide efforts to improve these linkages. In June 2004, TMP initiated a collaborative project to determine standards for college readiness in mathematics to be applied throughout the state, and the final College Readiness Mathematics Standards (CRMS) were released in March 2007. Two studies assessed the alignment of the Math Placement Tests (MPT) with the CRMS. Achieve, Inc. aligned a single form of the Intermediate MPT (MPT-I) with the standards, mapping each item to only a single standard. OEA expanded the study to include three forms of both the Intermediate and Advanced tests, mapping items to multiple standards, as appropriate. Based on this work and in response to specific legislation, OEA collaborated with TMP to propose that we modify the Intermediate Math Placement Test (MPT-I) currently used by Washington state public baccalaureate institutions to bring it into alignment with the newly defined CRMS. The revised test will be used at two- and four-year post-secondary institutions throughout the state to provide a standard definition of college readiness in mathematics. The project will also create a version of the exam that will be administered to 11th-grade students throughout the state to provide advance notice of additional preparation required for college entrance.
College Readiness Mathematics Standards (CRMS)
OEA alignment study (212K PDF)
Second Substitute House Bill 1906 (see Sections 10 and 11)
OEA College Readiness Mathematics Test (CRMT) Development Plan (86K PDF)
Timeline
What has happened so far?
Meeting
Date and Time: May 11, 2007, 9:00 am - 5:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Develop test specifications for CRMT 12th-grade test (CR-2) and create
sample items to "operationalize" CRMT components.
Resources: Agenda (18K PDF), Slides (310K PPT), Minutes (26K PDF)
Meeting
Date and Time: June 22, 2007, 9:00 am - 5:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Continue item development and begin planning for pre-test.
Resources: Agenda (26K PDF), Minutes (23K PDF)
Prepare for Autumn 2007 item pretest
Participants of the May and June working meetings determined that the existing Intermediate Math Placement Test (MPT-I) should be retained in its current form to optimize placement into precalculus courses, and a new General Math Placement Test (MPT-G) should be created for placement into general entry-level college math courses other than precalculus. The new test would contain items assessing each of the five CRMT Content Standards in contrast to the heavier weighting given to algebra and functions by the MPT-I. Both tests would generate a college readiness score. Drawing on sample items created during the working meetings, OEA staff created a pool of potential MPT-G items and formed them into five 30-item tests (one for each Standard). Each test consisted of 24 content items and six benchmark items taken from the Intermediate MPT (MPT-I), Version F. A second version of each test was created by reversing the order of the items, in case students were unable to finish in the time allotted. The resulting ten test versions were then administered during an item pretest during autumn 2007.
Autumn 2007 item pretest
The item pretest was carried out between September 25 and October 8, 2007. Nine institutions participated and we tested a total of 1566 students in 38 entry-level math courses (23 precalculus and 15 general math).
Participating institutions : Eastern Washington University, Pierce College, Spokane CC, Spokane Falls CC, Seattle Central CC, UW-Bothell, UW-Seattle, Whatcom CC, and Yakima CC.
Prepare for Winter 2008 instrument pilot-test
Extensive statistical analyses were carried out on the results of the item pretest, including internal consistency analyses of each test version, and assessment of distractor effectiveness, item reliability, and item difficulty relative to MPT-I benchmark items. These analyses identified items that performed well, and those that should be modified, or eliminated. Analyses also indicated that very few items had 4 (or even 3) functional distractors. This, together with the possibility that students would be able to complete more items if the number of response alternatives was reduced, suggested that the format of MPT items should be changed from 5-options (used until now for all MPT tests) to either 4-options (used by OSPI) or 3-options (suggested by testing literature as preferable). Selecting best-performing items from the item pretest, OEA staff created two parallel forms of the new MPT-G and two types of each form (one with three options for each item and one with four options). Two versions of each type were created by reversing the order of the items. The resulting eight test versions were then administered during an instrument pilot test during winter 2008.
Meeting and conference call
Date: December 10, 2007
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11 th Ave. NE
Purpose: Review analyses of item pretest and discuss plans for instrument pilot test.
Resources: Agenda (23K PDF)
Winter 2008 instrument pilot test
The instrument pilot test was carried out during the first three weeks of January, 2008. Five institutions participated and we tested a total of 1566 students in 38 entry-level math courses ( 23 precalculus and 15 general math).
Participating institutions : Eastern Washington University, Spokane Falls CC, UW-Seattle, Walla Walla CC, and Whatcom CC.
Meeting
Date and Time: April 7, 2008, 9:00 am - 3:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Review results of instrument pilot test and discuss implementation issues.
Resources: Agenda (23K PDF), Minutes (51K PDF)
Finalization of the General Math Placement Test
Based on results of the item pretest and instrument pilot, we developed two parallel versions of the MPT-G. This is described in OEA Report 08-01.
Formalize testing procedures to expand testing program
To provide statewide student access, Math Placement Tests are administered at 1) college Testing Centers throughout the year and 2) independent Testing Sites each May and June. Students at any institution (4-year, 2-year, secondary) may sit for the tests, however, we may need to establish additional testing dates and locations to optimize access for high school and community and technical college students. We have formalized APTP Testing Center and Testing Site manuals, along with other procedural documents, to facilitate this process.
Survey of CTC test administrators
OEA staff polled test administrators at community and technical colleges via phone interviews and an online survey in May and June. Questions focused on testing resources of the institutions and usefulness of the MPT for placement at the CTCs. Although it was initially anticipated that combining placement and college readiness testing in a single instrument would reduce the testing load on students, survey results (OEA Report 08-02) suggest this would not be the case at two-year schools. The vast majority of students at these institutions require placement into courses below college level whereas college readiness testing necessarily focuses on college level courses.
Prepare for Spring statewide Math Placement testing
Although we had initially planned to introduce the new MPT-G during the APTP annual statewide testing in May and June of 2008 for the use of students entering two-year schools, discussions among the CTCs regarding testing procedures were still ongoing. Additionally, development of the MPT-G in 3-option item format introduced the need reformat existing MPT-I and MPT-A tests. For these reasons, we determined to use this testing season to introduce what will be our ongoing test renewal process (see diagram below).
Since the inception of the APTP, new versions of the Intermediate and Advanced tests have been introduced periodically so that the specific test content would not become widely known. We will formalize this process by testing new items during each spring statewide testing. New items will be interspersed in each test and the testing period will be expanded from 60 minutes to 75 minutes. New items will not contribute to the student test score, but will be analyzed for inclusion in new test versions.
For the spring 2008 testing season, OEA staff created five versions of the MPT-I by interspersing 3-option Version G items throughout the current primary MPT-I (Version F). Another five MPT-I versions were created using 4-option items. Similarly, OEA staff created ten versions of the MPT-A by interspersing 3-option and 4-option Version H items into the MPT-A (Version G).
Spring statewide Math Placement testing
The annual APTP spring testing is conducted in May and June at over 50 testing sites around the state. This year we administered approximately 1500 Intermediate and 1000 Advanced tests. Implementation of the expanded testing time went smoothly with no apparent disruptions.
Meeting
Date and Time: Tuesday, September 16, 2008, 9:00 am - 5:00 pm
Location: University of Washington, Roosevelt Commons B, Room 416, 4311 11th Ave. NE
Purpose: Review results of statewide testing and test administrator survey. Discuss methodology of setting college-readiness cutoff scores and expansion of APTP Governing Committee.
Resources: Agenda (19K pdf), Minutes (27K pdf)
What is happening now? (10/6/08)
The new General Math Placement Test (MPT-G) has been distributed to testing centers at public four-year institutions, along with revised (three-option) versions of the Intermediate and Advanced tests (MPT-I and MPT-A, respectively). We are actively seeking administrative sites at two-year schools and the high schools to obtain data for setting the college readiness cutoff score.
What is coming up next?
Over the next few months we will be gathering information to set a common college readiness cutoff score for the MPT-G. This process will include administering the test under genuine testing conditions at a variety of secondary and post-secondary institutions; collecting student grades in subsequent math courses at those institutions; carrying out statistical analyses of item characteristics and the relationship between test scores and course grades; and incorporating results of the analyses in a standards setting procedure (to include representatives from K-12, two-year, and four-year institutions). Results of this process will be summarized and a specific college-readiness cutoff score proposed by the end of September 2009.