Given the consequences associated with the PARCC accountability test for students, their parents and their teachers, the New Mexico Public Education Department should consider this simple yet informative option. Provide a complete item analysis for each student’s test. This analysis should include performance data on each item showing correct answers, incorrect answers or incomplete answers. The analysis should also include details supporting the scoring of extended response items, and if possible, the student’s response to the prompt or task.
Ideally, the state would release all the items that appeared on the tests so parents, teachers and students could gain a more thorough understanding of why a given performance level is assigned to each student. Our neighboring state Texas does this. Unfortunately, releasing items here is not likely because it would reduce the testing company’s profit and increase its effort. A reasonable alternative is for the state, through the testing contractor, to provide proxy items that are similar to those on the tests. These proxies should include, in addition to item stems and answer options, any related content, such as reading passages, illustrations and other elements.
An item analysis of this type is important for two main reasons. The first is that it can help parents and teachers understand the test content that students have mastered and what remains to be learned. This content, it should be added, does not reflect all the knowledge and skills that students have had an opportunity to learn, just the material that can be readily measured by a test.
The second reason for this type of item analysis is that it is the best way for students, parents and teachers to ensure the accuracy of a student’s score and understand how well it conforms to the performance level descriptions for each grade. This level of oversight is certainly warranted, given the frequency with which high-stakes tests and technology-based record-keeping have been compromised. One needs to look no further than the recent issue with electronic grading at Belen High School. A list of testing problems encountered in other states can be found here: www.fairtest.org/computerized-testing-problems-2013-2015 .
There are very high stakes associated with the state-mandated accountability tests. It is essential that their fairness and accuracy should be demonstrated. In a sense, a comprehensive item analysis is the ideal way for the testing company and those who favor the single-incident approach to achievement to demonstrate their credibility and show that they are accountable for the process they support.
Michael Milone, Ph.D. is a research psychologist who develops assessments and instructional materials for major publishers. He has taught in both general and special education.