NCEO - OOLT Report 14-States' Procedures for Ensuring Out-of-Level Test Instrument Quality

2010

The present study was initiated to gather information about states' out-of-level testing practices. Specifically, we wanted to see whether states attempted to align out-of-level tests with grade of enrollment content standards, what processes are used to complete this task, and what psycho- metric information states offer as evidence of that alignment. We collected narrative data in this study from two sources: technical information about states' large-scale statewide assessment and information gleaned from interviews with state assessment directors or other individuals knowledgeable about the state's large-scale assessment. We used the technical information gathered prior to the interviews to provide context and support for our interview data analysis process. We then compiled the results thematically. Five critical issues were highlighted in the results. 1. There was an increasing need for states to provide easily accessible technical information that includes out-of-level testing information. 2. States' arguments supporting their decision to not equate out-of-level test scores with on-level test scores were stronger than arguments supporting this practice. 3. States provided incomplete and inconclusive information about the psychometric properties of out-of-level test scores. 4. States were not consistent in their opinions about the use of out-of-level tests. 5. States made questionable assumptions about out-of-level testing. Overall, the wide variability of out-of-level testing practices among the states raises many concerns about the practice of out-of-level testing. Available online at: http://www.cehd.umn.edu/nceo/onlinepubs/oolt14report.pdf

Reports and monographs

Electronic (disc, CD, 508 compliant web posting)

N/A