RUME 26
2024
Omaha, Nebraska
Creating Statistically Equivalent Versions of a Test of Quantitative Literacy in Physics Contexts
Page: 1447
The Physics Inventory of Quantitative Literacy (PIQL) is a 20-item multiple-choice test designed to measure the development of students’ physics quantitative literacy (PQL) across three facets: ratios and proportions, covariation, and signs and negativity. Repeated testing across multiple courses, coupled with requiring up to 40 minutes for students to complete the test, could lead to testing fatigue and unreliable results. We seek to create two shorter versions of the PIQL (a.k.a. piqlets) that are statistically equivalent to each other in terms of student performance on all three facets of PQL. We considered 240 combinations of items for the piqlet versions, subject to constraints that both the content and the format of items were equivalent across the two versions. Data were collected by administering the PIQL in three introductory physics courses at a large public university (N ~ 2500). Piqlet versions were compared by calculating differences in average scores, reliability coefficients, and psychometric parameters from item response theory (IRT). The combination of items that produces the most similar piqlet versions in our data have score differences less than 1.3%; Crobach's alpha values of approximately 0.8, with differences less than 0.01 between versions; and IRT parameters that differ by no more than 0.1 for items included on both versions.