Unpacking Response Process Issues Encountered When Developing a Mathematics Teachers' Pedagogical Content Knowledge (PCK) Assessment

It is essential for items in assessments of mathematics' teacher knowledge to evoke the desired response processes - to be interpreted and responded to by teachers as intended by item developers. In this study, we sought to unpack evidence that middle school mathematics teachers were not consis...

Full description

Saved in:
Bibliographic Details
Published in:Investigations in mathematics learning 2023, Vol.ahead-of-print (ahead-of-print), p.1-17
Main Authors: Epstein, Martha L., Malik, Hamza, Wang, Kun, Orrill, Chandra H.
Format: Article
Language:English
Subjects:
Assessment item development
Classroom Environment
construct-irrelevant responses
constructed response
Mathematics Instruction
Mathematics Teachers
Mathematics Tests
Middle School Teachers
Pedagogical Content Knowledge
Protocol Analysis
response process validity evidence
Teacher Attitudes
Test Items
Test Validity
Vignettes
Work Environment
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:It is essential for items in assessments of mathematics' teacher knowledge to evoke the desired response processes - to be interpreted and responded to by teachers as intended by item developers. In this study, we sought to unpack evidence that middle school mathematics teachers were not consistently interacting as intended with constructed response (i.e. open-ended) items designed to assess their pedagogical content knowledge (PCK). We analyzed recent data derived from think-aloud interviews with 13 teachers involving 38 assessment items designed to tap PCK regarding proportional reasoning. Five key issues associated with undesired response processes were identified: (1) scenarios provided insufficient information, (2) content knowledge (CK) and PCK elements were confounded, (3) questions asked about the scenarios lacked specificity, (4) items contained distracting text and/or visual elements, and (5) differences between math education research and classroom teacher work cultures led to unanticipated interpretations of items. These issues were associated with teacher responses that were problematic (e.g. vague, off topic, etc.). In addition, we suggest that obtaining response process evidence is critical, and the way it is obtained may impact the average difficulty of the final pool of assessment items developed.
ISSN:1947-7503
2472-7466
DOI:10.1080/19477503.2023.2201115