Evaluating Item Response Format and Content Using Partial Credit Trees in Scale Development

Abstract The type of response options selected for items on a survey, along with how many response options to include and whether to allow neutral midpoints, impacts data obtained from survey collections and the interpretations made using the results. Further, if subgroups within a population (e.g.,...

Full description

Saved in:
Bibliographic Details
Published in:Journal of survey statistics and methodology 2024-07
Main Authors: Asamoah, Nana Amma Berko, Turner, Ronna C, Lo, Wen-Juo, Crawford, Brandon L, McClelland, Sara, Jozkowski, Kristen N
Format: Article
Language:English
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Abstract The type of response options selected for items on a survey, along with how many response options to include and whether to allow neutral midpoints, impacts data obtained from survey collections and the interpretations made using the results. Further, if subgroups within a population (e.g., racial/ethnic, gender, age) interpret response options differently, this variance can artificially inflate non-significant differences or mask true differences between groups. In this study, we apply two recursive partitioning procedures for investigating differential item functioning (DIF) in an experiment evaluating seven item response formats (five levels of an agree–disagree [AD] format and two levels of an item-specific [IS] format). Partial credit tree procedures allow for the evaluation of multiple covariates without prespecifying subgroups to be compared. We applied the procedures to items measuring adults’ attitudes toward legal abortion and all response formats functioned without DIF for age, gender, race, education, and religion when evaluated using global DIF screening approaches. Item-focused analyses indicated that odd-numbered response formats were less susceptible to content-based DIF. The combination of psychometric properties indicated that five-point AD and IS formats may be preferable for abortion attitude measurement based on the screening procedures conducted in this study.
ISSN:2325-0984
2325-0992
DOI:10.1093/jssam/smae028