A multi-source feedback tool for measuring a subset of Pediatrics Milestones

Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decis...

Full description

Saved in:
Bibliographic Details
Published in:Medical teacher 2016-10, Vol.38 (10), p.995-1002
Main Authors: Schwartz, Alan, Margolis, Melissa J., Multerer, Sara, Haftel, Hilary M., Schumacher, Daniel J.
Format: Article
Language:English
Subjects:
Classification
Clinical Competence - standards
College professors
Competency-Based Education
Educational Measurement - methods
Educational Measurement - standards
Factor Analysis, Statistical
Factor Structure
Faculty
Feedback
Formative Feedback
Hospitalization
Humans
Inpatient care
Internship and Residency
Internships
Measuring instruments
Multisource
Nurses
Pediatrics
Pediatrics - education
Pediatrics - standards
Performance evaluation
Professionalism
Psychometrics
Resident physicians
Rotation
Societies, Medical
Validity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.
ISSN:0142-159X
1466-187X
DOI:10.3109/0142159X.2016.1147646