How to assess communication, professionalism, collaboration and the other intrinsic CanMEDS roles in orthopedic residents: use of an objective structured clinical examination (OSCE)

Background Assessing residents’ understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) des...

Full description

Saved in:
Bibliographic Details
Published in:Canadian journal of surgery 2014-08, Vol.57 (4), p.230-236
Main Authors: Dwyer, Tim, MBBS, Takahashi, Susan Glover, MA, PhD, Hynes, Melissa Kennedy, MA, Herold, Jodi, PhD, Wasserstein, David, MD, Nousiainen, Markku, MD, MSc, MEd, Ferguson, Peter, MD, Wadey, Veronica, MD, MA, Murnaghan, M. Lucas, MD, MEd, Leroux, Tim, MD, Semple, John, MD, MSc, Hodges, Brian, MD, Med, PhD, Ogilvie-Harris, Darrell, MD, MSc
Format: Article
Language:English
Subjects:
Canada
Clinical Competence
Communication
Cooperative Behavior
Education, Medical, Graduate - standards
Educational Measurement - methods
Humans
Internship and Residency - standards
Methods
Orthopedics - education
Orthopedics - standards
Periodic health examinations
Physical diagnosis
Physician's Role
Professional workers
Reproducibility of Results
Surgery
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background Assessing residents’ understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid. Methods The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs). Results Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores. Conclusion An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.
ISSN:0008-428X
1488-2310
DOI:10.1503/cjs.018813