Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool

Background Errors in judgment during thyroidectomy can lead to recurrent laryngeal nerve injury and other complications. Despite the strong link between patient outcomes and intraoperative decision-making, methods to evaluate these complex skills are lacking. The purpose of this study was to develop...

Full description

Saved in:
Bibliographic Details
Published in:World journal of surgery 2018-02, Vol.42 (2), p.376-383
Main Authors: Madani, Amin, Gornitsky, Jordan, Watanabe, Yusuke, Benay, Cassandre, Altieri, Maria S., Pucher, Philip H., Tabah, Roger, Mitmaker, Elliot J.
Format: Article
Language:English
Subjects:
Abdominal Surgery
Adult
Annotations
Cardiac Surgery
Clinical Competence
Cognition & reasoning
Cognitive ability
Complications
Consistency
Correlation analysis
Correlation coefficient
Correlation coefficients
Decision Making
Decision Making, Computer-Assisted
Decision Support Techniques
Distance learning
Female
General Surgery
Humans
Intermediates
Internet
Machine learning
Male
Mathematical analysis
Medical Errors - prevention & control
Medical personnel
Medicine
Medicine & Public Health
Middle Aged
Original Scientific Report
Reliability
Reproducibility of Results
Skills
Statistics, Nonparametric
Surgeons
Surgery
Task analysis
Thoracic Surgery
Thyroidectomy
Thyroidectomy - education
Thyroidectomy - methods
Training level
Vascular Surgery
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background Errors in judgment during thyroidectomy can lead to recurrent laryngeal nerve injury and other complications. Despite the strong link between patient outcomes and intraoperative decision-making, methods to evaluate these complex skills are lacking. The purpose of this study was to develop objective metrics to evaluate advanced cognitive skills during thyroidectomy and to obtain validity evidence for them. Methods An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from four institutions completed a 33-item assessment, developed based on a cognitive task analysis and expert Delphi consensus. Sixteen items required subjects to make annotations on still frames of thyroidectomy videos, and accuracy scores were calculated based on an algorithm derived from experts’ responses (“visual concordance test,” VCT). Seven items were short answer (SA), requiring users to type their answers, and scores were automatically calculated based on their similarity to a pre-populated repertoire of correct responses. Test–retest reliability, internal consistency, and correlation of scores with self-reported experience and training level (novice, intermediate, expert) were calculated. Results Twenty-eight subjects (10 endocrine surgeons and otolaryngologists, 18 trainees) participated. There was high test–retest reliability (intraclass correlation coefficient = 0.96; n  = 10) and internal consistency (Cronbach’s α  = 0.93). The assessment demonstrated significant differences between novices, intermediates, and experts in total score ( p  
ISSN:0364-2313
1432-2323
DOI:10.1007/s00268-017-4322-y