Competence Ratings in Psychotherapy Training – A Complex Matter

Background The reliable assessment of competence is crucial for promoting the professional development of therapists. However, competence assessments are rarely included in training and research as these procedures are resource-intensive and costly, commonly relying on independent raters with high l...

Full description

Saved in:
Bibliographic Details
Published in:Cognitive therapy and research 2024-06, Vol.48 (3), p.500-510
Main Authors: Paunov, Tatjana, Weck, Florian, Heinze, Peter E., Maaß, Ulrike, Kühne, Franziska
Format: Article
Language:English
Subjects:
Cameras
Clinical Psychology
Cognitive Psychology
Emotions
Empathy
Medicine
Medicine & Public Health
Original Article
Psychotherapy
Quality of Life Research
Training
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Background The reliable assessment of competence is crucial for promoting the professional development of therapists. However, competence assessments are rarely included in training and research as these procedures are resource-intensive and costly, commonly relying on independent raters with high levels of expertise and extensive training. This study aimed to compare the interrater reliability (IRR) of raters with different levels of expertise. We also examined the impact of different camera perspectives on IRR. Methods We examined the IRR of six independent raters based on competence assessments in a standardized setting. Two raters were experienced psychotherapists (experts), and four were psychology students (novices; with/without supervision). All raters evaluated N  = 359 videos of students performing role plays with standardized patients who were simulating depressive symptoms and behavior. For each video, the raters independently assessed basic communication skills (Clinical Communication Skills Scale–Short Form; CCSS-S), psychotherapeutic competence (Cognitive Therapy Scale; CTS), empathy (Empathy Scale; ES) and therapeutic alliance (Helping Alliance Questionnaire; HAQ). Results IRR varied depending on rater expertise and assessment measures, with the lowest intraclass correlation coefficients (ICCs) for empathy (ES; ICCs = 0.39-0.67) and the highest ICCs for psychotherapeutic competence (CTS; ICCs = 0.66-0.78). The concordance between expert raters and supervised novice raters was good (ICCs = 0.71-0.86). The camera perspective did not influence the reliability of the ratings. Conclusions With appropriate training and regular supervision, the novices assessed therapeutic behavior in standardized role plays with reliability comparable to that of the experts. Further research is needed regarding the reliable assessment of more complex therapy situations.
ISSN:0147-5916
1573-2819
DOI:10.1007/s10608-023-10445-x