Studying The Effectiveness of Workplace-Based Assessments

Recent trends in medical education are moving rapidly away from gaining a certain number of marks in high-stakes examinations and towards gathering evidence of clinical competence and professional behavior observed in clinical environments.

The aim of this PhD study, The Quality, Value and Effectiveness of Workplace-Based Assessment in Postgraduate Medical Training, is to explore the use of workplace-based assessments in postgraduate medical training.

Workplace-based assessments, such as Mini-Clinical Evaluation Exercises (mini-CEX) or Direct Observation of Procedural Skills (DOPS), are one method of assessing trainee doctors’ performance. Trainees are expected to negotiate the completion of these assessments with their trainers at regular intervals throughout their training.

Workplace-based assessments are designed to be formative and provide a framework for giving constructive feedback to trainees in real-time clinical settings.

The aim is to help trainees identify their strengths and weaknesses so that they can address problems immediately.

Anecdotal evidence has suggested however, that workplace-based assessments are being used in a summative - as opposed to formative - way, with most assessments being completed towards the end of the training post, leaving little time to make improvements based on the feedback.

Why It Matters

Workplace-based assessment (WBA) has been the subject of much debate since its introduction; it remains unclear whether the intended learning value of these tools has been realised. In order to better understand the learning value of WBA and thereby provide an improved learning experience for trainees, we explored three specific research questions:

How has WBA been implemented in RCPI training programmes?
What are the perceptions of trainers and trainees of the learning value of WBA and how have experiences shaped these perceptions?
How can WBA be used to detect and/or remediate underperformance among postgraduate medical trainees?

Our study will contribute to an ongoing international debate regarding the applicability of workplace-based assessments as a meaningful formative assessment approach within the context of postgraduate medical education.

Study Findings

This project main findings are:

workplace-based assessments continue to be viewed by trainees and trainers as 'tick box exercises' in their current format
trainees who had used WBA felt there was a learning value attached to WBA, mainly by protecting time with their trainer for feedback and in structuring that feedback to be conducive to a learning plan
While the current implementation methods appear to limit the value of the mini-CEX, DOPS or case-based discussion in detecting or managing generalised underperformance, these WBA tools may assist in detecting specific areas of underperformance
Generalised underperformance may potentially be better detected using multi-rater WBA tools e.g. multisource feedback

Read our Publications

Profiling workplace-based assessment implementation in Ireland; a retrospective cohort study. Barrett A, Galvin R, Steinert Y, Scherpbier A, O'Shaughnessy A, Walsh G, Horgan M (2016); SpringerPlus 20165:13

Is the learning value of workplace-based assessment being realised? A qualitative study of trainer and trainee perceptions and experiences. Barrett A, Galvin R, Scherpbier AJJA, Teunissen PW, O’Shaughnessy A, Horgan M (2016) Postgraduate Medical Journal 0:1–5

Protocol: A BEME (Best Evidence Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees. Barrett A, Galvin R, Steinert Y, Scherpbier A, O’Shaughnessy A, Horgan M, Horsley T (2015); Journal of Systematic Reviews 4:65

Dr Lucia Prihodova

Manager, Research Department

Tel: +353 1 863 9645

Dr Lucia Prihodova is the Research Manager at the at the Royal College of Physicians of Ireland. Lucia’s research focuses on psychosocial, medical and health-care system determinants of long term health outcomes. Lucia has a PhD from the Faculty of Medicine at University of Groningen, Netherlands and is an author and co-author of 15 journal articles in the area of chronic diseases.