The United Kingdom National Health Service (NHS) is the 5th largest employer in the world with one of the most diverse workforces in the public sector. Nearly 40% of its employees are of BAME (Black, Asian, and Minority Ethnic) background, have migrated from outside the UK, or have disabilities or other protected characteristics. This rich diversity of professional and cultural experience of its workforce would ideally serve as a national resource for the NHS, but instead, the recognised phenomenon of differential outcomes or attainment has become a hurdle for many, based on gender, ethnicity, disability, or other protected characteristics. The experience is often far worse for employees who have qualified overseas.
Differential Attainment or Outcomes or award (DA) is a euphemistic phrase that describes the historical and persistent differences in award levels based on factors other than academic/ professional capability or effort. The impact of this endemic discrimination of affected individuals/ groups has a profound impact on careers and wellbeing on a personal (micro) level on them, productivity, team-working or patient safety impact at organisational (meso) level and at a much larger societal/ socio-economic level (macro) level for the country.
This roundtable sponsored jointly by the British Association of Physicians of Indian Origin (BAPIO) and the Royal College of Physicians (RCP) is the first of a series of events organised by the BAPIO Institute for Health Research (BIHR) exploring DA across the journey of a medical professional from entry to medical school all the way to retirement.
This roundtable is focussed on DA as observed in summative assessments in high stakes professional examinations, which are essential for entry, progression, or accreditation. The findings and recommendations of this series will be published in the rainbow paper “Bridging the Gap” celebrating diversity in the NHS. This report should be read in conjunction with the scoping paper which presents the evidence base for the discussion and recommendations. (Dave et al., 2020)
Key Findings
- In the UK doctors from BAME groups, and International Medical Graduates (IMG) i.e. doctors whose primary medical qualification (PMQ) is from a medical school outside of the UK have, consistently, poorer demonstrated outcomes in assessments when compared to white doctors and UK medical school graduates.
- The quantum of this difference varies across examinations and cohorts but usually equates to about 10-15% gap between UK BAME and UK White doctors and about 30-50% gap between IMGs and UK graduates. Clearly, these are group differences with many individual outliers that defy the norm.
- The impact of DA is manifold and pernicious.
- It is a moral imperative that assessment systems and practices are equitable. Implicit bias and discrimination lead to a non-compassionate organisational perception amongst its staff fundamental principles of ‘Our People Plan’ (NHS England » We Are the NHS: People Plan for 2020/2021 – Action for Us All, n.d.) and the ‘NHS Constitution’.(The NHS Constitution for England, n.d.)
- Assessments are important in maintaining clinical standards and patient safety. However, inequitable assessment systems lead to loss of psychological safety and reduced freedom to speak up with consequences for patient safety.
- The financial cost of inequitable assessments for the NHS is significant. It is estimated that in 2019, it costs £88,000 for an additional year of training for general practitioners. Moreover, it limits the number of new trainees that can be trained, creating further pressures on workforce capacity.
- There is a significant wellbeing cost of inequitable assessments in terms of psychological and physical weathering, moral injury, and demoralisation. Demoralisation in the workforce is associated with production costs and poorer patient outcomes. Individual stories of personal and professional tragedy abound, in many cases associated with significant morbidity and sadly in some with untimely mortality.
- DA in summative examinations offer a lens to the differential experiences and outcomes that occur throughout the educational journey and as such systemic issues should not be lost sight of, whilst focussing on high-stakes examinations. Workplace-based assessments or supervisor assessments may not be free from bias or from grade-inflation and therefore may not offer a silver bullet to replace summative examinations. Improving educational supervision, educational, and clinical supervisors’ confidence and skill in addressing the differential learning needs of trainees including IMGs and BAME trainees is important.
- However, the disproportionate impact of COVID19 on BAME patients and the BAME workforce and the Black Lives Matter movement have shone a spotlight on the real impact of structural inequalities. There are a necessity and an opportunity to be bold and courageous in our response, in contrast to the traditional ways of dealing with complex issues that are often ponderous even if incremental and this applies specifically to the inequalities in outcomes associated with summative examinations.
- Summative examinations offer a single point of assessment to determine competence and progression. Making a judgment about competence to practice is a complex process. The reliance on this single point to determine progression ignores the richness of assessment data available from the educational journey. Integrating and formally linking information available from multiple assessment points conducted by a range of assessors over a period of time, allows for multi-dimensional assessment and can potentially neutralise the bias that may reside in a single point assessment.
- Discussions about DA are not new. DA has been highlighted for over two decades but previous efforts at addressing DA have failed to bear fruit despite high profile legal challenges (BAPIO) and high-level meetings with Colleges. This coalition comes at a time when many summative examinations and assessments have had to be reformed or reshaped completely in the wake of COVID19. This offers a unique opportunity to transform the assessment landscape for postgraduate summative assessments in the UK.
- Developing an equitable assessment process that does not discriminate between candidates is not only fair but is also vital in retaining public confidence in the assessment system. Patients and the public need to have the confidence that doctors who are excellent clinical practitioners are able to progress and not failed by the system of assessment.