Article Text
PDF
Abstract
Introduction Leadership and crisis resource management (CRM) skills are important skills for doctors, however there is a recognised lack of undergraduate leadership education. There remains debate over how best to teach leadership and CRM skills, and poor leadership skills among clinicians are associated with adverse patient outcomes. We examined whether high-fidelity battlefield and prehospital scenarios can improve leadership and CRM skills.
Method This was a prospective observational study with students self-reporting their leadership and CRM skills using the Ottawa Crisis Resource Management Global Ranking Scale (OCRMGRS) before and after completing the Cambridge University Emergency Medicine Society Battlefield and Pre-Hospital Trauma course. The course involves a mixture of small group tutorials and practical high-fidelity battlefield and prehospital trauma scenarios. Faculty also completed the OCRMGRS for the first and last candidates at the scenarios. The mean precourse versus mean postcourse score of the OCRMGRS was analysed using a two-tailed t-test.
Results 46 students completed paired OCRMGRS before and after the course. The mean precourse scores for each of the domains (leadership, communication skills, resource utilisation, problem solving skills and situational awareness) were calculated. There was a statistically significant (p<0.05) increase in both self-reported and faculty-reported scores across all domains, and the increase remained at 1-year follow-up.
Conclusions Leadership and CRM skills are important non-clinical skills for doctors, however there is debate over how best to teach them. High-fidelity battlefield and prehospital trauma scenarios are an effective means of teaching leadership and CRM skills to civilian medical students.
- accident and emergency medicine
- trauma management
- orthopaedic and trauma surgery
- medical education and training
Statistics from Altmetric.com
- accident and emergency medicine
- trauma management
- orthopaedic and trauma surgery
- medical education and training
Key messages
Leadership and crisis resource management (CRM) skills are mandated objectives for doctors in training, however, are seldom specifically taught.
Lack of leadership skills among doctors is associated with worse clinical outcomes for patients.
Effective teaching methodologies for leadership and CRM skills are an area currently being researched, with no consensus opinion.
High-fidelity battlefield and prehospital trauma simulations for civilian medical students improve leadership and CRM skills.
Introduction
The importance of leadership skills for doctors is recognised by the General Medical Council. Their publication, ‘Leadership and Management for all doctors,’1 provides a framework outlining the leadership responsibilities of a doctor in their workplace. However, recent examples and reviews2 3 have shown a lack of leadership skills among UK doctors and the Francis report cited a lack of clinical leadership by doctors leading directly to patient harm.4
Positive leadership behaviours among healthcare professionals have been associated with increased patient satisfaction, reduced adverse events,5 lower patient mortality, lower medication errors and fewer hospital-acquired infections.6
Despite the recognised importance of leadership skills in clinical practice, there remains a paucity of undergraduate teaching focused towards developing leadership skills.7 An intercollegiate report in 20108 identified the need for more teaching at undergraduate level on leadership and management. The commentary on this report from the Director of Medical Leadership at the NHS Institute for Innovation and Improvement noted, ‘Introducing medical students to relevant management and leadership competencies will enhance their contribution to the improvement of health services throughout their medical careers.’9
The UK postgraduate programme for newly qualified doctors states that by the end of the second postgraduate year, doctors must be able to ‘demonstrate clinical effectiveness, leadership and decision-making responsibilities.’10 This is achieved against specific learning outcomes such as ‘Demonstrates extended leadership role within the team by making decisions and taking responsibility for managing increasingly complex situations across a greater range of clinical and non-clinical situations.’11
The Faculty of Medical Leadership and Management recently announced the formation of a national working group to improve the way in which leadership and management training is delivered to medical students,12 but as yet no curriculum changes have been implemented.
Crisis resource management (CRM) is an umbrella term referring to the non-technical skills required for effective participation in a critical situation. There remains no consensus on how best to teach leadership or CRM skills, with some UK medical schools integrating medium fidelity simulation throughout curriculums, however many UK undergraduates qualify with only one or two simulation sessions achieved.
The Ottawa Crisis Resource Management Global Ranking Scale (OCRMGRS) is a self-assessment questionnaire designed and validated13 14 for healthcare professionals to self-assess their CRM and leadership skills. The OCRMGRS asks candidates to score themselves across six domains of assessment: leadership, communication skills, situational awareness, problem solving, resource utilisation and overall performance.
The course
Cambridge University Emergency Medicine Society (CUEMS) and 254 Medical Regiment, British Army, jointly run a yearly Battlefield and Pre-Hospital Trauma course for 50 civilian undergraduate medical students who are in the clinical phase of their courses. The course is nationally advertised and at this course seven UK medical schools were represented. The course includes five high-fidelity simulations which are designed to test candidates’ clinical and non-clinical skills. The learning outcomes for this course include developing candidates’ leadership and CRM skills.
Over the 1-day course, candidates visit five ‘skill’ stations, which focus on knowledge and procedural skill acquisition (such as airway management or breathing assessment) and five ‘scenario’ stations. Candidates attempt the scenarios in a group of 5, within one candidate acting as team leader at each scenario. The scenarios were designed to test and develop both clinical skills and non-clinical skills (such as leadership and CRM). They were a mix of high-fidelity civilian and military scenarios however required no previous military knowledge nor experience. This course format has been designed to maximise learning using educational theories such as Blooms Cognitive Taxonomy of Knowledge12 by getting students to apply skills in an unfamiliar environment and context. The scenario themes are summarised in table 1.
Scenario stations Clinical and CRM / Leadership focus
Methods
This study had a prospective observational design. The objective of the study was to determine if Battlefield and Pre-Hospital Trauma scenarios are an effective educational means to teach leadership and CRM skills. Precourse and postcourse questionnaires assessing candidate’s leadership and CRM skills were completed on the day of the course by candidates wishing to participate in this study (n=46) who were attending the CUEMS Battlefield and Pre-Hospital Trauma course. Candidates were provided with a brief of the study at the beginning of the course and invited to participate. Forty-six of 50 participants consented to talking part.
The questionnaire used was the OCRMGRS which has previously been used and validated for self-assessment for simulations.10 11 The OCRMGRS assesses six domains of CRM (leadership, communication skills, situational awareness, problem solving, resource utilisation and overall performance) on a numerical scale of 0–7. On course completion, students repeated the same questionnaire to identify and quantify improvement in the above skills.
To further validate the subjective self-assessment aspect of this research methodology, the course faculty completed the OCRMGRS for the first and last groups at their scenario, to demonstrate any improvement over the course.
Students were then contacted via email 12 months after the course and asked to complete the survey again online, in order to demonstrate any long-term benefit of this educational method.
Data analysis
Mean scores for each area within the questionnaire were compiled for both the precourse and postcourse student questionnaires and the precourse and postcourse faculty questionnaires. A two-tailed t-test was used to compare all the parametric data between the two groups. Data were analysed using Microsoft Excel and a p value <0.05 was considered statistically significant.
Results
Forty-six of 50 students on the course consented to participation on the day (92%). The mean scores for each of the areas assessed using the OCRMGRS were calculated, and then the difference between the mean precourse and postcourse scores for each area was assessed using a two-tailed t-test. The data were assumed to be parametric. Mean results and statistical analysis are shown in table 2.
Students’ precourse and postcourse mean scores
The course faculty teaching scenarios also completed the OCRMGRS survey for the first group and the last group on the course to validate the self-assessment methods and add in an objective assessment of change in leadership or CRM skills over the duration of the course. This gave an extra 10 data sets which were analysed in the same manner. The faculty scores came from 10 faculty members working independently from each other, and data analysis was performed independently of the course faculty. Mean scores and statistical analysis are shown in table 3.
Faculty precourse and postcourse mean scores
All students were contacted via their registration email address 12 months after the course and invited to do the OCRMGRS again online in order to demonstrate any lasting educational benefit of the course. Eighteen students (39%) replied and their results are summarised in table 4.
Students’ precourse self-reported scores and 1-year follow-up
Given the much smaller sample size of the 1-year follow-up data, the authors were unable to perform the same statistical analysis on this data set as on the immediate postcourse data set.
Discussion
These results show a convincing, and statistically significant improvement in leadership and CRM skills in students who attended this course. This improvement is reported both subjectively by the students, but also objectively by the course faculty, and a lasting improvement can be seen at a 1-year follow-up.
The magnitude of score increases was also interesting. Across all domains of CRM measured, the faculty scored a higher mean increase than the students. Both sets of data had statistically significant increases in scores across all measured outcomes. We were encouraged by this and saw it as further validation of the OCRMGRS for self-reporting.
The course represents a very cost-effective means of teaching leadership and CRM skills to students. Course running costs were covered between sponsorship and attendance revenue (£20 per student), which is significantly lower than course costs for other similar courses such as ATLS/ETC/ATACC where course costs are over £600 GBP.
This course was advertised to students, and over the years the course has proved very popular. Given that the students who attend are there under their own volition, and often very keen on acute care specialties we are cautious to extrapolate these findings to all UK undergraduate students. The current course set-up infers a selection bias and we suspect if this course were made mandatory to the whole medical student population the results may not be so convincing. Due to the research methodology, we were unable to blind the students or faculty. We aimed to minimise the effects by doing the data analysis independently.
With our 1-year follow-up, we used an online survey platform which provided anonymous data sets. A disappointingly high proportion of students were lost to follow-up and our response rate was only 39%. Having a sample size of only 18 responses made the statistical analysis more difficult. While we are confident that there is an increase in leadership and CRM skills observed at the 1-year interval, we cannot evidence that this is a statistically significant increase. Furthermore, the observational cohort study design merely shows an association between improving leadership and CRM skills over time, we cannot imply causation from this study design.
We would hope that students’ leadership and CRM skills would naturally improve over time with increasing clinical exposure and progression through medical school; we do not have a control group of students who did not attend the course to compare our results with in order to account for this.
The student-reported data are purely subjective and vulnerable to bias. We have attempted to minimise this by using a questionnaire validated for self-assessment and comparing results with objective data reported by course faculty. We see the narrow distribution of faculty scores (small SD) as confirmation of interobserver reliability, however acknowledge that the faculty scores are unpaired and not truly reflective of an individual’s performance before and after the course.
The OCRMGRS has previously been validated for in-hospital use by doctors in simulated environments. As far as we know, it has never been validated for use by either medical students or prehospital trauma scenarios. A secondary outcome of this work is that we have now validated the OCRMGRS model both for undergraduate use and for use within prehospital emergency trauma care.
The course has several other benefits beyond teaching students clinical, leadership and CRM skills. It has proved a potent recruiting tool, with up to 40% of students asking for more information on a career in the Army Reserves. We have also donated over £5000 in course proceeds to our local air ambulance charity, MAGPAS Helimedix, over the 5 years that the course has run. Furthermore, the faculty partly comprised clinicians and medics from the Regiment. Hosting this course has allowed our clinicians to maintain military clinical currency and develop both their clinical and teaching skills.
Conclusion
Battlefield and Pre-Hospital Trauma high-fidelity simulations are associated with both a subjective and objective quantitative increase in leadership and CRM skills in civilian undergraduate medical students.
This represents a novel and effective means of teaching a mandatory skill set for which there is currently a lack of evidence-based effective teaching strategies.
Supplemental material
Acknowledgments
Preliminary results presented at Military Pre-Hospital Emergency Care Special Interest Group Conference, UK 2018.
Footnotes
Contributors ME and SF contributed equally to study design, data collection, analysis, paper writing and revision.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information.