Exploring the perception of medical students and lecturers on the consequential validity of medical long case | BMC Medical Education

The traditional long case has become unpopular in medical schools in the Global North in favor of Objective Structured Clinical Exam (OSCE) and Work Base Assessment such as Mini Clinical Evaluation Exercise (Mini-Cex) which are believed to yield more reliable and easily defensible results [1,2,3]. Yet, in our previous investigation on the experiences of medical students and lecturers with the long case, we reported that the assessment method is still utilized in both formative and summative assessment of medical students in the Global South (low-resource settings) [4]. This form of assessment exposes students to patients, and provides an opportunity for students to be observed and assessed in the performance of the real tasks of a physician [4, 5]. The long case provides a comprehensive assessment of clinical skills unlike OSCEs, which encourage a reductionist approach by splitting clinical tasks into constituent parts and testing them separately [2, 6]. This is in agreement with the notion that the whole is more than the sum of its parts [7], as artificially breaking down a one-hour long case clinical encounter into several bites may not be sufficient to assess the complex clinical skills and competence of a medical trainee. Therefore, the long case assesses an integrated approach to clinical medicine which enables students to see patients wholly rather than as sum of their diseased organ systems. The long case is thus an educationally valuable assessment with high face validity [8]. It is authentic, located in a clinical platform, and provides a holistic assessment of clinical skills, doctor patient relationship, professionalism, communication and teamwork [9,10,11,12]. In keeping with constructivism, the long case is a meaningful assessment which reflects the practice of the profession or discipline being assessed [13] providing an assessment of the ability to apply knowledge in the context in which one would work after graduation.

However, there are two main arguments against the long case that is its inter-rater and inter-case reliability. The low inter-rater reliability is due to the lack of standardization of the long case [4]. The examiners often use flexible, subjective, and global judgment, with a lack of clarity in marking scale. Thus a student examined via the long case is often scored differently by different examiners due to variation in leniency or stringency of the examiner [14,15,16]. Additionally, inter-case reliability is a major concern due to variation in diagnostic complexities, as some cases are generally more complex than others. Thus, a single long case cannot assess the full breadth of skills or sample the curriculum widely [17, 18], and it is difficult to generalize from one long case encounter about student ability in other cases [19]. However, reliability is a major concern mainly when a student performance is judged based on a single long case as is often done summative assessment. In formative assessment however, a student makes several long cases over time and encounters varying degrees of case complexities on different topics and is often assessed by different lecturers [4] which improves reliability.

Despite the concerns over reliability, the long case continues to be a cornerstone in clinical assessment in the Global South probably because of its educational impact. The construct of educational impact, also referred to as consequential validity, has two dimensions [20]. Firstly, the assessment needs to have accurate descriptions in order to guide interpretation. Secondly, and more important to this study, is that the consequential validity appraise the potential and actual consequences of using an assessment method including both intended and unintended outcome extending to even social consequences of an assessment [20,21,22]. However, there is a sizeable research gap on the long case with no published literature on its consequential validity in the Global South. Therefore, the aim of this study was to use a qualitative research method to explore consequential validity of the long case as perceived by medical students and lecturers to offer guidance to educators on how to optimize, effectively utilize and exploit the potentials of the long case in clinical assessment while providing quality medical education and patient care.

Methodology

Study design

An interpretivist approach was taken in this exploratory qualitative study. The data was collected through focus group discussions and interviews, and analyzed by deductive thematic analysis.

Study setting

The study was conducted at Makerere University School of Medicine in Uganda, where the Bachelor of Medicine and Bachelor of Surgery (MBChB) degree is a five-year program. The first two years are for pre-clinical training and the last three years are dedicated to clinical clerkship. The medical students complete a junior and senior clerkship in Pediatric, Internal Medicine, General Surgery and Obstetrics and Gynecology in the third- and fifth- years respectively. The fourth year is dedicated to clinical experience in various medical and surgical specialties. The long case is used in formative assessment during clerkship in all four major disciplines. In this formative assessment, medical students present their long case during ward rounds on a daily basis. The students assume the role of a doctor who watches over patients assigned under his/her care and presents the patient for discussion with the lecturers, residents and their peers. The student does a minimum of 10 long cases over the clerkship period of 7-weeks for which the student is formatively assessed. In internal Medicine Clerkship however, the long case is also used in summative assessment during which a student presents only one case at the end of the semester for which they are questioned and assessed. For each long case, the medical students interact with patients to take a medical history, perform physical examinations, formulate a diagnosis and treatment plan. The student then presents their long cases to a lecturer or a resident in the presence of the patient and peers. This is then followed by questioning and discussion of the case.

Study participants

The study participants (n = 42) were third- and fifth-year medical students and internal medicine lecturers. At the time of data collection, the total numbers of third and fifth year medical students were 191 and 188 respectively and the department of internal medicine had 42 physicians/lecturers. The study participants were recruited by purposive sampling and the sample size was determined by data saturation point. The third-year students (n = 19) recruited were those who had completed long case assessments in junior clerkship while the fifth-year students (n = 18) were recruited on basis of having completed the long case in both junior and senior clerkships. Of the third-year students recruited, there were ten male and nine female while the fifth-year participant comprised of eight male and ten female. The lecturers in the internal medicine department (n = 5) were recruited to participate because they assess the long case during both formative and summative assessment. Out of the five lecturers who participated, three were female and two male. The students were invited through their class representative and a brief recruitment presentation was made at the study site, while the lecturers were approached via email and telephone invitation.

Data collection

Ethical approval to conduct the study was obtained from Makerere University School of medicine research and ethics committee with ethics ID Mak-SOMREC-2022-524. Informed consent was obtained from all the study participants.

Data were collected through Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs). The FGD and KII guides were developed and piloted prior to the study (supplementary file 1 and 2). It was a semi structured guide which provided a set of questions while allowing freedom to ask in any order, following tangents or seeking clarification of previous answers or elaboration of responses.

The study was conducted in May & June 2023 during recess term when student have just completed academic year. A total of 36 medical students participated in FGDs, reflecting on their experiences with the long case. Six face to face FGDs were conducted, three for junior clerks and three for senior clerks. Each FGD comprised of 5–7 participants with balanced male and female gender representation. A research assistant proficient in qualitative research methods, without prior knowledge about the long case and having no relationship to the students facilitated the FGDs. The discussions lasted between 55 min and 1 h 10 min and were audio recorded, and transcribed verbatim. Data saturation was achieved by the fifth FGD, at which point no additional new information was shared.

Semi-structured KIIs were used to collect data from Internal Medicine faculty. Five KIIs were conducted, and data saturation was achieved by the fourth interview, at which point no new information was shared. The Principal Investigator, a graduate from the same medical school more than 4 years before the study, having no current ties with internal medicine faculty conducted the KIIs via Zoom. Each interview lasted between 25 and 50 min and was recorded and transcribed verbatim. The data collected were securely stored on a hard drive and Google Drive with password protection to prevent unauthorized access.

Data analysis

Data analysis was done through deductive thematic analysis method. The use of predetermined categories to analyze data makes deductive analysis more structured, systematic and efficient [23]. It begins with a research question that clearly connects to the theory or frame work to be used in analysis. The researchers then operationalize the theory or framework by either generating sensitizing constructs, or key concepts from them that are revised throughout analysis [23, 24]. It is noteworthy that deductive analysis may limit the discovery of unexpected patterns and promote confirmation bias, as the analysis is confined to predefined themes [24]. In our data analysis, Cook and Lineberry framework for organizing data on the impact of education assessment [25] was used. The framework proposes that the education impact of an assessment should be explored by considering the impact on various stakeholders such as students, educators, schools, or the end target of practice (e.g., patients or health care systems); and the downstream impact of classification like the different score cut points. The impact can result from the assessment activities or use of the score; they can be intended or unintended, beneficial or harmful [25]. The framework enabled systematic analysis of the consequence of the long case assessment in medical education. During data analysis, we defined the research question then applied the predetermined coding framework, then coding segments of text according to the predefined themes. The data was coded in multiple iterations, resulting in rich, nuanced understanding of the phenomenon under investigation. Finally, the coded data were reviewed and organized to ensure they align with the thematic framework, allowing researchers to refine or consolidate key themes as necessary. Throughout the analysis, the researchers stayed true to the framework while remaining open to unexpected findings to limit the loss of subtle nuances or other possible understanding.