Evaluating podcasts as a tool for OSCE training: a randomized trial using generative AI-powered simulation | BMC Medical Education

This study is the first randomized controlled trial to evaluate the impact of a podcast specifically designed for OSCE preparation on students’ OSCE performance. The findings demonstrated that students who incorporated the podcast into their study regimen showed significantly greater improvement in OSCE scores compared to those relying on traditional learning methods alone, with a 17% higher improvement in scores (p = 0.002). Additionally, these students expressed significantly higher confidence in their nephrology clinical skills, suggesting that podcasts may serve as a valuable resource in medical education [9]. Moreover, this study is the first randomized controlled trial to use a generative AI-powered virtual OSCE platform and assess OSCE performance [10]. The platform, DocSimulator, demonstrated comparable baseline OSCE scores between the two groups, reporting its reliability in evaluating clinical skills within a randomized cohort.

Podcasts represent a flexible and engaging medium for delivering medical education content, allowing students to reinforce their learning outside of traditional lecture-based settings [11, 12]. As an accessible, time-efficient resource, podcasts enable learners to study at their own pace, which may enhance knowledge retention [13]. In this study, only 4.1% of students identified traditional academic classes as their preferred resource for medical learning, suggesting a shift in student preferences toward more flexible, digital learning tools like podcasts [11, 14]. This finding supports the notion that students increasingly value on-demand educational resources over rigid, lecture-based formats, particularly as digital technologies become more integrated into medical education.

Moreover, audio-based learning caters to various learning styles, including auditory learners who benefit from listening and repetition over visual or text-based materials [6]. The high proportion of students in the podcast group who found OSCE preparation podcast useful and expressed willingness to continue using it in the future further supports the development of such digital tools by other specialties. Nevertheless, it is important to emphasize the role of blended learning strategies. While digital tools such as podcasts are valuable, they should complement rather than replace traditional curricula, ensuring that students receive a comprehensive and well-rounded education [15].

The observed improvement in student confidence with nephrology-related clinical skills is particularly promising. Confidence in clinical competencies is crucial for preparing medical students to transition into clinical practice. This aligns with existing literature suggesting that targeted educational interventions can significantly enhance students’ self-efficacy in clinical settings [16]. However, the lack of change in students’ intention to pursue nephrology as a specialty or undertake a nephrology internship is a notable finding. Despite increased competency and confidence, career choices in medicine are often influenced by factors beyond education alone, such as personal interests, perceived work-life balance, and mentorship [17].

Despite these encouraging results, several limitations must be acknowledged. First, this study was conducted at a single center, and the sample size was small, which may restrict the generalizability of the findings to broader student populations or other medical specialties. Second, the study employed a text-based virtual OSCE rather than a real-time vocal OSCE. While this offers a novel and standardized approach ensuring greater consistency across participants, it may not fully replicate the nuances of live, vocal clinical examinations. This may affect how well the results translate to actual clinical performance. Third, this study relies on a generative AI-based assessment of OSCE performance. To establish external validity and fully evaluate the effectiveness of this automated approach, further studies are needed, including comparisons with human assessments [18]. Fourth, the scope of the nephrology content covered in the podcast was not exhaustive, which limits the conclusions we can draw about its utility for comprehensive nephrology education or its applicability to broader medical disciplines. Fifth, there is a risk of selection bias, as students included in the intervention group may have been more motivated to perform well. Furthermore, the reliance on self-reported confidence levels introduces the possibility of response bias. Finally, NephrOdio OSCE training podcast represents a specific format, and the study did not explore how different formats (e.g., case-based, discussion-led) might affect learning outcomes.

Overall, the study highlights the potential of podcasts like NephrOdio to serve as an effective supplementary resource in medical education, particularly in enhancing clinical competencies. However, more research is needed to determine the long-term impact of podcasts on medical learning, including their role in knowledge retention, clinical application, and career choices. Future research should also evaluate how different podcast formats (e.g., case-based, discussion-led) compared to other digital learning tools, such as video-based or interactive simulation resources, to better understand how these innovations can be integrated into broader medical education strategies.