Unveiling the ultimate advantage: a meta-analysis of 3D visualization and problem-based learning in orthopedic education | BMC Medical Education

Search result
The initial search yielded 1,684 records, from which 628 were removed due to duplication. Following a review of titles, abstracts, and full-text articles, 18 studies were identified as potentially suitable for inclusion criteria [4, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27]. Post the application of these criteria, one English-language trial and 17 Chinese-language trials were incorporated into the meta-analysis were found to have been published. Figure 1 illustrates the selection process and the number of included and excluded studies. However, this study includes both Chinese and English literature, which may introduce some bias. Therefore, for the Chinese studies, we ensure accurate translation to maintain the integrity of the data and reduce bias.

The flowchart of the study
Study characteristic
This meta-analysis included 18 RCTs with a total of 1,077 patients, evaluating the effects of 3D + PBL teaching versus LBL teaching in orthopedic education. Participants’ ages ranged from 14 to 53 years, and the sample sizes ranged from 28 to 106. Most studies focused on trainees (n = 14), with 4 studies targeting resident doctors. Nine studies lacked participant age information, five did not specify teaching subjects, and ten were missing data on study duration. The basic characteristics of the included studies are detailed in Table 1.
The bias risk assessment results of the included studies
The risk of bias in the RCTs was assessed using the Cochrane tool. The results for each quality item were presented as percentages across studies. One study did not report RCT design details, 10 studies had ambiguous random sequence generation, and 7 studies explicitly stated RCT design. One study did not report details of allocation concealment, 15 studies provided unclear descriptions of allocation concealment, and 2 studies explicitly detailed the specifics of allocation concealment. Two studies did not report details of the blinding method, nine studies provided unclear descriptions of the blinding, and seven studies explicitly detailed the specifics of the blinding method. The quality assessment of the included studies is illustrated in Fig. 2 The quality assessment at the outcome level, conducted using the GRADE methodology, is summarized in Table 2. The overall evidence quality, evaluated according to GRADE criteria, was determined to be very moderate to very low.

Results of quality assessment using the Cochrane risk tool
Primary meta-analysis results
Theoretical scores and operational scores
Twele [4, 12,13,14,15,16, 18, 21, 23,24,25, 27] studies (N = 751) reported the theoretical scores, and outcome level quality for theoretical scores assessed by GRADE was “very low”. Significant heterogeneity was observed (P < 0.00001, I2 = 88%), prompting the use of a random-effects model. The analysis revealed that 3D + PBL teaching resulted in higher theoretical scores compared to LBL teaching (SMD = 1.62, 95% CI: 1.14–2.10, P < 0.00001; see Fig. 3). A sensitivity analysis was conducted to explore potential sources of heterogeneity, but no specific source was identified. Through the included basic information tables, we identified differences in participant characteristics (participants in the studies by Sun, Yang, and Zhang were Resident doctors, while participants in the remaining nine studies were Trainees). We also observed variations in teaching subjects across each study, which could be a primary source of heterogeneity. Additionally, we noted that the duration of 3D + PBL teaching varied across studies, which is another significant source of heterogeneity.

A forest plot showing the theoretical score
Nine [12, 14,15,16, 21, 23,24,25, 27] studies (N = 565) reported the operational scores, and outcome level quality for operational scores assessed by GRADE was “very low”. Significant heterogeneity was detected (P < 0.00001, I2 = 94%), necessitating the use of a random-effects model. The analysis indicated that 3D + PBL teaching yielded higher operational scores compared to LBL teaching (SMD = 2.29, 95% CI: 1.41–3.16, P < 0.00001; see Fig. 4). A sensitivity analysis was performed to identify potential sources of heterogeneity but found no specific source. Through the included basic information tables, we identified differences in participant characteristics (participants in the studies by Yang and Zhang were Resident doctors, while participants in the remaining 7 studies were Trainees). We also observed variations in teaching subjects and duration of 3D + PBL teaching across each study, which could be a primary source of heterogeneity.

A forest plot showing the operational score
Understanding ability and communication ability
Seven [14, 15, 17, 18, 20, 23, 27] studies (N = 354) reported on understanding ability, and outcome level quality for understanding ability assessed by GRADE was “very low”. High heterogeneity was observed (P = 0.0007, I2 = 74%), requiring a random-effects model. The meta-analysis indicated that 3D + PBL teaching improved students’ understanding of orthopedic anatomical structures compared to LBL teaching (SMD = 1.18, 95% CI: 0.71–1.65, P < 0.00001; see Fig. 5). A sensitivity analysis indicated that the heterogeneity was potentially influenced by the study by Wu AM et al., which reduced I2 from 74 to 50%. Through a detailed analysis of the study by Wu AM et al., it was found that this research lacks the average age of the participants. Additionally, the teaching subjects in this study focus on 3D spinal models, which differs from the other included studies. This discrepancy could be a major source of heterogeneity.

A forest plot showing the understanding ability
Six [4, 14, 15, 22, 23, 27] studies (N = 340) reported on communication ability, and outcome level quality for communication ability assessed by GRADE was “very low”. High heterogeneity was observed (P = 0.02, I2 = 64%), necessitating the use of a random-effects model. The analysis demonstrated that 3D + PBL teaching improved communication ability compared to LBL teaching (SMD = 1.10, 95% CI: 0.70–1.51, P < 0.00001; see Fig. 6). A sensitivity analysis indicated that the heterogeneity was potentially influenced by the study by Zhang BW et al., which reduced I2 from 64 to 0%. Through a meticulous analysis of the study by Zhang BW et al., it was found that this research not only lacks the average age of the participants but also does not provide information on Teaching subjects and the duration of 3D + PBL teaching. These omissions could be primary sources of heterogeneity.

A forest plot showing the communication ability
Teaching interest and teaching satisfaction
Seven [4, 14, 15, 17, 18, 23, 27] studies (N = 370) reported teaching interest, and outcome level quality for teaching interest assessed by GRADE was “very low”. High heterogeneity was observed (P = 0.006, I2 = 67%), so a random-effects model was applied. The meta-analysis showed that students exhibited greater teaching interest with 3D + PBL teaching (SMD = 1.48, 95% CI: 1.05–1.90, P < 0.00001; see Fig. 7). sensitivity analysis indicated that the heterogeneity was potentially influenced by the study by Shi LJ and Zhang XD et al., which reduced I2 from 67 to 31%. Through a careful analysis of the basic information tables of the included studies, it was found that the participants in these two studies were all Resident doctors, while the participants in the remaining studies were Trainees. This could be a major source of heterogeneity.

A forest plot showing the teaching interest
Four [4, 17, 18, 20] studies (N = 274) reported on teaching satisfaction., and outcome level quality for teaching satisfaction assessed by GRADE was “very low”. No significant heterogeneity was found (P = 0.12, I2 = 49%). The meta-analysis indicated that 3D + PBL teaching led to higher teaching satisfaction (SMD = 0.85, 95% CI: 0.49–1.22, P < 0.00001; see Fig. 8).

A forest plot showing the teaching satisfaction
Orthopedic skill
Six [4, 14, 15, 22, 23, 27] studies (N = 340) reported the orthopedic skill of students, and outcome level quality for orthopedic skill assessed by GRADE was “very low”. High heterogeneity was observed (P = 0.02, I2 = 62%), so a random-effects model was applied. The analysis showed that 3D + PBL teaching resulted in better orthopedic skills compared to LBL teaching (SMD = 1.21, 95% CI: 0.81–1.61, P < 0.00001; see Fig. 9). A sensitivity analysis suggested that the heterogeneity might be attributed to the study by Zhang BW et al., which reduced I2 from 62 to 0%. Through a meticulous analysis of the study by Zhang BW et al., it was found that this research not only lacks the average age of the participants but also does not provide information on Teaching subjects and the duration of 3D + PBL teaching. These omissions could be primary sources of heterogeneity.

A forest plot showing the orthopedic skill
Excellent theoretical grades
Four [19, 22, 25, 26] studies (N = 229) reported excellent theoretical grade for students, and outcome level quality for excellent theoretical grades assessed by GRADE was “moderate”. No heterogeneity was observed (P = 0.77, I2 = 0%), so a fixed-effect model was used. The meta-analysis showed that students achieved higher excellent theoretical grades with 3D + PBL teaching (OR = 2.95, 95% CI: 1.57–5.55, P = 0.0008; see Fig. 10).

A forest plot showing the excellent theoretical grade
Overall satisfaction
Four [19, 21, 26, 27] studies (N = 230) reported the overall satisfaction, and outcome level quality for overall satisfaction assessed by GRADE was “moderate”. No significant heterogeneity was observed (P = 0.32, I [2] = 14%), so a fixed-effect model was applied. The meta-analysis indicated that students had higher overall satisfaction with 3D + PBL teaching compared to LBL teaching (OR = 3.32, 95% CI: 1.84–5.99, P < 0.0001; see Fig. 11).

A forest plot showing the overall satisfaction
Secondary meta-analysis results
Self-learning ability and analyzing and solving problem ability
Four [14, 15, 23, 27] studies (N = 186) reported self-learning ability, and outcome level quality for operation time assessed by GRADE was “very low”. No significant heterogeneity was observed (P = 0.71, I2 = 0%), so a fixed-effect model was used. The meta-analysis found that students demonstrated stronger self-learning abilities with 3D + PBL teaching compared to LBL teaching (SMD = 1.28, 95% CI: 0.96–1.60, P < 0.00001; see Figure S1).
Four [14, 15, 23, 27] studies (N = 186) reported students’ ability to analyze and solve problems, and outcome level quality for operation time assessed by GRADE was “very low”. No significant heterogeneity was observed (P = 0.48, I2 = 0%), so a fixed-effect model was applied. The meta-analysis showed that 3D + PBL teaching significantly enhanced students’ problem-solving abilities compared to LBL teaching (SMD = 1.26, 95% CI: 0.94–1.58, P < 0.00001; see Figure S2).
Learning motivation
Two [15, 27] studies (N = 98) reported on learning motivation, and outcome level quality for operation time assessed by GRADE was “very low”. No significant heterogeneity was observed (P = 0.80, I2 = 0%). The meta-analysis demonstrated that 3D + PBL teaching significantly improved learning motivation compared to LBL teaching (SMD = 1.10, 95% CI: 0.67–1.53, P < 0.00001; see Figure S3).
Passing rate of exam
Two [16, 17] studies (N = 128) reported the passing rate of the examinations, and outcome level quality for operation time assessed by GRADE was “very low”. No significant heterogeneity was observed (P = 0.51, I2 = 0%). The analysis found that 3D + PBL teaching significantly improved the passing rate of exams compared to LBL teaching (OR = 8.13, 95% CI: 2.44–27.10, P = 0.0006; see Figure S4).
Publication bias
A funnel plot was employed to evaluate publication bias. In the studies that reported theoretical scores, the funnel plot displayed asymmetry (see Fig. 12), moreover, Egger’s test (see Figure S5) yielded a P-value of 0.009, indicating the presence of publication bias, indicating a possible occurrence of publication bias.

A funnel plot showing publication bias for theoretical score
link