Effectiveness of multiple teaching methods in standardized training of internal medicine residents in China: a network meta-analysis | BMC Medical Education

Table of Contents

Literature search

Primary search found 19,783 relevant literatures, including PubMed (n = 872), Embase (n = 725), Web of Science (n = 1144), Cochrane Library (n = 1640), CNKI (n = 2508), WanFang Data (n = 6322), VIP (n = 17), and CBM ui(n = 6555). 74 literatures fulfilled the inclusion and exclusion criteria, they are all Chinese documents. These papers are published from 2016 to 2023. A flow diagram of the literature selection process can be seen in Fig. 1.

Study characteristics

A total of 74 studies were included, of which 65 were RCT [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80] and 9 studies did not report whether randomization were used [81,82,83,84,85,86,87,88,89]. A total of 13 different teaching methods were involved: LBL, PBL, PBL + scenario-based teaching(PBL + SBT), PBL + CBL, CBL, PBL + multi-disciplinary team (PBL + MDT), CBL + clinical pathway(CBL + CP), CBL + FC, PBL + evidence-based medicine(PBL + EBM), PBL + CP, PBL + Wechat, CBL + WeChat and TBL. During data processing subgroup mergers were done using the three-arm experiment, and the data were analysed according to the two-arm experiment. See Supplementary Material S1 for the study characteristic included in the study. The outcomes of the comparison network plots between different teaching method are shown in Fig. 2.

Literature quality evaluation

The literature quality of the 65 RCT studies were evaluated. In terms of random allocation methods, 2 articles were randomly grouped by student number and enrollment order, and were rated as high-risk [55, 58]. 19 articles were randomly assigned using the random number table method or computer random allocation sequence, and were rated as low-risk [22, 23, 25, 26, 28, 33, 35, 38, 54, 57, 59, 62, 65, 67, 73,74,75, 79, 80]. 44 articles only mention randomness, but did not specify the random allocation scheme, and were rated as unclear risk [16,17,18,19,20,21, 24, 27, 29,30,31,32, 34, 36, 37, 39,40,41,42,43,44,45,46,47,48,49,50,51,52,53, 55, 56, 58, 60, 61, 63, 64, 66, 68,69,70,71,72, 76,77,78]. Regarding allocation concealment and blinding, 65 studies lacked detailed documentation on concealed and blinded allocation procedures, resulting in their classification as having unclear risk. However, the data from all 65 studies were deemed complete and assessed as low risk. The criteria for evaluating selective reporting of research outcomes involved cross-verifying the consistency between the methodological descriptions in the literature and the reported results. For selective reporting, all 65 studies were low-risk. Other sources of bias are unclear. RevMan 5.3 software was used to illustrate the results as shown in Fig. 3. Newcastle–Ottawa-Scale(NOS) was used to evaluate the cohort study, which includes three aspects: selection, comparability and outcome. The results are as shown in Table 1.

Table 1 The Newcastle–Ottawa quality assessment scale to evaluate risk of bias in three domains for cohort study

Meta-analysis

Theoretical test score

In the evaluation of the theoretical test score, 72 studies were included with a total of 4874 standardized training trainees from DIM. When compared to LBL method, PBL + CBL method (SMD = 1.61, 95% CI [1.12, 2.1]) PBL + CP method (SMD = 2.07, 95% CI [0.81, 3.34]), CBL + Wechat method (SMD = 1.74, 95% CI [0.5, 2.99]), PBL + Wechat method (SMD = 2.3, 95% CI [1.19, 3.42]), TBL method (SMD = 1.57, 95% CI [0.28, 2.85]), PBL method (SMD = 1.35, 95% CI [0.78, 1.92]), PBL + EBM method (SMD = 1.28, 95% CI [0.3, 2.25]) and CBL method (SMD = 0.98, 95% CI [0.21, 1.75]) have significant advantages, and the difference is statistically significant (P < 0.05). Other statistically significant pairwise comparisons can be found in Supplementary Material S2. Compared with other teaching methods, PBL + Wechat had the greatest possibility of improving theoretical test score (88%), followed by PBL + CP (80.3%), CBL + Wechat (67.9%) and PBL + CBL (66.1%). Other combination therapies did not show obvious advantages in theoretical test score, as shown in Table 2 and Fig. 4A.

Table 2 The SUCRA values of each intervention

Practical test score

In the evaluation of the practical test score, 55 studies were included with a total of 3485 standardized training trainees from DIM. When compared to LBL method, TBL method (SMD = 2.32, 95%CI [0.74, 3.9]), CBL + CP method (SMD = 1.97, 95%CI [0.88, 3.06]), PBL + Wechat method (SMD = 2.15, 95%CI [0.02, 4.29]), PBL + CBL method (SMD = 1.74, 95%CI [1.14, 2.34]), PBL + CBL method (SMD = 1.74, 95%CI [1.14, 2.34]), PBL + MDT method (SMD = 1.66, 95%CI [0.16, 3.15]), PBL method (SMD = 1.43, 95%CI [0.88, 1.97]), PBL + EBM method (SMD = 1.39, 95%CI [0.43, 2.35]) and CBL method (SMD = 1.25, 95%CI [0.44, 2.05]) have significant advantages, and the difference is statistically significant (P < 0.05). Other statistically significant pairwise comparisons can be found in Supplementary Material S3. Compared with other teaching methods, TBL had the greatest possibility of improving practical test score (80.1%), followed by CBL + CP (71.3%), PBL + Wechat (71.2%), PBL + CBL (64.4%), CBL + Wechat (63.6%) and PBL + MDT (57.9%). Other combination therapies did not show obvious advantage in practical test score, as shown in Table 2 and Fig. 4B.

Medical record test score

In the evaluation of the practical test score, 27 studies were included with a total of 1746 standardized training trainees for internal medicine residents. The NMA showed that in terms of medical record test score, among the 11 types of teaching methods. When compared to LBL method, we found that PBL + CP method (SMD = 4.84, 95%CI [3.08, 6.59]), PBL method (SMD = 2.45, 95%CI [1.66, 3.23]), CBL method (SMD = 1.91, 95%CI [1.15, 2.67]), CBL + Wechat method (SMD = 1.77, 95%CI [0.58, 2.95]), PBL + CBL method (SMD = 1.74, 95%CI [0.99, 2.49]) and PBL + MDT method (SMD = 1.22, 95%CI [0.05, 2.4]) have significant advantages, and the difference is statistically significant (P < 0.05). Other statistically significant pairwise comparisons can be found in Supplementary Material S4. Compared with other teaching methods, PBL + CP (99.9%) had the greatest possibility of inproving medical record test score. The single teaching method has been effective in improving the performance of medical record analysis, with the results of PBL (83.4%) and CBL (67.2%). Other combination therapies did not show obvious advantages in medical record test score, as shown in Table 2 and Fig. 4C.

Score of autonomous learning ability

In the evaluation of the score of autonomous learning ability, 9 studies were included with a total of 615 standardized training trainees for internal medicine residents. The NMA showed that in terms of score of autonomous learning ability, among the 5 types of teaching methods. When compared to LBL method, we found that PBL method (SMD = 1.98, 95%CI [0.05, 3.91]) and PBL + CBL method (SMD = 1.87, 95%CI [0.49, 3.24]) have significant advantages, and the difference is statistically significant (P < 0.05).The results of pairwise comparison can be found in Supplementary Material S5. Compared with other teaching methods, the PBL teaching method has the highest likelihood of improving students’ self-learning ability (75.8%), followed by PBL + CBL (74.2%), PBL + MDT (45.2%), and CBL + FC (40.5%). Other combination therapies did not show obvious advantages in medical record test score, as shown in Table 2 and Fig. 4D.

Number of dissatisfied people

At the end of the experiment, the number of dissatisfied people with the various teaching methods mentioned in the article were calculated. 42 studies were included with a total of 2921 standardized training trainees from DIM. When compared to LBL method, we found that PBL + Wechat (OR = 0.06, 95% CI [0.01, 0.27]), TBL (OR = 0.11, 95% CI [0.02, 0.52]), CBL + Wechat (OR = 0.13, 95% CI [0.06, 0.3]), CBL + FC (OR = 0.13, 95% CI [0.02085]), PBL + MDT (OR = 0.18, 95% CI [0.06, 0.5]), PBL + CBL (OR = 0.19, 95% CI [0.12, 0.3]), CBL (OR = 0.19, 95% CI [0.08, 0.44], PBL + EBM (OR = 0.2, 95% CI [0.06, 0.65]), PBL (OR = 0.24, 95% CI [0.12, 0.47]) and PBL + CP (OR = 0.25, 95% CI [0.11, 0.59]) have significant advantages. It means that the number of students who are unsatisfied with the new teaching method is much less than that of the traditional LBL teaching method, and the difference is statistically significant (P < 0.05). The results of pairwise comparison can be found in Supplementary Material S6. Compared with other teaching methods, PBL + Wechat is more likely to satisfy students (89%), while students have the most aversion to the LBL teaching method (0.7%), as shown in Table 2 and Fig. 4E.

Small sample evaluation

The correction comparison funnel plot of theoretical scores, practical scores, medical record analysis scores, and number of dissatisfied individuals shows that all research points are roughly symmetrically distributed on both sides of the midline, indicating a low possibility of publication bias in this study. Some patterns of symmetry on the funnel charts for the score of autonomous learning ability are observed. This suggests that there is a certain level of bias or small sample events occurring. The results are shown in Fig. 5.

Cluster analysis

Cluster analysis is used simultaneously to determine the most promising teaching method among different teaching methods in terms of theoretical scores, practical skills scores, medical record analysis scores, autonomous learning scores, and the number of dissatisfied individuals. As shown in Fig. 6A, the results of cluster analysis suggest that compared with other teaching methods, PBL + Wechat, TBL and CBL + Wechat are beneficial in improving students’ theoretical performance and reducing the number of dissatisfied individuals. In contrast, the CBL + CP and LBL method did not see satisfactory advantage in improving students’ theoretical performance and reducing the number of dissatisfied individuals. As shown in Fig. 6B, the PBL + Wechat, TBL, and CBL + Wechat method have good benefits in improving students’ practical performance and reducing the number of dissatisfied individuals, while the effects of PBL + CP, PBL, and LBL methods are not satisfactory. As shown in Fig. 6C, CBL + Wechat, PBL + CBL, and CBL methods may bring better results in improving students’ theoretical performance and reducing the number of dissatisfied individuals. As shown in Fig. 6D, PBL + CBL method is the most effective teaching method for improving students’ self-learning ability and reducing the number of dissatisfied individuals, while the LBL method remains unsatisfactory.