Mentor communication skills training: development, feasibility, and preliminary efficacy | BMC Medical Education

Participants

Participants were academic medicine faculty at a comprehensive cancer center. Selection of participants was based on convenience sampling, with attention to interest in the training; representation of different departments, career trajectories, and career stages; and having some mentoring experience. The participant’s career path was as either focused on clinical care or research based on how most (> 50%) of their time was allocated. Additional faculty data (demographics, number of mentees, work location) were provided by self-report. The training and evaluation reported in this paper received exemption from Memorial Sloan Kettering Institutional Review Board (Protocol Number X21-049), as per 45 CFR 46.104(d)(1). The study did not require informed consent from participants. Oncology faculty members participated voluntarily in the study following the invitation sent by the last author (LL). The participants indicated interest and availability via email, and this agreement to participate was documented.

Comskil mentor training curriculum

The Comskil Mentor Training curriculum was developed by engaging in the following sequential steps: (a) literature review, including a review of Entering Mentoring, (b) consensus review meetings (with faculty leaders/mentors, researchers, and communication skill experts), (c) modular blueprint development, (d) training methods development, (e) scenario development (for role plays and exemplary video clips), (f) making necessary revisions and adaptations, and (g) assessment of the training module. The training curriculum followed the guidelines of the Comskil Model, a skills-based approach to teaching communication skills in a cancer setting [14]. As per that model, we recommended eight communication strategies (a priori plans that direct communication behavior toward the desired communication goal) for mentors to use in guiding mentees: (a) introduction; (b) initial establishment of relationship – develop ground rules of communication, create a safe space, establish rapport; (c) aligning expectations (e.g., logistics, communication, meetings, responsibilities, etc.); (d) assessing mentor and mentee’s skills and needs; (e) setting goals that are SMART: specific, measurable, attainable, relevant, and time-based [15]; (f) providing and eliciting feedback [16]; (g) expressing empathy; and (h) closing the conversation. The strategies are accomplished by using communication skills (standalone verbal utterances) and process tasks (set of verbal and nonverbal behaviors that create a conducive environment for effective communication; Table 1). Although presented consecutively, the strategies can be used, according to the mentor-mentee interaction context. For instance, if a mentee is working on writing their first manuscript for publication, the mentor could focus on Strategy (e) and focus on SMART goal setting with the mentee.

Training format

The Comskil Mentor Training was a 3-hour training in either December 2021 or January 2022. The program was developed and implemented over ZOOM due to the pandemic. Training included a didactic component (45 min), facilitator-led small group role plays (90 min), evaluation (30 min), and a 15-minute break. The didactic component included a lecture and interactive discussion addressing mentoring terminology, importance of mentoring in an academic setting, SMART goals, feedback, barriers to mentoring, and a conversational blueprint to provide mentors with communication skills necessary to guide mentees (Table 1). The blueprint was discussed and provided in digital format to participants. Exemplary videos were embedded into the didactic presentation to illustrate key communication skills.

After the didactic (using the breakout room feature in Zoom), participants were split into groups of three for the role play sessions, which were co-facilitated by Comskil faculty and other MSK faculty mentors. Prior work in communication skills highlights that small-group role sessions, made up of 2–3 learners are usually preferable for skills acquisition as they allow each learner dedicated time in their particular role [17]. Most of the facilitators were faculty who were involved in the development of the Comskil Mentor Training program. All facilitators had previously participated in a Comskil Facilitator Training that teaches our approach to training and feedback, as well as the structure and format of the training sessions. These experiential exercises allowed each participant to practice specific mentoring strategies with a standardized mentee (SM), portrayed by a trained actor, and then to engage in feedback discussion with fellow peer participants and facilitators.

Finally, for evaluation, participants completed post-training surveys and one Standardized Mentee Assessment (SMA). The SMA included a 10-minute video recorded interaction between participant “mentor” and SM on a provided scenario, using standardized scripts for the SM. At the end of the SMA, the participants were thanked for their participation and logged out of the Zoom training.

Measures

Study measures included training evaluation (post-training), self-efficacy in communication with mentees (pre- and post-training), communication skills uptake and mentoring-specific language use (assessed using SMAs, at both pre- and post-training). The pre-training questionnaires and SMAs were completed within 2 weeks prior to the training, and post-training questionnaires and SMAs were completed immediately after-training.

Training evaluation This measure was modeled after prior program evaluation measures created by the study team [13]. Participants completed a post-training evaluation consisting of 15 statements using a 5-point Likert scale with anchors of (1) “strongly disagree” to (5) “strongly agree.” The statements measured post-training attitudes regarding engagement (e.g., “I got easily distracted during the role play”), novelty (e.g., “The role play was different than other mentoring trainings I have participated in”), and reflectiveness (e.g., “This role play made me think about specific things I can do about my communication skills when engaging with my mentees”). A higher score indicated a more favorable evaluation.

Self-efficacy in communication with mentees Self-efficacy was modeled after prior self-efficacy measures created by the study team [18]. The measure consists of five items using a 5-point Likert scale with (1) “strongly disagree” to (5) “strongly agree,” such as “I am confident in my ability to guide mentees to formulate career goals”, and “I am confident in my ability to guide mentees in planning how to achieve career and personal goals.” A higher score indicated high confidence.

Communication skills uptake via standardized mentee assessment (SMA) Before and after training, each participant completed a SMA, a 10-minute video recorded interaction between participant “mentor” and SM on a provided scenario. The SMA scenario was based on concepts taught in the training, including introduction and aligning expectations. We adapted the established Comskil coding system [19] for mentoring [mentoring Comskil Coding System or mCCS] to code all video recorded SMAs. The mCCS codes presence/absence of verbal utterances (skills) that are present in the mentor-SM interaction but does not code for nonverbal behaviors. The mCCS includes 20 individual skills, grouped under five communication skills categories: agenda setting, checking, questioning, information organization, and empathic communication.

Communication skills uptake was assessed by indicating the presence/absence of each of the 20 skills used in the SMA. We measured the count of how many unique skills were used, regardless of how many times each was used, and could range from 0 to 20. We also created an overall combined score for communication skills by summing together all scores received in the five categories, for pre- and post-training. A higher score indicated more utilization of communication skills in the interaction.

Mentoring-specific language use via SMA Based on our blueprint that contains strategies, skills, and process tasks, we developed a checklist of 28 mentoring-specific language uses including: “ask about the mentee’s prior experience (clinical, research, or teaching),” “ask about SMART goals (must mention the term “SMART goal”)”, and “discuss institutional goals.” Coders coded the presence/absence of each mentoring-specific language use. We measured the count of how many unique skills were used, regardless of how many times each was used, and could range from 0 to 28. Summed scores were created; a higher score indicated more utilization of mentoring language in the interaction.

Coding

Two trained coders coded all the SMA videos using the mCCS. We assessed inter-coder agreement at the beginning of coding and at the midpoint by double coding (i.e., both coders coded the same set of SMAs to ensure agreement) 10% of data. We asked all coders to provide an example utterance for each code and highlighted where there were disagreements. The coders then met with the team to discuss and provide additional training around disagreements or incorrect codes before allowing coders to proceed independently.

Data analysis

The data were analyzed using SPSS 24 for Windows (IBM Corporation Armonk, New York). For training evaluation, a rating of “agree” or “strongly agree” on each evaluation item was considered as endorsement of the training and was analyzed descriptively. For assessing improvements in self-efficacy, communication skills and mentoring-specific language use respectively, paired t tests were used to assess significant pre-post-training differences. Two-tailed significance tests were used and p < .05 was considered statistically significant [11]. In the results, both significance levels and effect sizes (Cohen’s d) are reported (d = 0.2, small; d = 0.5, medium; and d = 0.8, large effect).