This variance component is comparable to the subject-by-case-interaction variance in a PR-171 nmr generalizability study and indicates the residents’ performance inconsistency.

By standardizing the random slopes variance, we calculated an Inconsistency Coefficient for scores between the first and second consultations. From the multilevel regression equations, we estimated the residents’ CELI scores of the first and second consultations that were not influenced by error components such as rater unreliability. From these estimated scores, we calculated the average score of, and the score differences between the first and second consultations for each resident. We used the absolute value of the scores’ differences as Inconsistency scores of the residents. Since the inconsistency scores were not normally distributed, we used non-parametric tests for further analyses of this variable. We calculated Spearman correlation coefficients

between the inconsistency scores and the average scores, and tested the differences in inconsistency scores between the similar and dissimilar consultation combinations with Mann–Whitney U tests. We used ANOVA analyses to establish the effect see more of CST background on the estimated CELI scores and used Mann–Whitney U tests to establish the effect of CST background on inconsistency scores. Appendix A contains the three-level model and explains the symbols used in the model. The appendix also contains the formulas used to calculate additional means, variances, covariances, and coefficients from the parameter estimates of the multilevel analyses. We used PD184352 (CI-1040) MLwiN 2.26 [44] for the multilevel analyses and IBM SPSS Statistics 20 [45] for the additional analyses. Table 2 contains the parameter estimates of the three-level models for the prediction of CELI scores for all consultation combinations, and for the

similar and dissimilar consultation combinations. Table 2 also contains the variance components, inconsistency coefficients, and correlation coefficients derived from the models. The CELI scores were normally distributed. The overall mean of estimated scores (μ0) for all consultations was 6.03, which means that the average communication performance was less than adequate (=6.70). The mean scores for the first and second consultations did not differ, as indicated by the non-significant mean of difference scores (μdif) of 0.207 (0.167). The mean inconsistency score (μinconsist) for all consultations was 0.948. The standard deviation of score differences between the two consultations (σdif) was 1.18 score points, illustrating the extent of the inconsistency. The normal curve areas indicate that 28% of the residents with a score of 6.7 (=adequate) in one of the consultations would have a score of 6.0 (=moderate) or lower, and 7.5% would have a score of 5.0 (=mediocre) or lower in the other consultation.