A Central Line Workshop (CLW) was developed by the Mayo Clinic Multidisciplinary Simulation Center (MCMSC) to integrate experiential techniques for improving learner performance to safety and proficiency standards set by Mayo experts in education and practice. This study describes inter-rater reliability assessment of a validated instrument for demonstrated proficiency exercises.
A checklist based assessment tool was designed for performance evaluation. Guidelines from Mayo Clinic clinical practice and literature were compiled. After achieving group consensus, those items which were both important to clinical practice and feasible to be trained and tested within a simulation center environment were chosen. Detailed behavioral anchors were developed to help raters to identify learners’ actions during testing session. In the simulation setting, learners’ central line procedures were taped and rated by two raters independently using the checklist. Behavior anchors were revised after resolution of discrepancies within an initial testing sample.
Total of 20 videos were reviewed. The overall kappa value of all videos review was 0.71 with the inter-rater agreement was 88.9% (84.6–92.1%).
The checklist based central line performance assessment tool is a reliable instrument and can be used for assessing learner performance during simulated central line procedures. Its reliability can be improved by rater training.
Demonstrated performance to a safe level of proficiency standard in this experiential learning CLW format has strong applicability for clinical training. More work is needed to add other components of the competence domain into assessment methodologies, including overt clinical performance and outcome metrics at the bedside. Having such a validated assessment instruments may facilitate future competency processes in healthcare.
Yue Dong, No Financial Disclosure Information; No Product/Research Disclosure Information