Performing a colonoscopy is difficult. Therefore, CHI’s partners have developed a feedback system for competence assessment of doctors performing colonoscopies. The past two months PhD Student Andreas Vilmann has been testing whether the feedback improves the learning of unexperienced doctors.
After a 40-minute introduction the newly qualified doctor Tine Monberg is holding a colonoscope in her hand and is about to perform her first colonoscopy. Fortunately, she will be performing the procedure on a rubber simulation model and not on a patient, as it would be an unpleasant and painful experience for the patient to have an untrained doctor perform the procedure. For two hours Tine practises on her own, while a doctor and PhD Student Andreas Vilmann merely observe and record her progress and challenges.
Andreas Vilmann is affiliated to the University of Copenhagen and the simulation centre CAMES at Rigshospitalet, and he is doing a research project together with PhD Student David Norsk from the Technical University of Denmark, among others. Here he is testing the effect of an automated feedback system for competence assessment of doctors performing colonoscopies – a procedure the doctors will be making a lot more of in the years to come due to the national bowel cancer screening programme (see facts box). We therefore need more doctors, who are able to perform the procedure safely and gently.
Facts: Bowel Cancer Screening
Bowel cancer is one of the more frequent forms of cancer found in Denmark. Therefore, bowel cancer screening is offered to all citizens aged 50-74 years, who must submit a stool sample, and if the sample shows signs of blood, the citizen is offered a colonoscopy to determine whether this is a result of a precancerous lesion. This makes it possible to detect cancer as early as possible and thus increases the survival rate.
A Score for Stable Skills
The feedback system is based on the so-called Colonoscopy Progression Score (CoPS), which is calculated on the basis of the colonoscope’s movement through the bowel and reveals how well the colonoscope operator performs the procedure.
‘A skilled coloscopist will perform a smooth progression through the bowel, regardless of whether it is a relatively straight section which is easy to navigate, or it is one of the sharp corners or an unpredicted loop, which is significantly more difficult to navigate’, Andreas Vilmann explains. In other words, if the operator rushes through the ‘easy part’, but spends 30 minutes jostling painfully through the bowel in order to get around a corner, it will be reflected in the score, which will be lower. The higher the score, the better the performance, and the less painful it will be to the patient.
Scores of Experienced Doctors Are Used as Motivating Yardstick
On this basis Andreas is now testing the effect of the CoP score on the learning of untrained doctors practising the procedure. Therefore, Andreas informs half of the newly trained doctors he is testing of their CoP score each time they have completed a practice colonoscopy on either an easy or a difficult simulation model. For the sake of comparison, they are also given the score of nine experienced specialists also tested by Andreas.
‘The nine doctors are experienced coloscopists. They each perform between 250 and 1,000 colonoscopies each year. Their scores are a good benchmark for determining when a procedure is good and for the test persons to relate to’, Andreas says. ‘Our hypothesis is that feedback on the quality of the performance will increase the motivation and the amount of time spent practising, which results in increased learning. We will therefore also look at whether feedback increases their technical skills’, says Andreas Vilmann.
The other half of the test group, on the other hand, is a control group that does not receive feedback. Tine Monberg is part of this group, and she spends her two hours of self-training struggling with the difficult simulation model and only manages to complete the procedure a few minutes before the session is over, with no indication of how well she did. When she later hears that other participants receive feedback, she regrets not having been a part of this group. ‘It would have been a big help’, she says.
Feedback Appears to Change Learning Patterns
Andreas Vilmann began working on the project in December 2016 and will be completing the last of his 44 tests in February 2017. Even though it is still too early to draw any conclusions, he nevertheless shares his thoughts on the project. ‘I have a feeling that the learning patterns change in the group that receives feedback. Contrary to the control group they practise on the easy model several times in order to improve their score before trying their strength against the difficult one. They want to do well before moving on. But it is really too early to draw any conclusions’, he stresses.
Goal: All Colonoscopies Are Performed with Feedback
Version 2 of the CoP score will be tested in 2017. In this connection the score will be based on a 3D tracking of the movements of the entire colonoscope and not just, as is the case now, the movements of the tip. This makes it possible to give a more accurate CoP score and live feedback on the movements of the entire colonoscope, revealing to the operator if he or she is about to do a painful loop of the bowel. The 3D score will be used in the next round of tests next summer, which will look at the effect of feedback in a diagnostic simulation trial.
Once the feedback system has been further qualified for clinical studies, it will eventually be integrated into all clinical colonoscopy wards. Then the feedback can, in addition to training of inexperienced doctors, also be used as a guiding tool in the performance of all colonoscopies of the bowel, providing live data on the quality of the examination. This means that a more experienced doctor can step in if necessary, giving the patient as comfortable and gentle an experience as possible.
The research project is part of the Copenhagen Health Innovation project Quality Assurance of Colonoscopies, which will also include a study programme based on this kind of simulation training for training and certifying doctors in the procedure.
The project is funded by the Capital Region of Denmark’s Regional Growth and Development Strategy.