Bridging the gap between theory and practice: Reflections from missing data KM workshops

Elinor Curnow ran 3 knowledge mobilisation (KM) workshops to demonstrate Multiple Imputation Doctor (midoc) to researchers. The workshops were funded by a University of Bristol Knowledge Mobilisation Catalyst Award. Here she reflects on what happened.

Missing data is a very common problem in health and social studies. Data can be missing because people don’t want to answer some questions, or forget to give some information, or drop out of studies completely. Missing data presents a risk that results of a study will be wrong (“biased”), unless the analysis approach is chosen carefully. One statistical approach that can correct this bias is called multiple imputation (MI). The problem is that MI can be challenging and complex to use in practice, and there is little guidance available.

Developing midoc

With Kate Tilling, Jon Heron, and Rosie Cornish (Medical Research Council Integrative Epidemiology Unit at the University of Bristol), and James Carpenter  (London School of Hygiene and Tropical Medicine), I developed software to bridge the gap between the theory and practice of MI. The software is called the “Multiple Imputation DOCtor”, or midoc.

Workshops

I used the Knowledge Mobilisation Catalyst Award to run 3 workshops this summer to demonstrate midoc. These involved groups working in clinical trials, as well as statisticians working in the NHS with large health registries, and at the Office for National Statistics.

The workshops had 2 objectives. The first objective was that participants would understand how midoc could help them choose when and how to use MI. The second objective was for me to find out which features of midoc were useful, which needed improving, and whether any extra functions should be added.

Anyone who has ever used R software will know that perfunctory output and obscure error messages are the norm! One of the priorities when developing midoc was including clear interpretation of results as part of the output. I wanted to make midoc  as useful and accessible as possible. So I was also keen to find out whether participants in the workshops found midoc user-friendly and with the right level of detail.

Feedback

Participants gave some really positive feedback on midoc. They said they liked walking through a real example and the structured guidance. One participant said:

“Thank you for the demo. I found it very useful to understand more about  Directed acyclic graphs (DAGs) and how to check the validity of imputation. I will consider more about missing data in my future studies!”

However, it was sometimes surprising what participants found confusing. I have already updated some of midoc’s functionality as a result.

Lack of time was often mentioned as a barrier to the in-depth approach suggested by midoc. It was also clear there was a wide range of priorities and levels of experience in missing data methods across the participants.

I’m now working on ways to streamline and simplify the analysis process used in midoc. I’ve also reflected on how to incorporate time-saving tips in the workshop format.

Next steps

I’m now applying for further development funding for midoc. It’s been incredibly useful to have workshop feedback to incorporate into my funding applications. The workshops have also helped me develop future collaborations and identify suitable studies to apply midoc to.

I plan to hold follow-up events with participants to showcase the improvements I’ve made to midoc as a result of their feedback.

Final reflection

Software tools are often developed alongside new statistical methods. However, obtaining user feedback on these tools is frequently overlooked. My Knowledge Mobilisation Catalyst Award has really helped me with this. As a result, I have gained valuable insight into the midoc user-experience. This will ultimately encourage wider take-up of midoc and ensure that as many users as possible are following best practice in missing data methods.

About the Knowledge Mobilisation Catalyst Awards

The Knowledge Mobilisation Catalyst Awards were funded by the University of Bristol Population Health Science Institute (PHSI). Over 2 funding rounds 13 research teams were supported to organise stakeholder engagement workshops and events. These involved a wide range of professionals, industry partners and members of the public on diverse topics including:

• Social care housing
• Care of the elderly education
• Use of AI in healthcare
• Methods for managing missing data in healthcare data sets

Blogs from all the award holders describing the outcomes of their work are available on the PHSI website.