Better Statistics Leads to Better Research

“The best thing about being a statistician is that you get to play in everyone’s backyard.” This quote from the eminent mathematician, John Tukey, sums up Judith ter Schure’s love of statistics. “Statistics is a way to be involved in very diverse fields of science,” she says. Researchers, regardless of their field of work, need statisticians to help them unlock the meaning of their data.

Judith is a PhD student in the Machine Learning Group at the Centrum Wiskunde & Informatica (CWI)—the national research institute for mathematics and computer science in the Netherlands. She was motivated to do a PhD in statistics at CWI by the innovative and inspiring work of her supervisor, Professor Peter Grünwald. He was working on a new statistical method that would allow researchers to analyse data from experiments while they were still ongoing. “It’s a methodology that allows you to stop, start, monitor, and continue whenever you want. And the decisions to do so do not affect the validity of the analysis,” Judith explains.

Judith saw the value of using this approach in meta-analysis of clinical trials. Meta-analysis is a statistical method commonly used to assess data from research that has happened in the past. Data from different related studies are pooled and evaluated. In the standard approach to meta-analysis, there’s no room for flexibility. It’s a retrospective summary that assumes everything from the number of participants to the data collection and endpoints has been planned in advance. It cannot be used to make decisions while studies are ongoing. So researchers keep their focus on individual studies instead of the broader line of research. Over the years, this strategy has contributed to biased results and invalid research. It’s thought that 85% of global health research is wasted because results were never published, studies were poorly designed or the wrong experiments prioritised.

Judith worked with her colleagues to develop a new model called “ALL-IN meta-analysis” to help combat the prioritisation problem. The term ALL-IN comes from the interpretation of the mathematics used when betting in poker. It's also an acronym for Anytime, Live and Leading INterim meta-analysis. Judith says, “With our approach, you can analyse the body of research at any time, even when some trials are still in their interim stages. This steers the focus away from individual trials and visualises what new studies need to offer to avoid research waste. Scientists need to make sure that all relevant trials are in when they hedge their bets on future research." 

The flexibility of this approach has been a major advantage during the COVID-19 pandemic. Judith and her colleagues formed a new collaboration with two Dutch hospitals to examine the impact of BCG vaccination in healthcare workers and the elderly during the pandemic. ALL-IN meta-analysis has allowed them to work on data not just from the Netherlands but from ongoing trials around the world.

While CWI has always fostered research links both at home and abroad, these ties became even more vital during the pandemic. Judith says, “I really noticed many colleagues stepping up, feeling involved, feeling responsible and really thinking, ‘what can I do with my expertise?’” She praises the fact that at CWI, researchers have the freedom to work on interesting topics without constant pressure to be part of the rat race. It was this kind of atmosphere that nurtured the creativity of noteworthy CWI scientists like Guido van Rossum, who invented the Python programming language, and the early career of Constance van Eeden, the pioneering statistician. She was the first woman to graduate with a PhD from CWI in 1958, and the institute is setting up the Constance van Eeden PhD fellowship for female PhD candidates in her honour.  

For Judith, this relaxed atmosphere and her interest in her PhD topic have made working at CWI a very enjoyable experience. She also mentions the supportive library staff who promote open science and help researchers publish their work online. She says, “There’s a reproducibility when you’re completely open and transparent about what you’ve been doing and want to help others repeat your analysis, proofs, or algorithms. The people from our library really support that.”

Judith adds that CWI’s location in Amsterdam is a real bonus. It’s situated at Amsterdam Science Park, a hub of digital innovation, which positions CWI at the heart of European mathematics and computing. It’s a place that foreign scientists like to visit. She says, “It often happens that a scientist who’s a giant in their field says, ‘I happened to be in Brussels and since I was so close, I also wanted to visit you in Amsterdam.’ If we were in a less well-known place, I think it would happen less often. People like Amsterdam.”