We need a remedy for the vaccine data epidemic

Who among us is not confused by the sheer volume of news and information about Covid-19 and the vaccines meant to halt the pandemic? There are deliberate anti-vaccine lies, such as rumours spread in Asian communities that the injections contain animal products. But the biggest difficulty is knowing what is scientifically true.

This week, it emerged that a preliminary study in South Africa had found that the Oxford/AstraZeneca vaccine failed to stop mild to moderate infections from a viral variant there, and scientists in Australia called for a pause in the jab’s rollout. Meanwhile, the World Health Organization approved its use in all adults, including over-65s, saying the study was “no reason” to shun it.

The Sun reported that one shot of the BioNTech/Pfizer vaccine had given over-80s 64 per cent protection against symptomatic infection in the UK’s vaccination drive. But a scientist told the FT that it was “misleading to the public to give exact figures at the moment” because the numbers changed daily.

All clear? No, of course not.

The problem with all of this data is that very few of us are trained to interpret medical research, or grasp the significance of sample sizes and confidence intervals. We crave accessible and unambiguous answers, not a range of possibilities.

Science is making progress remarkably fast. But the speed at which the advance is occurring and the amount of specialist data being shared is unprecedented and unsettling. We not only need to protect ourselves from the risk of coronavirus infection, but adjust to a data epidemic.

In one sense, medicine is no different from other fields. The internet has opened access to unfiltered information and anyone can scroll Twitter for news, rather than picking up a newspaper or listening to the radio. The accuracy of the latter varies but the best are checked and edited.

Today’s world brings great benefits in allowing us to find an extraordinary wealth of analysis and scholarship at the click of a button, and also encourages propaganda, fake news and rumour mongering. We are wearily familiar with the perils of this constant onslaught of information.

But medical studies contain specialist research that is not written for everyone: it has traditionally found its way to the public via a series of filters. They have been reviewed by experts, edited and published in journals and released to the media under embargo to give journalists time to digest.

The pandemic has accelerated a transition that was happening already — more bioscience research being published on the internet shortly after being written and before studies are peer-reviewed. They are called preprints and they appear on servers such as bioRxiv, operated by a US laboratory.

That has enabled scientists to share information rapidly around the world — the coronavirus genome was posted publicly in January last year, letting vaccine scientists start work. Most researchers have not waited for publication in professional journals, a process that typically takes several months, although it has been shortened during the pandemic.

The first Covid article also appeared on bioRxiv that month and some 28,000 Covid preprints were this week listed on a European server, while journals have published fewer than one-fifth of that number. You are free to read most of them, although I would not advise beginners to try.

This resource is valuable for science. Many papers will go on to be assessed and published and only a few get withdrawn — notably one study briefly posted on bioRxiv that suggested an “uncanny similarity” between Covid-19 and HIV viruses. “There will be some noise and nonsense, but the volume of data helps,” says Robert Kiley, head of open research at the Wellcome Trust.

But putting out so much unedited also has side effects. The initial screening of preprints to prevent posting of material that could cause harm has been tightened. BioRxiv also carries the warning that “journalists may misinterpret preprint results, overlook context critical to understanding, and base news coverage on preliminary findings that might be wrong.”

No doubt true, but it raises the question of why the unreliable findings are there to begin with. There is a risk of scientists speeding up and sidestepping expert review and oversight, then wanting the media to slow down and exercise more restraint to compensate.

It is neither possible nor desirable to put the preprint genie back in the bottle — speed is crucial in a pandemic. Nor is there much point in two tribes of professionals blaming each other. The media are not uniform and some journalists — notably my colleague Tim Harford — specialise in sifting data carefully.

More such scrutiny is needed, along with scientists who find new ways to assess each other’s work fast, and politicians who do not leak selectively. While we are waiting, take comfort in facing too much information about Covid, rather than too little. We may be confused, but we are better off in the end.

john.gapper@ft.com