The Pandemic’s Risky Bet: Publishing Science Before It Was Checked

A new study found that four in five rapid COVID studies kept their conclusions after expert review – showing that speed doesn’t mean sloppy.

What was it like at the start of the pandemic?

In the early months of 2020, fear hung in the air as much as the virus itself.

No one knew what was safe. Could you catch COVID-19 from a doorknob? Was it in the air? Would a vaccine ever be possible?

Advice shifted daily. One day masks were discouraged, the next day they were essential. Families disinfected groceries. Schools shut overnight. Hospitals braced for an unknown enemy.

Where was all this new information coming from?

Behind those shifting headlines were scientists working nonstop, trying to understand a virus spreading faster than the world could keep up.

Normally, research sits for months, sometimes years, while other scientists check it for mistakes, a process called peer review.

But during COVID, there was no time to wait. Scientists posted their findings instantly on preprint servers. In life sciences, the most widely used are bioRxiv (for biology) and medRxiv (for medical research). Each lets researchers share their findings instantly so others can read, reuse, or build upon them before formal publication.

The papers shared there are called preprints, rough drafts released before expert review. Think of it like releasing the rough cut instead of the final film.

These preprints shaped government policies, hospital protocols, and the everyday choices of millions.

That raised a critical question: if the world was making life-or-death decisions based on preprints, how reliable were they?

What did this study look at?

To find out, scientists from the University of Liverpool and the Alan Turing Institute looked at almost 15 000 early research papers (preprints) posted online in the first months of COVID-19.

They picked 184 of them that were later published in scientific journals after formal peer review.

Then they compared the two versions, the early preprint and the final published paper, using both computer programs to track wording changes in the abstracts (the short summaries) and human expert reviewers to judge whether those changes altered the scientific meaning.

They asked three simple questions:

Did science change? Were new figures, tables, or data added?

Did the story change? Did the main message stay the same or shift?

Did the process improve? Were peer-review reports shared publicly? Was the data easier to access? Were there new authors or changes in credit?

What did they find?

Did science itself change?

Many feared that rapid research would collapse under scrutiny. But the results were reassuring.

Most of the studies kept their main take-home message the same – about 8 in 10 COVID papers and more than 9 in 10 non-COVID papers. And when it came to the actual figures and tables, roughly three-quarters barely changed at all.

The same experiments. The same graphs. The same evidence.

In other words: the preprints that shaped policy and public debate weren’t wild guesses. They were early drafts of science that mostly stood firm.

Did the story change?

The data may have stayed steady, but the way results were told sometimes shifted.

Abstracts, the short summaries that most people read, were less stable:

17% of COVID-19 abstracts changed their main conclusions before publication.

7% of non-COVID abstracts did the same.

In one rare case, a published abstract directly contradicted its preprint.

Most of these shifts weren’t about overturning results. They were about nuances.

Published versions often include stronger statistics or larger samples. Authors sometimes reframed from findings more cautiously.

Still, in a global crisis, even small changes in the tone could ripple far beyond the lab.

Did the process improve?

Here the news was less encouraging.

Very few papers shared peer-review reports publicly. Readers couldn’t see what experts had challenged.

The rules on sharing data weren’t consistent. A few COVID papers actually made less data available once published, while many non-COVID papers added more.

Science was moving at an emergency speed. But the usual checks and balances weren’t always visible.

So, can we trust preprints?

The answer is mixed.

The reassuring part: most preprints held up. More than four out of five COVID-19 preprints kept their main conclusions intact. Policymakers and journalists weren’t built entirely on sand.

Caution: some cracked under pressure. Nearly one in five COVID abstracts changed in meaningful ways. For a public desperate for clarity, those changes could feed confusion or mistrust.

Where does this study fall short?

Like all research, this analysis had limits.

First, it only looked at preprints that got published, leaving thousands that didn’t. That likely skewed results toward stronger studies.

Second, it covered just the first chaotic months of 2020, when everything moved unusually fast. Later patterns might differ.

Third, some authors revised their preprints based on informal feedback before even submitting to journals—making it hard to know which changes came from peer review versus colleague comments.

This was a snapshot of science under extreme pressure, not the full story.

What does this mean for the future?

The pandemic turned preprints from a niche academic tool into a central pillar of global science. They are here to stay.

But if they are going to remain part of everyday decision-making, they need strong safeguards:

Transparent peer review, so readers can see what experts questioned.

Consistent data sharing, so results can be checked independently.

Change tracking, so readers can follow how a preprint evolved into a final paper.

Handled well, preprints could make science not just faster but more trustworthy.

Why does this matter for all of us?

If traditional journals are meant to be like hardcover books, carefully edited, polished, and slow to appear, then preprints are like live blogs.

They show science in real time: messy, dynamic, sometimes corrected on the fly.

During COVID-19, the world read live blogs. It wasn’t flawless, but it mostly told the truth.

Science bent under pressure but didn’t break and the next time a crisis arrives, whether health, climate, or something we can’t yet imagine, preprints will likely be an important tool.

Knowing what they are, and how to read them with a critical eye, won’t just be for scientists. It will be for all of us.