For years, a growing chorus of critics has warned that the global scientific process is fundamentally broken. However, a new audit has shattered that cynical narrative, proving that modern research is highly reliable — as long as academics are compelled to share their underlying data.
According to a study published in the journal Nature, the vast majority of recent economics and political science papers are entirely accurate and computationally reproducible.
The research, led by Abel Brodeur, a professor of economics at the University of Ottawa and founder of the Institute for Replication, offers a powerful counter-narrative to recent academic panic, proving that a radical shift toward “open science” is actively saving the integrity of global research.
The reproduction crisis
The new findings were published alongside the results of a sweeping seven-year international audit known as the Systematizing Confidence in Open Research and Evidence (SCORE) project.
The SCORE project investigated nearly 4,000 older social science papers published between 2009 and 2018. The results of that historical audit were deeply troubling: researchers found that they simply could not replicate the results in exactly half of the studies they tested.
However, Professor Brodeur’s concurrent study provided a stark, highly encouraging contrast regarding modern scientific practices.
Between 2022 and 2023, Brodeur and his team hosted a series of rigorous, single-day events dedicated to scrutinising 110 recent articles from top economics and political science journals. The team discovered that a staggering 85 per cent of the modern papers were fully computationally reproducible.
“We now have systematic, large-scale evidence on how reliable social science research really is,” Professor Brodeur explained.
“The direct impact of this will see encouragement for higher research standards like better coding, data sharing, and general transparency so errors can be identified before they influence policy. Indirect impacts should lead to an improvement of trust in science by showing transparency and self-correction to help policymakers rely on more robust evidence.”
Levelling the academic playing field
The researchers attribute this dramatic improvement in reliability directly to the recent, aggressive shift toward open science. Compared to the older papers analysed by the SCORE project, the modern studies in Brodeur’s audit adhered to much stricter norms regarding the mandatory public disclosure of data and coding files.
Moving forward, Professor Brodeur argues that leading academic journals must continue leaning heavily into mandatory data sharing, given how much global public policy relies on these academic findings. Furthermore, he believes that checking a peer’s work should become a routine part of science, rather than an exceptional event.
Beyond simply catching errors, the team noted that publishing open-source data and code acts as a powerful equaliser for the global scientific community.
“This would allow researchers from lower-level universities, those in developing nations, and others who cannot afford expensive licenses to learn from elite scholars,” Professor Brodeur said. “The scale of this ongoing project has the potential to change research norms and researchers’ behaviour.”
