Two weeks ago, when Till Sawala heard the news of a peer-reviewed paper that purported to upend our understanding of the universe, he immediately suspected that something was off. Then again, the paper was published in Nature, one of the world’s most authoritative and influential scientific journals.
“I thought, ‘Okay, this is either one of the most important results in cosmology in the last 10 years, or it’s wrong,’” says Sawala, a cosmologist at the University of Helsinki. “And my instinct is that it was wrong.” In his experience, the more a claim flies in the face of expert consensus, the less likely it is to withstand expert scrutiny. In this case, the Nature paper argued that, at multibillion-light-year scales, the universe’s contents weren’t spread out as uniformly as scientists had thought. The assertion, if true, would overturn decades of cosmic dogma.
“If something this big had been missed, it would have been quite an embarrassment to the community,” Sawala says. “So I thought it was important to correct the record.”
On supporting science journalism
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
The Nature paper concerned a massive dataset of 47 million galaxies and quasars across more than 11 billion years of the universe’s 13.8-billion-year history, as captured by the Dark Energy Spectroscopic Instrument (DESI). The DESI dataset, like many before it, showed that intergalactic matter glommed together into a vast “cosmic web” of galaxy-rich filaments and sheets surrounding enormous galaxy-sparse voids. But the authors of the Nature paper claimed the DESI data also showed that these filaments stretched farther than anyone realized: billions of light-years. Most crucially, the authors said these filaments were oriented in certain directions more than others. If the universe’s large-scale contents indeed had such “preferred” directions, that would violate a rigid dogma known as the cosmological principle.
Upon closer inspection, however, Sawala found problems with how the authors calculated the scale of the DESI data. He argues that they measured the distances of galaxies with a unit called “luminosity distance” when they should have used a different unit called the “comoving distance.” They also neglected to scale these distances to account for how fast the universe is growing. After correcting for these issues, his independent analysis suggests the DESI data are consistent with the prevailing consensus: no mysterious mega-alignments of filaments; no violation of the cherished cosmological principle.
Francesco Sylos Labini, one of the Nature paper’s authors and a physicist at the Enrico Fermi Research Center in Rome, points out that Sawala’s analysis relies on the patchiness of the universe’s large-scale structures rather than their orientation. But Sawala says that the mistakes he’s uncovered apply in either case.
Major journals such as Nature maintain their prestige by featuring the most impactful research—and what could be more impactful than research with revolutionary implications? But as Carl Sagan famously put it, “Extraordinary claims require extraordinary evidence”—peer review is especially crucial in such cases. “In order for a paper to be in Nature, it has to be groundbreaking,” Sawala says. “This was definitely groundbreaking, so it cleared that hurdle. But it turned out not to be correct.”
“It is disappointing that this made it past the reviewers,” says David Spergel, an astrophysicist and president of the Simons Foundation. “Nature’s editors need to be more careful in the future.”
But even if the journal had assigned Sawala as one of the paper’s two referees, he says, he’s not sure that he would have caught such an “elemental” mistake—though he would have had some basic questions. “Being a reviewer is difficult,” Sawala says. “You are usually an expert in only some parts of the paper.”
Cosmologist Daniel Eisenstein of Harvard University, who was not involved in either manuscript, agrees. “Unfortunately, it’s easy to see how this kind of bug could sit unnoticed in a code for a long time,” he says. “It is not obvious to me that a reviewer should reasonably have caught it.”
Sawala has submitted his rebuttal for its own peer review, and the preprint is already making rounds in the cosmology community. But a corrective follow-up to a sensational claim rarely draws the same splashy headlines from mainstream outlets. This tendency to avoid revisiting “yesterday’s news” can misalign the public’s understanding with the science.
These pitfalls of peer review are why physicists increasingly rely on preprint servers, such as arXiv.org, which allow the whole community to judge a paper in concert. “You’d have to be lucky, with one or two reviewers, if they happened to catch this,” Sawala says. “But someone else surely would have if it had been on arXiv.” The Nature paper was not posted to arXiv.org or elsewhere prior to its publication.
When scientists submit a flashy result to a top journal such as Nature, however, they often opt to keep that result secret until a few days before publication, when journalists are given a heads up. This practice—called placing it under “embargo”—makes a paper’s publication a newsier event but does so at the expense of scientific openness.
“I think these embargoes serve the publication more than the science,” Sawala says. “And I think the science should come first.
It’s Time to Stand Up for Science
If you enjoyed this article, I’d like to ask for your support. Scientific American has served as an advocate for science and industry for 180 years, and right now may be the most critical moment in that two-century history.
I’ve been a Scientific American subscriber since I was 12 years old, and it helped shape the way I look at the world. SciAm always educates and delights me, and inspires a sense of awe for our vast, beautiful universe. I hope it does that for you, too.
If you subscribe to Scientific American, you help ensure that our coverage is centered on meaningful research and discovery; that we have the resources to report on the decisions that threaten labs across the U.S.; and that we support both budding and working scientists at a time when the value of science itself too often goes unrecognized.
In return, you get essential news, captivating podcasts, brilliant infographics, can’t-miss newsletters, must-watch videos, challenging games, and the science world’s best writing and reporting. You can even gift someone a subscription.
There has never been a more important time for us to stand up and show why science matters. I hope you’ll support us in that mission.