Google search basis undermines sunspot-winter coldness link

Franck Sirocko’s 2012 study incorrectly dated this 1929 postcard identifying a year that the Rhine froze as being from 1963, which is one of many problems Geert Jan van Oldenborgh and his colleagues found with it. Image from van Oldenborgh et al, used under Creative Commons license, see citation below.

European researchers have strongly criticised a recent study linking cold winters in the continent to cycles affecting the sun for relying on a shallow internet search. In August 2012, Franck Sirocko at University of Mainz, Germany, and his teammates linked cold years to sunspot activity lows using historical reports of when the river Rhine froze. But their results disagree with previous research, and previously unpublished findings from Geert Jan van Oldenborgh from KNMI, the Royal Netherlands Meteorological Institute, in De Bilt. And when Geert Jan looked into why this was, he found problems common in research on this topic over 50 years ago, updated for the internet age.

“These problems are fundamental – all the results that they claimed are spurious,” Geert Jan told me. “It is simply an incorrect paper. Usually incorrect results are just ignored, they do not get cited much and are quickly forgotten. However, this time we took the unusual step to write a comment on the paper. This decision was based on the low quality and the wide publicity it was given.”

That publicity came largely because the American Geophysical Union, which published the 2012 paper, put out a press release about it that the media reported widely. It tells how Franck’s team used historical documents to find that the Rhine froze in multiple places fourteen different times between 1780 and 1963. 10 of the 14 freeze years occurred close to the point in an 11 year cycle when there are fewest sunspots. “We provide, for the first time, statistically robust evidence that the succession of cold winters during the last 230 years in Central Europe has a common cause,” Franck said in the press release.

Sunspot cycles had been linked to weather throughout the 19^th and 20^th centuries, until Barrie Pittock started going over the evidence in the 1970s. Barrie, who led the Climate Impact Group at the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia until his retirement in 1999, found no link beyond day-to-day weather effects. He also found many studies had used bad or incomplete data to say otherwise.

Internet and climate research: Dos and don’ts

Directly measured temperatures in Frankfurt do not increase with sunspot number, shows the flat line in this graph, a similar version of which can be easily created using Geert Jan’s KNMI Climate Explorer website. Image from van Oldenborgh et al, used via Creative Commons license, see citation below.

Geert Jan had read these studies, and also created the KNMI Climate Explorer website, which includes a wide range of data, including temperature and sunspot numbers. On it, a few mouse clicks quickly show no links between winter temperatures and sunspots that pass statistical significance tests back to 1871 in Frankfurt, near the Rhine. So Geert Jan and four other scientists decided to examine the 2012 paper’s findings further. And with three of them lead authors frantically preparing the upcoming UN Intergovernmental Panel on Climate Change report, they had little choice but to sacrifice their little spare time to do so.

First, they checked if they could see the basic pattern Franck’s team reported in four other long temperature records. Two of those use ice on lakes and canals to judge winter severity, one back to 1200 in the Netherlands, and the other covering 1525-1989 in Switzerland. The third is the longest directly-measured data set in the world, the Central England Temperature record that runs continuously from 1659. The fourth also records directly-measured temperatures, reaching back to 1706 in the Netherlands. Geert Jan’s team found no obvious connection with the 11-year sunspot cycle.

Franck’s team had also explained the link it found through changes in an atmospheric pressure pattern called the North Atlantic Oscillation (NAO). But Geert Jan’s team found that in doing this they’d changed their definition of what is ‘close’ to a sunspot low. The ten occasions when the river froze close to a sunspot low were spread over a four-year period from two years before to two years after the low itself. But the only time Franck’s team looked at NAO for was one year after the low point, when just one of the sunspot-river freeze links happened. And when Geert Jan used his website to redraw the 2012 paper’s explanatory maps, one year after was the only definition of close to a sunspot low that showed any link to the NAO.

As well as these contradictory definitions, closer analysis revealed more serious problems. “We found that the freezing years had been selected totally arbitrarily using an ‘internet survey’,” Geert Jan said. “They had not been verified against the available instrumental and documentary data. Even the postcard that accompanied most newspaper articles turned out to be from 1929 and not 1963 as claimed.” He calls this a “gentle reminder that a quick Google search is no substitute for real science, which should be common knowledge”.

Peering at science’s innards

Geert Jan’s team plotted winter temperatures in the Netherlands and Switzerland from before direct measurements were taken against each other. They made the years where Franck’s team found the Rhine froze near to a sunspot low larger. There are as many years not near a sunspot low as cold as those near a sunspot low. Image from van Oldenborgh et al, used via Creative Commons license, see reference below.

Geert Jan suggests that science wasn’t working properly in this case. “The scientific process tries to balance two forces: The creativity to find new connections between seemingly separate phenomena; and the criticism to check that the new ideas are in accord with the known analyses and observations. This paper failed on both counts: the idea that low solar activity is connected with severe winters has been around for a long time, and they failed to do the most elementary checks against the available literature and data.”

One key stage in the scientific process is publication in a scientific journal that uses ‘peer review’. Scientists send in their work, and the journal decides whether to send it to other, independent scientists. They in turn decide whether it’s good enough to be published, much like a driving test examiner decides if you’re good enough to drive. And as glaring as the problems in the 2012 paper seem, Geert Jan’s team had problems highlighting them publicly thanks to this process. The paper they wrote containing them was first rejected without review by Geophysical Research Letters, where the original paper had appeared.

But an ‘unholy alliance’ of scientists helped make sure the issue was made public. One was University of Colorado, Boulder’s Roger Pielke Sr, who has a different view from many climate scientists in claiming that added CO2, while important, is not the only highly significant driver of changes in climate. He urged team-member Jos de Laat, ‘not to let this pass’. The other was Potsdam University’s Stefan Rahmstorf, who agrees with the vast majority that CO2 is the most significant driver. Stefan encouraged Geert Jan’s team to send their paper to Environmental Research Letters, where he’s an editor. He assured them it would be reviewed there, and that’s where it was published last week.

That the 2012 paper was published while the response struggled shows that this way of criticising science isn’t perfect – even if it usually isn’t this bad. “Peer review is not waterproof,” Geert Jan said. “Reviewing is a volunteer job, and some editors and journals are better at enforcing high standards in reviews than others. Rejected papers at one journal are often resubmitted at other journals, enhancing the chance that they slip through. These kind of very wrong papers are not common, but they do appear every now and then.”

Geert Jan’s team also plotted winter temperatures in the Netherlands and Switzerland from since direct measurements were taken against each other. They made the years where Franck’s team found the Rhine froze near to a sunspot low larger. Many of those years were not the coldest in this period. Image from van Oldenborgh et al, used via Creative Commons license, see reference below.

• I have made multiple attempts to contact Franck Sirocko, and Eric Calais, Editor-in-Chief of Geophysical Research Letters, and the journal’s publishing team by email and telephone to give them the opportunity to comment on this story. As yet I haven’t succeeded. I’ll post any comments as I receive them.
• Update 0908 BST, May 4 2013: Eric Calais has said the following in response to my request for comments on this story: “As in all major scientific journals, our review process is confidential. I therefore cannot comment on the questions you are asking.”
• Updated 1827 BST, May 4 2013 to more accurately describe Roger Pielke Sr

Journal references:

Sirocko, F., Brunck, H., & Pfahl, S. (2012). Solar influence on winter severity in central Europe Geophysical Research Letters, 39 (16) DOI: 10.1029/2012GL052412
Pittock, A. B. (1983). Solar variability, weather and climate: An update Quarterly Journal of the Royal Meteorological Society, 109 (459), 23-55 DOI: 10.1002/qj.49710945903
van Oldenborgh, G., de Laat, A., Luterbacher, J., Ingram, W., & Osborn, T. (2013). Claim of solar influence is on thin ice: are 11-year cycle solar minima associated with severe winters in Europe? Environmental Research Letters, 8 (2) DOI: 10.1088/1748-9326/8/2/024014