Revisiting the 'stick'
Despite proof that the official 1,000-year temperature history (the hockey stick) is wrong, government scientists refuse to correct the flaws in the data

By Steve McIntyre
Financial Post (Toronto), June 17, 2005

highlighting by meteoLCD

In the global warming debate, one of the most potent tools of Kyoto treaty advocates was the "hockey stick diagram," which became famous a few years ago when the Intergovernmental Panel on Climate Change (IPCC) used it to argue that the "1990s were the warmest decade in the millennium and 1998 the warmest year." These sound bites were used in speeches advocating Kyoto during the 2002 ratification debate; the government of Canada promoted the hockey stick on its Web site, sent it to schools across the country and quoted its conclusion in pamphlets mailed out to all Canadians.

The "hockey stick" theory overturned the findings in the first IPCC report that the world's climate had been warmer in the medieval era, when, for example, Vikings settled in Greenland.

In two peer-reviewed articles published this past winter, Ross McKitrick and I showed that there had been no effort by the IPCC to verify the hockey stick study, and that there were problems in the calculations sufficiently serious to overturn its conclusions. Our main article was published in the same scientific journal that published the hockey stick graph used by the IPCC.

The story was reported around the world. Coverage began in the National Post and the Dutch science magazine Natuurwetenschap & Techniek. Since then articles have appeared in, among others, Nature, Science, The Economist, and the front page of The Wall Street Journal. The story has been reported on the BBC and Global, as well as German and Dutch television. My Web site -- www.climateaudit.org -- has received more than 250,000 hits since mid-February.

Our most publicized claim has been pretty much universally accepted: We showed that an unreported step in the original calculations mines datasets for hockey-stick shaped series. We showed that this method can produce hockey sticks even from random data. Since we published our computer code, many others easily verified this result.

The authors' original study puts the maximum weight on the most controversial data, in that the hockey stick relies on indexes of tree ring widths to project temperatures. Amid more than 400 tree ring series, the authors included a controversial set of 15 U.S. bristle-cone pine records, which have a pronounced hockey stick shape. However, the specialists who studied bristle-cones had explicitly stated their hockey stick shape is not a temperature signal but is likely due to aerial carbon dioxide fertilization. The hockey stick program loads maximum weight on these bristle-cone records: If they are removed from the data, the hockey stick shape disappears. We showed that the authors had discovered this themselves and they not only failed to disclose it, they claimed the opposite in a later commentary on their own work.

We also showed that the hockey stick authors (Mann, Bradley and Hughes) had withheld vital data (certain verification statistics) that showed their conclusions were statistically insignificant, and that their interpretation of the one verification statistic they did report was incorrect.

The reaction from climate scientists has been varied. Richard Muller of Berkeley likened our contribution to removing a piece of a jigsaw puzzle that was in the wrong place so that investigation about climate history can resume with a clean slate. Hans von Storch, a famous German climate scientist, said it was "good that debate about the temperature history of the last millennium can be resumed again without reservations," and that we were entitled to "thanks" for this contribution. On the other hand, Andrew Weaver of the University of Victoria, a prominent Canadian climate scientist, said our original paper should have been "rejected" and he believed that giving equal space to both sides in a dispute can be dangerous, particularly when applied to scientific matters.

To date, none of our claims has been disproved. This is not to say they have all been accepted or that our work has not been criticized. There has been much more effort by climate scientists to try and disprove our results than ever went into checking the original hockey stick. We made the process easy by publishing all our computer code, unlike the hockey stick authors, who still refuse to release theirs seven years after the original publication. They told the Wall Street Journal that to show the algorithm they used would be "giving in to intimidation."

We know of five submissions thus far to academic journals commenting on our most recent results (in addition to two submissions last year on some earlier results). In the United States, the mere submission of two papers criticizing our results prompted the University Corporation for Atmospheric Research (UCAR), a prominent, federally funded institution that receives hundreds of millions of dollars for climate research, to issue a nation-wide press release declaring our criticisms were "unfounded." Although one of the two papers was shortly thereafter rejected by the journal (the other is still undergoing review), UCAR has not announced its rejection and the original press release remains on the UCAR Web site.

Without getting into particulars beyond what has been publicly disclosed, none of the papers commenting on our work actually contest any of our specific findings. None dispute the undisclosed computational step. None contest the unacceptable dependence of the results on the bristle-cone pines; none try to argue that bristle-cone series are a valid "proxy" for temperature history. None address the failure of the hockey stick to pass simple verification tests.

Instead scientists are trying to argue that the hockey stick errors "don't matter." One style of comment does not test the impact of the erroneous method on the hockey stick itself but on completely different data sets or on unrelated computational problems. Our reply to these responses is more or less: "So what?" The only context we are interested in is the actual hockey stick itself.

The other type of response is to argue that a hockey stick can be produced even without the erroneous method by, for instance, increasing the number of principal components used to represent the North American tree ring network. But every such permutation that we have seen boils down to a back-door method of allowing the bristle-cone series to dominate the final results. Once you are aware of the role of these defective proxies in the hockey stick, you can't simply ignore them or reintroduce them (as the authors did). But this is what is being attempted. Further, these salvage attempts fail common statistical verification tests. But in every example we have seen, these failed statistical tests are withheld from the reader, as they were in the original article and as they are in the papers cited in the UCAR press release.

A third type of response has been to mischaracterize our work. As Muller and others have clearly understood and as we have explained on many occasions, our work to date has been entirely critical. We are not advocating our own reconstruction of climate. We are simply arguing against "flawed intelligence" which is not backed by the data. If this reopens debate for other interpretations, including those held by the IPCC in the pre-hockey-stick-author era (see the lower half of the chart above), then that would be a welcome outcome.

What has been the reaction from the government and the IPCC? Not once have we been contacted by Environment Canada or any other Canadian government ministry dealing with climate research to discuss our work. I contacted Canada's then-chief climate science advisor (Henry Hengeveld) last fall and took him to lunch to explain our work. He shrugged it off and never followed up. Environment Canada has a comment on its Web site dismissing our work, based only on a claim by the original authors that the errors did not matter. A reader from Manitoba forwarded to us an e-mail from Environment Canada responding to his question about why they still promote the hockey stick. Apparently they have dismissed our research on the basis of some unpublished and fallacious commentary they found on the Internet, without ever asking for our input. We have had no contact from the IPCC either.

Our efforts to promote the concept of auditing important climate studies prior to usage in public policy is getting increased attention. We have learned that people have the wrong idea about journal peer review. Users of scientific research for policy-making generally assume that when an article is published in a peer-reviewed journal it means someone checked the data, checked the calculations and checked that the stated conclusions are supported by the evidence presented. But peer review does not guarantee any of this. Influential papers in climate research can go for years without
the data or methods even being disclosed, let alone independently checked, even as huge policy investments are made based on them. So we have urged policy-makers to put in place formal processes to ensure complete disclosure of data and methods for any scientific work that is being used to drive policy debates. We urge the development of audit procedures to verify compliance with such requirements. We believe such innovations would be good for science and good for the policy-making process, even if a few more scientific icons get broken as a result.

One of the first places we would recommend such procedures is the temperature data set used by the IPCC. Other researchers have tried without success to get access to the supporting data. One of them shared with us the response he received from the principal author of the dataset: "We have 25 years invested in this work. Why should we let you look at it, when your only objective is to find fault with it?"