Some time back, I wrote a post about basic truths everyone should be able to agree to. Unsurprisingly, some people refused to agree to them. One such person banned me from his blog (And Then There’s Phsyics) because of an exchange about them. That banning came up at Judith Curry’s blog recently, and more people have decided to join in on the denial.
I normally wouldn’t bother writing a post about this. However, one such individual, Tom Curtis, wrote a lengthy response to some of what I said, and he gets things incredibly wrong. He’s so incredibly wrong, I have to discuss it. Before I do though, I want to highlight Curtis’s description of me and my being banned (referencing a discussion on this post of mine:
After a discussion with Shollenberger on his site, I heartily endorse Anders sentiment of never wanting to discuss anything with Shollenberger again. It is not a moral or a personality flaw to dislike discussing things with people who do not discuss in good faith, show a lack of personal integrity, and behave like complete pricks.
If you agree with his description of me, I suggest you stop reading. You’ll never believe what I say, no matter how right I am.
Anyway, if you’re not familiar with this topic, the current discussion revolves around the blogger who banned me (Anders) having said:
What seems indisputable, though, is that the 10 hockey sticks presented in MM05 (one of the papers, you probably know which one) were not selected randomly from their sample of 10000. They were chosen to be most hockey-stick like. People, however, clearly interpret the results of MM05 as implying that random red noise typically produces produces hockey sticks, rather than random red noise sometimes (probably quite rarely) produces hockey sticks.
You can find the paper in question here. While Anders claims it “seems indisputable… the 10 hockey sticks presented… were chosen to be most hockey-stick like,” the reality is that paper doesn’t even show “10 hockey sticks.” The paper only has six graphs. There couldn’t possibly be “10 hockey sticks” shown in it.
This paper is five pages long. It is completely impossible anyone would look at it and think ten cherry-picked graphs were included in it. It might take all of thirty seconds to scroll through the paper and see such graphs aren’t in it.
Anders made a serious claim about this paper, saying it “seems indisputable,” even though anyone who even glanced at the paper would know it was false. When directly challenged on it, he repeatedly refused to acknowledge it was false (then he banned me). Now, Tom Curtis acknowledges that falsity:
Now, it is clear that Anders made a couple of mistakes. To start with, although the pseudo reconstruction that was published in M&M05 “… were not randomly selected from their sample of 10000?, being in fact selected from their sample of the 1% with the highest HSI; and though the 10 pseudo proxies graphed by the code provided with the supplementary information of M&M05 “… were not randomly selected from their sample of 10000?, the later 10 were not shown in the paper.
Pointing out the graphs Anders criticized were made, but not published, by Steve McIntyre. One obviously cannot criticize McIntyre for misleading people by showing them graphs he didn’t show them. This should have been the end of this. It’s not. Curtis apparently decided to defend Anders by saying McIntyre deceived people with these graphs he didn’t display.
I need to make note of something before I get to that though. Curtis refers to the simulations McIntyre ran as creating “pseudo reconstructions.” That is wrong. These are pseudo proxies. Reconstructions are created by combining multiple proxies. Mixing such things up suggests a poor grasp of the issues at hand. It’ll become important later in this post. In the meantime, Curtis says:
It is clear, however, that while these are mistakes, they are trivial mistakes. They do not effect the substance of the issue. A cherry pick of the top 1% of pseudo reconstructions in terms of the HSI is still a cherry pick, and not informing readers either in the paper or notes on the supplementary information that the graphs generated by the program in the SI are a biased selection is a serious breach of normal standards of publication. This is particularly the case as the only tests conducted by M&M05 to show the ability of red noise to generate MBH98 like pseudo reconstructions is by visual comparisons with the cherry picked selection.
He bolded the last sentence of this for emphasis, but I can’t get the HTML tags to work for it right now. Regardless, he apparently believes that sentence deserves the most focus. I agree. He claims the only test performed in the paper to show Michael Mann’s methodology can create hockey sticks out of red noise is “visual comparisons with the cherry picked selection.”
That is an important claim because it is completely false. Of the three figures in the paper, only one made any visual comparison with “cherry picked” results. That was this figure:
Which the paper clearly described as not using a randomly selected graph:
The simulations nearly always yielded PC1s with a hockey stick shape, some of which bore a quite remarkable similarity to the actual MBH98 temperature reconstruction – as shown by the example in Figure 1. A sharp inflection was regularly observed at the start of the 1902–1980 ‘‘calibration period’’.
The paper described running 10,000 simulations (pseudo proxies) then said “some of” those looked like the Mann et al final result (temperature reconstruction). Saying only “some of” the simulations looked like that clearly informs the reader the example used was not randomly selected. There is nothing remarkable or problematic about that.
Curtis is clearly wrong as a test cannot use “visual comparisons with the cherry picked selection” of 100 simulation runs if only one simulation was displayed. More importantly though, Curtis simply dismissed other tests performed in the paper. The text I quoted from the paper refers to such tests, and the very next figure in the paper:
Shows the “Hockey Stick Index” (HSI) of the 10,000 simulations, proving the Mann et al methodology artificially produces hockey stick shapes out of red noise while the correct (centered) approach does not. This figure, which has two of the six graphs in the paper, is a key result of the paper. Unlike the rest of the figures, it shows the results for the full 10,000 simulations.
You might wonder how Curtis could pretend this figure doesn’t exist. The answer is, he doesn’t. He acknowledges it. In fact, he discusses it. For instance, he says:
That last may seem an odd claim, in that surely a comparison was made using the HSI itself, but in fact M&M05 never publish the HSI of either the MBH98 reconstruction (1.13), the MBH98 580 year PC (0.94), or the MBH 99 reconstruction (1.13). These are 22.7 (MBH98 and 99) and 27.9 (MBH98 PC1) standard deviations less than the mean HSI of the selected 1% of pseudo reconstructions. The are also in the bottom one percentile (MBH98 PC1) or, probably, 2.5%ile (MBH98 & 99) of results for all 10,000 pseudo reconstructions. I say “probably” for the later because M&M05 do not give full statistics. The do state that less than 1% have a HSI less than 1, and that only 27% have a HSI less than 1.5. From the histogram (figure 2), it is possible to determine that about a fifth of that 25% have a HSI less than 1.25.
Curtis begins by comparing the HSI of Mann et al’s temperature reconstruction to the HSI of the pseudo proxies not used in Steve McIntyre and Ross McKitrick’s paper. Again, they didn’t use the “cherry picked” selection of 100 pseudo proxies in their paper. Curtis is using results not displayed to condemn the paper for what it displays.
Beyond that, we’re back to the issue of Curtis mixing up proxies with reconstructions. He suggests a comparison should be made between these (unused) pseudo proxies and Mann et al’s temperature reconstruction (and a PC of that reconstruction). Why would we compare proxies to reconstructions? Maybe so Curtis can say:
You can see, therefore, why McIntyre and McKitrick were loath to do more than visual comparisons. Had they done a statistical comparison using their chosen measure of similarity (the HSI), the paper would have (apparently) reported that the MBH98 method applied to red noise generates hockey stick like shapes but that statistical tests show at the 90% (certainly) or 95% (probably) confidence levels, the MBH 98 was not a random outcome from red noise.
He bolded part of this for emphasis. Curtis apparently believes it is very important we compare Mann et al’s temperature reconstruction to results gathered from simulations of pseudo proxies. That’s silly. The paper clearly states it is looking at the effect on individual proxies (specifically, principal components):
The ‘‘hockey stick’’ shaped temperature reconstruction of Mann et al. (1998, 1999) has been widely applied. However it has not been previously noted in print that, prior to their principal components (PCs) analysis on tree ring networks, they carried out an unusual data transformation which strongly affects the resulting PCs. Their method, when tested on persistent red noise, nearly always produces a hockey stick shaped first principal component (PC1) and overstates the first eigenvalue.
The paper is clearly about proxies. That’s why it generates 10,000 pseudo proxies. Curtis simply fails to grasp this basic point, repeatedly criticizing the paper on its failure to “to show the ability of red noise to generate MBH98 like pseudo reconstructions.”
In the end, Tom Curtis denies the undeniable by doing two things: 1) He repeatedly compares results for pseudo proxies to temperature reconstructions instead of comparing results for pseudo proxies to actual proxies. He goes so far as to deny the existence of certain tests because people failed to compare apples to oranges like he did.
2) He repeatedly acts as though people should only care about the 100 cherry picked series which weren’t used in the paper. He focuses almost entirely upon those 100 unused series rather than looking at the 10,000 series the paper actually uses.
It’s a pretty remarkable way to deny what should be a truth we can all agree on. Michael Mann’s methodology is biased to mine for hockey sticks. Nobody can seriously argue otherwise. Anyone who denies it, a decade after it was made obvious, is a denier.
And that include Anders. He responded to Tom Curtis by saying:
Thanks, I did wonder if that would come up again. Your summary seems about right.