Mann’s Screw Up #3 – Statistics is Scary

People who have been following my series of posts about Michael Mann know I’m trying to dumb things way down. Today, I’ve hit rock bottom. I don’t think I can dumb things down any further.

My last post in this series discussed the fact Mann et al had 22 proxies that extended back to 1400. It also mentioned three of these proxies were created via a process called principal component analysis (PCA).

Don’t worry. Your eyes don’t need to glaze over. Many people have spent a great deal of time arguing about PCA, but I’m not going to discuss it. In fact, I’m not going to discuss any math today. What I’m going to do is much simpler. I’m going to show you Mann’s 22 proxies. That’s all.

We’ve discussed how Michael Mann did testing which showed him two proxies were vital to his hockey stick. Can you tell which they are?

1400_1

1400_2

You can see a list of those proxies here. You can see the data for those proxies here.


So statistical calculations scare you, you say? That’s okay. Math hurts your head? That’s fine. Phrases like principal component analysis make you fall asleep? No prob. Forget all of that.

Just look at those 22 graphs and realize, that’s Michael Mann’s hockey stick. Mix those images together, and you have a recipe for becoming a world famous climate scientist.


The truth is Michael Mann probably had no idea what he was doing when it came to the math of his paper. He was probably as frightened by the math as you are. That’s why this has never really been about the math. It’s always been about interpreting the results of the math.

Statistics are scary and math is confusing. Simple charts are neither.

Advertisements

34 comments

  1. Brandon: From one of your links I viewed the list of the twenty-two proxies and in the left hand column it shows WGT, which I assume means weighting? If so, then these were weighted on a range of 0.33 to 1.50 – is this correct? Would I also be correct in assuming that the rationale for this weighting was not disclosed, or am I becoming too cynical? Much thanks for the continuing series and the convenient links to the data.

  2. To think this information was supposedly reviewed by several science panels that found the no issues with Mann’s hockey stick.

  3. Which makes it even clearer – as you have said in an earlier post – that the first of the hockey-stick shaped proxies mentioned above is a “cheat”. That is, it doesn’t actually go back to 1400 at all, but to 1404, which is evidenced by the short, horizontal line segment at the start of the series.

    There is no mention of this in MBH98. In one part, it states:
    “The multiproxy network of 22 indicators available back to 1400…”
    http://tinyurl.com/l7lwtxu

    That series is, of course, the Gaspé cedar series. Which McIntyre discusses in the post linked below, about which he said (amongst other things):
    …in the early portion of the Gaspé chronology, there is only one tree – a point that was widely publicized back in 2005. Standard chronological methods require a minimum of 5 cores and preferably more. The early portion of the Gaspé chronology did not meet quality control standards.
    http://climateaudit.org/2008/03/19/tamino-and-the-adjusted-gaspe-data/

  4. Sundance, sadly, one thing I think McIntyre did wrong was focus on the technical issues so much. There was so much focus on PCA and statistical issues a lot of people missed the point. Even if someone believed Michael Mann’s use of PCA was fine and dandy, there’s still no denying the hockey stick exists in only a tiny fraction of the data.

    I saw McIntyre make that point on numerous occasions, but it was always buried amongst discussions of issues most people would never read through. That’s a shame because even Mann and his supporters acknowledge where the the hockey stick originated. They don’t dispute what this post shows. They just say it’s okay the hockey stick exists in such a limited amount of data.

    I think a lot of people who didn’t pay much attention would have been more interested if they saw this set of graphs and realized everyone agrees they’re the foundation for the hockey stick. I think taking breaks from the technical issues to highlight this sort of thing in a direct manner from time to time would have been effective. Of course, I could be wrong about that.

    Anto, the extension of the Gaspe series is troubling, but I’m more baffled by the duplication of it. Using the same data twice is wrong, but why was it used differently each time? Why did Gaspe get used as a proxy and a series combined via PCA? If it could be used both ways, how did Mann choose which data he’d combine with PCA and which he wouldn’t?

  5. Why did Gaspe get used as a proxy and a series combined via PCA?

    Brandon – yes, that’s a mystery, isn’t it? The only thing I can think is that including it in the PC01 gave the series the sort after “shape”.

    However, I would imagine that Mann would justify it on the basis that the PCA is supposed to be of North American tree ring series and, seeing as Gaspe is a North American tree ring series, well it’s right to be included. Of course, I agree with you that you shouldn’t include it twice and, if you do, you should adjust the weighting of both the series to reduce their impact on the final result. Patently, that wasn’t done!

    [BTW, that very bottom-right hockey-shaped graph above is the North American tree ring PC series, isn’t it?]

  6. Anto, I doubt the duplication was known to Mann, much less done intentionally. Remember, they only extended Gaspe back to 1400 when it was used on its own. When it was used in the NOAMER network (via PCA), its original starting date (1404) was kept. That meant it didn’t get included until the 1450 step.

    As for your question, yeah. The bottom row has Gaspe on the left, NOAMER PC1 on the right. The other 20 series are posted in the order they’re listed in in the data set. I only pulled the two we’ve been discussing out so it’d be easier to compare them to the rest.

  7. I doubt the duplication was known to Mann, much less done intentionally.

    Mmmm, perhaps. However, Mann appears to have been perfectly able to exclude certain series from his NOAMER PCA when he wanted to. See this from Steve McIntyre:
    On the right is another network – Stahle.SWM AD1750, showing an opposite pattern. In this case, MBH retained nine PCs although only three are “significant” under the realclimate version of Rule N. This is also a relatively small network (located in southwestern U.S. and Mexico and inexplicably excluded from the NOAMER network with which it somewhat overlaps.)
    http://climateaudit.org/2008/03/14/mbh-pc-retention-rules/

    [That post, BTW, raises some very interesting issues regarding what Mann said they did, vs what the evidence appears to show they actually did, or did not, do.]

  8. Anto, that post may not be interesting in quite the way you think. It’s important to remember Mann et al did not say how they determined how many PCs to retain for the tree ring networks. They said they used that Preisendorfer rule for their modern temperature PCs. The idea Mann et al used the Preisendorfer rule for the tree ring PCs didn’t come about until years after the hockey stick had been published. That (plus the tests McIntyre describes) suggests Mann et al used some other, undisclosed, criteria for determining their choice of tree ring PCs to retain. If so, the later claims about having used Preisendorfer’s rule are false and should have been known to be false.

    One thing I find especially interesting about that is people claim all of the code and data have been released, yet here’s an example of an issue where no code was provided. The code released (under pressure) by Mann et al did not contain anything which explained their tree ring PC retention rates. That makes it remarkable so many people believe all data and code was provided. Another example of this is there was no code provided explaining how the error margins in MBH99 were calculated.

    (You may have already known all this. I wasn’t sure, and I figure it wouldn’t hurt to discuss it even if you did.)

  9. Personally, it seems extremely charitable to assume negligence rather than malfeasance with everything that has been done with MBH 98 and 99. It seems remarkable that a key series would be altered without disclosure…and without acknowledgement that the series at that most early reading actually contained only one core…it doesn’t have the standard 5 core minimum until decades later. And, why use that same series again?

    It seems wholy unreasonable that Mann would use “short centered PCA” instead of standard PCA, especially when the methods in the paper describe standard PCA. It certainly seems plausible that he tried standard PCA, found that it did not achieve the desired result, and looked at alternative approaches that would provide the desired outcome.

    Why not release data and code? He regularly seems to “protest too much”.

    Bruce

  10. Brandon,

    I think it’s interesting, because of what Mann is claiming and what Steyn, et. al. are defending. One charge is that the hockey-stick itself was a “fraudlent” graph. A separate imputation is said to be that he “molested and tortured data”. The first is what Steyn said, the second is what Simberg said. Simberg’s comments didn’t necessarily relate this to a particular paper, nor are they restricted to a specific timeframe. So, if it can be shown that he has done that anywhere along the line, it is significant.

    The context of the article by Simberg is about not just the origination, but the subsequent things done by Mann, et. al. to justify and defend their original work. In fact, Simberg talks about what Mann was doing, “…to keep the blade on his famous hockey-stick graph”.

    In the blog post at Climateaudit linked above, McIntyre (referring to one of his and McKitrick’s letters to Nature) says in the comments section:
    Mann et al. state that they use an “objective criterion” to decide how many principal component series to retain for each region and each calculation step. In the SI, they refer to consideration of both Preisendorfer’s Rule N and to a Scree Test but do not state their “objective” criterion. Preisendorfer’s Rule N describes simulations from white noise series. The Supplementary Information to Mann et al.’s second reply describes a simulation process based on red noise modeled with lag-one autocorrelation – a quite different procedure. Can you obtain a provide an exact and replicable description of the procedure used to decide the retention of principal component series?
    http://www.climateaudit.info/correspondence/nature.040810.htm

  11. Are the 2 proxies at the bottom real proxies? Or are they subjected to the splicing trick, entering measured (calculated) temparatures at the right end?

  12. Visually, I think that this post from Steve Mac is a pretty powerful visual representation of how skewed MBH’s sampling is:
    http://climateaudit.org/2007/11/24/another-eas8100-assignment/

    However, I think that even that presentation gives a very flattering interpretation of things, given that the Gaspe and Graybill series provide almost all of the hockey-stick shapes to the ultimate reconstructions. For a potential jury to understand just how much of the hockey-stick reconstruction of over 1,000 years of world temperatures relies on proxies from two particular sites, you need to see what it looks like in geographical format:

  13. Øyvind Davidsen, I believe you’re setting up a false dilemma, but those series definitely did not have instrumental data spliced onto them.

    Anto, that was a good post. I always thought it was a shame it got so little attention.

  14. Although I haven’t looked into the vaildity (from a dendro perspective) of the Gaspe series being a marginal treeline series (and its relevance, therefore, to being a temperature proxy), these images from McIntyre raise some serious questions about it being characterised as such. [Recall that its filename in Mann’s directory was “treeline.dat”.]
    http://climateaudit.org/2007/03/22/wilson-pisaric-and-gaspe/

    For the uninitiated, trees are a very controversial means by which to measure past temperatures. This is because their response to temperatures is measured by the width of the “rings” in their bark. As trees grow older, their trunks expand in a series of approximately concentric rings. The more favourable the growing conditions, the faster they will grow in a given season, hence the wider the rings will be for that particular season. In dendro-climate theory, warmer = more favourable growing conditions, therefore widths are wider; versus cooler = slower growth, therefore narrower ring width.

    Sensible people will see the problem with this immediately – namely, that lots of other things affect tree growth over the centuries, apart from hotter/colder. Water, for a start. Also, the tree might have started out as a seed in a nice bit of loamy soil, but over time its micro-environment was eroded (or vis-a-versa) – these would change its growth patterns. Those searching for temperature signals in tree rings therefore like to search for trees in regions that are:

    (a) Geographically stable, therefore not subject to severe erosion patterns over millennial timeframes;
    (b) Relatively consistent rainfall patterns (as far as can be determined), because all plants obviously grow faster when there is more rain vs droughts;
    (c) trees which are closer to snow/glacial treelines, because once a tree is covered in snow/ice, it more or less stops growing, therefore its tree rings will grow more slowly, compared with a similar tree a couple of hundred metres down the slope, which is not so frozen.

    This last point is the most important for this particular post, because the Gaspe series from MBH98 was in a file labeled, “treeline.dat”. This implies that those who were using it assumed that the temperature signal it was producing (as opposed to a rainfall, nutrient, etc. pattern) was because it was at some important kind of boundary between high growth/low growth limits, depending on how cold it was (and therefore where the treeline was).

    However, as McIntyre shows above, it’s a bit difficult to see how the St Anne’s River region, from whence the Gaspe series came, is any kind of globally significant spot which could reasonably be hailed as a multi-millennial weathervane for global temperatures.

    There are other issues, however that is probably plenty enough for one post. The summary is this:

    Gaspe is undoubtedly the most important series in MBH98. Without it, there is no hockeystick. Gaspe is said to be a temperature (as opposed to precipitation, fertilisation, etc.) proxy, because it is at some important “treeline”. If it is not at such a treeline, there is no reason to believe that its ring pattern is because of temperature. It could be, but if lots of other dendro evidence points to different conclusions, then you would naturally exclude Gaspe as an outlier, with some unknown, but spurious influence on it,

    Instead, Mann excluded all of the other consistent evidence and promoted Gaspe to the head of the class.

    Emblematic.

  15. I went looking for something on Climate Audit this morning… I had remembered something about issues with coring cedars and their propensity to have more oval or oddly shaped trunks. The result of that irregular ring growth would impact the original data. Instead, I stumbled onto several Gaspe related posts. Re-reading these summaries by Steve McIntyre is maddening. This just is not science.
    http://climateaudit.org/2005/02/09/the-updated-gaspe-series/

    In many other areas of science, if one tried to do what dendros have done, they would not only be fired, they would end up in prison. Imagine these types of methods-behaviors in a pharmaceutical study!

    Bruce

  16. Anto, it is wrong to say without Gaspe there is no hockey stick. As this post shows, there are two series with hockey stick shapes. Either one is enough on its own. You either need the (arbitrarily extended) Gaspe series, or you need NOAMER PC1. If you use NOAMER PC1, you’re relying upon ~20 tree ring series,* predominantly from bristlecones (which are known to be a bad choice for temperature proxies).

    Of course, (a non-extended version of) Gaspe was included in the North America tree ring network. You’re still talking about a tiny number of tree ring series from one area of the globe.

    *Mann removed 20, but if I remember right, only 14 of them actually mattered.

  17. Hi,
    Would it be a useful graphical representation of the PCA method to scale the areas of each of these graphs to indicate that the “Principle” graphs (with a big area) are weighted more heavily while other graphs are assigned tiny little areas?

  18. Brandon – yes, fair point. Without Gaspe and NOAMER PC1 there is no hockey-stick. And, without the bristlecones and Gaspe included in NOAMER PC1, there is no hockey-stick.

    However, despite the presence of the bristlecones, is it not true to say that the hockey-sticked shape of the NOAMER PC1 network only arises if you process that data in a particular way? M&M contend that MBH’s processing method was inappropriate.
    http://climateaudit.org/2006/11/05/juckes-noamer-pcs/

    If the NOAMER PC1 network is processed using covariance/centered, rather than correlation/standardized methodology, then only Gaspe remains hockey-shaped.

  19. Anto, Mann’s faulty implementation of PCA is what caused the NOAMER PC1 to have a hockey stick shape. However, Mann’s methodology cannot create a shape which doesn’t exist within the data. All it can do is give extra weight to specific shapes.

    There were ~20 series (plus Gaspe) which could be said to have a hockey stick shape. As long as you include them, you get a hockey stick. It doesn’t matter how you include them. You can include them as PC1 via Mann’s faulty methodology. You can include them as PC4 via a correct implementation of PCA. You can include them by not using PCA at all and simply averaging all the series together.

    Bristlecones wind up in PC1 if you use Mann’s faulty methodology. They wind up in PC4 if you use the methodology he claimed he used. The number after the PC tells you how important the signal is to that data (1 = most important, 4 = fourth most important). That means Mann’s methodology gave undue weight to bristlecones because they had a particular shape, but it didn’t create that shape out of nothing.

    All you really need to know is Mann’s faulty implementation of PCA is how he managed to give bristlecones a lot of weight. The details aren’t that important. It’s not like things would have been any better if he had just cherry-picked series to use on their own like he did with Gaspe.

  20. No prob.

    I’m actually trying to work up the motivation to finish my post on PCA. After this post, it kind of seems pointless. Or maybe it’s just that I’m feeling lazy today. I think that’s it.

  21. I applaud your ambition to make this complex, inherently mathematical analysis accessible and intuitively understandable to the layman in a non-mathematical way. IMHO you’ve succeeded to a remarkable degree.

    If I can make one suggestion – Even the basic notion of using proxies for temperature will not be intuitively obvious to the casual reader, reducing these graphs to meaningless squiggles. If you simply labeled each graph with a brief description of the proxy source ( i.e. “North America bristlecone Pines”), it would sharpen the focus on the distinction between what Mann represented his work to be against what it actually showed.

    In an odd way, it turns the “consensus” canard on it’s head. Of the 22 temperature proxies used, a 91% “consensus” show no “hockey stick” of warming in the industrial era.

  22. Brandon:
    Thanks for the memories. I have been following this and contributing on some sites since CA started.

  23. Dividist, I originally included names for each proxy, but I decided the names would be less comprehensible than whitespace. I think the better solution is just to have a primer which introduces basic concepts like what a proxy is. In that vein, my intention has always been to write introductory material for the list once I’ve finished it. These posts are so I can write a general overview while including links to discussions of specific issues.

    That’s been my plan, anyway. I don’t know if it’s a good one or not.

    By the way, I should stress this isn’t a matter of warming in the industrial era. There are 112 proxies (31 of which are proxies generated from PCA applied to 384 series) which cover the period of 1820-1980.* A number of them can be said to show warming in that period.

    The hockey stick has never been about the “blade.” The hockey stick is about the shaft. We know temperatures have risen in the last 200 years. What we don’t know is what they’ve done in the last 500+ years.

    *Well, a few stop a bit shy of 1980, but that doesn’t change anything.

  24. miked1947, I’m happy to hear that. I’m also happy to hear it sounds like I out-date you. I’ve been following this since before Climate Audit started!

    I was going to say that makes me feel old, but according to your username, I’m still a kid 😀

  25. Before CA I was following Junk Science in all its forms. I retired from a position of systems analyst and trouble shooter. That makes me a sceptic of many claims.

  26. Thanks for this Brandon. As much as I love Steve’s detailed articles, I struggled to understand the technical analysis of PCA. Your visual representation really helps the average layman like myself.

  27. I think if you actually implement Mann’s claimed RuleN you would not get the results he claimed you get. So he is lying about having used that rule.

  28. I don’t know that he is lying. He definitely did not use that rule for determining how many PCs to keep with the tree ring data. However, it’s worth remembering he didn’t claim to in his paper. He did, however, claim to use that rule for instrumental PCs. I believe that claim was true.

    I could believe Michael Mann convinced himself he used the rule for both cases when he really only used it for one. It’s a small detail about a methodology which didn’t come up until years after he wrote his code. I could see how that, combined with the fact he’s lazy, quick to jump to conclusions and doesn’t check his own work, could lead him to thinking he did something he didn’t do.

    Self-delusions can go a long way in explaining behavior.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s