Mann’s Screw Up #9 – Overstated Conclusions

After some debate, I’ve decided to jump this series ahead to Michael Mann’s 2008 paper. This means skipping a lot of material, but it should keep us better focused on the big picture.

We’ll begin examining this paper by looking at its central conclusion:

Recent warmth appears anomalous for at least the past 1,300 years whether or not tree-ring data are used. If tree-ring data are used, the conclusion can be extended to at least the past 1,700 years, but with additional strong caveats.

You’ll recall Mann’s original results were dependent entirely upon a small amount of tree ring data (which was inappropriate for a temperature reconstruction). Mann placed this new conclusion, that tree ring data wasn’t necessary, front and center to address that issue. It was incredibly important as a way of rebutting criticisms of his work.

It was also greatly overstated. Michael Mann’s colleague and friend, Gavin Schmidt, said at their mutual blog:

Since the no-dendro CPS version only validates until 1500 AD (Mann et al (2008) ), it is hardly likely that the no-dendro/no-Tilj CPS version will validate any further back, so criticising how bad the 1000 AD network is using CPS is hardly germane. Note too that while the EIV no-dendro version does validate to 1000 AD, the no-dendro/no-Tilj only works going back to 1500 AD (Mann et al, 2009, SI).

CPS and EIV refer to two different methodologies Mann used in his 2008 paper. The details aren’t important. One (CPS) isn’t valid prior to 1500 without tree ring data (dendro). The other isn’t valid prior to 1500 without tree ring data or “Tilj.”

But this isn’t just a matter of whether or not Mann’s reconstruction validates. We also have to consider what happens to the shape of Mann’s reconstruction when we remove Tilj. Fortunately, Mann provided the answer to that. Before we look at it though, we should check just what “Tilj” is. According to Mann’s Supplementary Material:

In addition to checking whether or not potential problems specific to tree-ring data have any significant impact on our reconstructions in earlier centuries (see Fig. S7), we also examined whether or not potential problems noted for several records (see Dataset S1 for details) might compromise the reconstructions. These records include the four Tijander et al. (12) series used (see Fig. S9) for which the original authors note that human effects over the past few centuries unrelated to climate might impact records (the original paper states ‘‘Natural variability in the sediment record was disrupted by increased human impact in the catchment area at A.D. 1720.’’ and later, ‘‘In the case of Lake Korttajarvi it is a demanding task to calibrate the physical varve data we have collected against meteorological data, because human impacts have distorted the natural signal to varying extents’’)…. The Tijander et al. series constitute 4 of the 15 available Northern Hemisphere records before that point.

In other words, Mann used four Tiljander (note the correct spelling) series the originators of the data explicitly state were impacted by human influences beginning around 1720. Mann even recognizes these series are suspect, going so far as to test the sensitivity of his results to their removal. He removed them (and three other irrelevant series) and got:

Mann_2008_Fig8-1

Which shows his reconstruction is remarkably similar regardless of whether or not he includes the Tiljander series. Only, those graphs included tree ring data. He told us he didn’t need tree ring data for his reconstruction. Why would he test the effect of removing the Tiljander series, test the effect of removing tree ring data, but not test the removal of both? One might be inclined to think the reason is if he had, he’s have gotten this result:

Mann_2008_Fig8-2

It’s immediately obvious the green line (no-dendro/no-Tilj) is significantly different from the rest. The reconstruction from 1700-1900 is markedly different, and if you look closely, you can see the dashed green line goes way higher ~1300 AD than it does in more recent times.

That shows Mann can get his results if he includes the tree ring data which was called into question in the past. It shows he can get his results if he includes the highly suspect Tiljander series. Most importantly, it shows he can’t get his results if he excludes both.

It’s difficult to imagine how Mann could have not known this. Regardless, if he only found out later, he was surely obligated to publish an addendum to his 2008 paper warning people. Readers would certainly want to know his results depend upon either using the data he was previously criticized for using (bristlecones) or using data he admits is suspect (Tiljander). He never did that.

Instead, after this issue was repeatedly pointed out, Mann confirmed what Gavin Schmidt said in his Supplementary Material… for a different paper. He published the graph above deep in the Supplementary Material for a paper he published in 2009, where nobody reading the 2008 paper could possibly find it. He then said:

Additional basic tests were performed to evaluate the robustness of key features of the reconstruction with respect to data used. This includes a similar test to that shown in ref. S1 investigating the robustness of the RegEM EIV reconstruction of northern hemisphere mean temperature to the exclusion of particular types of data. In addition to the tests described by ref. S1 which removed alternatively (a) all tree-ring data or (b) 7 additional long-term proxy records associated with greater uncertainties or potential documented biases (showing the temperature reconstruction was robust to the removal of either of these datasets), we here removed both data sets simultaneously from the predictor network (Fig. S8). This additional test reveals that with the resulting extremely sparse proxy network in earlier centuries, a skillful reconstruction is no longer possible prior to AD 1500.

This acknoweldges what Gaving Schmidt said, what his critics said from the start: the only reason Mann’s 2008 reconstruction doesn’t require tree ring data is it uses highly suspect series (Tiljander). Without the Tiljander series, he reconstruction suffers from the same problem as his original hockey stick: relying upon a small amount of tree ring data.

In other words, Mann’s central claim is only true if he uses data where human influences “distorted the natural signal.” And he’d have known this if he performed “basic tests.”

Advertisements

7 comments

  1. Yea – when I read about this at CA I was amazed that the correction for the 2008 paper was in the SI of the 2009 paper. Amazing…

  2. David Jay, I’m guessing the reason it was allowed to appear in the 2009 SI is they don’t consider it a “correction.” In a sense, they’re right. The paper describes their conclusions accurately. They just don’t disclose the importance of the Tiljander series. That means the “correction” is really just additional information.

    The problem, of course, is the Tiljander series should never have been used. Michael Mann won’t acknowledge that, and nobody seems inclined to force him to. They’re perfectly willing to stand by his absurd and unjustifiable use of the series, to the point where nobody cares that he defended their use with a false statement, published in peer reviewed literature. And because nobody in the community will challenge him on a basic, factual issue he got wrong, he can use those Tiljander series as a fig leaf to hide behind.

    By the way, the next post will be discussing the reasons those Tiljander series shouldn’t have been used.

  3. A key finding of Mann08 was that treering proxies and non-treering proxies give similar pictures of the Earth’s temperature history, AD 500 – present!

    Only, as you say, Figure S8 of a different paper implies that they don’t. So this is a very important point. At least to the extent that the findings of articles in high-impact peer-reviewed journals should be supported by data.

    The graphics of Figure S8 of Mann09 are done in such a way as to “bury the lede” — its visuals make it hard for the reader to grasp the main point.

    I traced the nearly-invisible dotted Green line (no-dendro/no-Tilj) prior to 1500, so that it could be directly compared to the Blue line (no-dendro/yes-Tilj). That’s the essential thing to look at.

    See the third figure in this post. It becomes glaringly obvious that the two reconstructions are very, very different.

    Since Tiljander proxies cannot be calibrated to the instrumental record, the Green line is the only non-treering-based reconstruction that Mann08 and Mann09 could meaningfully compare to treering-based reconstructions.

    As you say, they are entirely different — undermining the paper’s claim.

  4. amac78, I had actually gone back and forth on which images I wanted to use for this post. I had thought about using the one in your link because it is so much clearer, but I’ve been trying to rely upon Michael Mann himself as a source as much as possible. In retrospect, I should have posted both and pointed out the visual difference.

    Hindsight is 20/20.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s