Mann’s Screw Up #9.1 – Nonsense Data

The last post in this series showed Michael Mann’s 2008 paper depended entirely upon the use of two things: tree ring data and the Tiljander series. It also showed those Tiljander series were suspect. Today I’ll show they are worse than suspect. Michael Mann’s use of them was absurd.

The two methodologies Mann used to create temperature reconstructions (CPS and EIV) both require data be calibrated against modern temperatures. The idea is simple. If a series doesn’t track temperatures in modern times, we can’t expect it to track temperatures in the past. The problem is the Tiljander series could not possibly be calibrated against modern temperatures. The lead author of the paper explicitly states:

Since the early 18th century, the sedimentation has clearly been affected by increased human impact and therefore not useful for paleoclimate research.

If the data since the early 18th century isn’t useful for paleoclimate research, one cannot justify calibrating the series to the modern temperature record for the 19th and 20th centuries. Mann didn’t contradict the authors’ when they said the modern portion of the Tiljander series were contaminated, but we saw in the last post he was aware of their concerns.

Mann knew the Tiljander series were contaminated in a way that made it impossible to calibrate the series to the modern temperature record, yet he still used them in methodologies which required the series be calibrated to the modern temperature record. It should come as no surprise this led to nonsensical results.

There are four Tiljander series: Lightsum, Darksum, Thickness and XRD (X-ray Density). Well, there aren’t really four. You see, Thickness and XRD are measured. The thickness of a sample is easily measured (such as with a caliper). An x-ray is then taken. Like in all x-rays, there is some light and some darkness in the result. The relationship between them gives a “density,” which is just the ratio of light to dark.

Lightsum and Darksum are just the total amounts of light and darkness. That’s easy to calculate if you know the proportions of light and dark (given by XRD), and you know the total thickness. You just multiply. If the thickness was 10, and the ratio is 60% light/40% dark, you’d have 6 light, 4 dark. (Actual formulas for the process can be found here.)

Of course, multiplying Thickness and XRD doesn’t magically give you twice as much data. It doesn’t give you twice as much information. You can’t just modify a series over and over and reuse it each time. Mann was wrong to use all four series.

But whatever. Let’s get back to the nonsensical results Mann got by deciding to calibrate proxies that didn’t reflect temperatures in the modern period to the modern temperature record. A question you should ask yourself about these series is, “What do higher values mean?” Instinctively, you might think a higher value means higher temperatures. However, if we’re measuring something like the amount of ice, higher temperatures might lead to lower values. According to Mia Tiljander:

High X-ray density corresponds to high amount of mineral matter (light grey value tints in X-ray film) and low X-ray density corresponds to
dark grey values caused by a higher proportion of organic matter.

High XRD means high Lightsum. Low XRD means high Darksum. As you can see, Lightsum and Darksum must have opposite patterns. High values in one must indicate low values in the other. That means if high values in one indicate warmer temperatures, high values in the other must indicate cooler temperatures. And no matter what, XRD and Lightsum must indicate the same thing.

Additionally, high amounts of mineral matter mean high Lightsum and XRD values. High amounts of organic material mean high Darksum values. With that in mind, we can continuing reading what Tiljander says:

The layer above the mineral matter is defined as organic, because of the less dense structure in X-ray images and low grey-scale values. A thick organic lamina probably indicates a warm summer and a relatively long growing season.

Tiljander says thicker organic layers (lamina) indicate higher temperatures. That means higher Darksum values indicate warmer temperatures while higher Lightsum/XRD values indicate lower temperatures. Thickness won’t have any coherent climatic interpretation since its the sum of Lightsum and Darksum.

Now then, suppose we tried to calibrate these series to the modern temperature record. We’d expect Darksum to have a positive correlation, Lightsum and XRD to have a negative correlation and Thickness to have no correlation. Mann found otherwise. The correlations he got when using his CPS methodology were:

Darksum:	0.3066
Lightsum: 	0.2714
Thickness: 	0.2987
XRD: 		0.1232

All four correlation coefficients are positive. That means Mann concluded higher values indicate higher temperatures for all four series. He found a “temperature signal” in a series with no climatic information (Thickness), and he found “temperature signals” the opposite of what we’d expect in two series (Lightsum and XRD).

How’d that happen? Simple. All four series had high values in recent times because they had been “affected by increased human impact.” That human impact caused recent values in all four series to be abnormally high. Mann took that as indicating all four series show warming. This shows when you use data that is “not useful for paleoclimate research,” you get nonsensical results.

But then there’s the EIV methodology. It’s more complicated, but you should remember from our discussion of Mann’s original hockey stick that temperature reconstructions are often done in “pieces.” Different data is available over different periods so they do calculations over different periods and stitch the results together. Let’s look at how the Tiljander series were used in two different periods. In the period beginning at 500 AD (graphs courtesy of Climate Audit regular UC):

ad500

We see all four show the same relationship as the CPS methodology – higher values indicate higher temperatures. That’s obviously wrong. However, there’s something stranger. This is the same image for the period beginning at 600 AD:

ad600

The XRD series is now flipped over. For this period, higher XRD values indicate lower temperatures. Not only does Mann use the data opposite its physical interpretation, he uses it opposite his own interpretation.

As I said, nonsensical results.

Advertisements

12 comments

  1. The question for a jury seems to be, “Does nonsense equate to fraud?” Where does the burden of proof lie in this dispute? Is it on Mann to prove he did his (let’s stipulate, just for argument) idiotic analysis in good faith, and had no way to know and no obligation that he should have known that he was an idiot? Or is the burden on Steyn (et al) to prove that Mann knew his method was nonsense, but published his analysis and results with reckless and perhaps malicious intent to deceive, perhaps for the fraudulent purpose of covering-up a previous fraud in MBH98?

    Or is it entirely sufficient to show that the RESULT is bogus? Steyn used the expression “the man behind the fraudulent climate change hockey stick graph” … if it can be shown that the GRAPH — or one of the latest and greatest versions thereof — is so badly flawed as to be unreliable for the intended purpose of directing public policy, and that Steyn reasonably had become aware of such flaws, then is there any point at all to determining whether or not the “man behind the graph”, Michael Mann, intended deception?

    The sad state of the stay on discovery keeps Steyn and others from asking for the peer-reviewers’ comments on Mann’s papers. One does wonder if any of the so-called experts in this field actually checked Mann’s references, or even asked any questions at all, and if so how Mann responded.

  2. Pouncer, none of those are actually necessary. All Mark Steyn has to do is prove he had some basis for believing what he said. That said, the more Steyn can show, the more compelling his case will be. A jury should find in his favor if it feels the graph could understandably be thought to be fraudulent, but it will definitely find in his favor if it feels Michael Mann committed fraud (in the common sense usage, not legal) when making the graph.

    This is a common situation in court cases. You present evidence for a bolder position (Mann committed fraud) to show it’s understandable, thus forcing the jury to accept a weaker position must be reasonable. In other words, “You may not agree Michael Mann committed fraud. I’ve shown evidence which suggests he has, but people can disagree about how convincing it is. However, if people can disagree about whether or not Mann committed fraud, it’s obviously understandable some people will think his results were fraudulent.”

    Of course, that’s just discussing the legal situation. Even if Steyn doesn’t need to argue Mann knew his conclusions were shaky (if not worse), he may still want to. Court cases aren’t just about verdicts. They’re also about influencing public opinion. Steyn winning the case may not be as appealing an outcome as him winning the case and convincing members of the public Mann is dishonest.

  3. In my opinion, the best way to think about the Tiljander data series is to conceive of them as numerical indications of physical entities. What are those things? The varves are annual compilations of particles that settled onto the Lake Korttajarvi lakebed, at the one spot that was cored.

    The authors of Tiljander03 characterized those particles in a simple way (later articles offer more complex interpretations). For each year, they attempt to answer, “How much mineral material settled to the bottom?” and “How much organic material settled to the bottom?” If you accept their methods, these correspond to Lightsum and Darksum, respectively.

    Pre-1720, Tiljander03’s authors suggested that a cold, snowy winter led to a deep snowpack and a vigorous spring runoff, bringing more mineral silt into the lake. A warm, less-snowy winter would cause a thin snowpack and a modest spring runoff, with less mineral material settling to the lakebed. So, colder and snowier winters led to higher lightsum values.

    Tiljander03 authors similarly proposed that higher darksum values correspond to warmer and wetter summers, pre-1720.

    My own view is that Mia Tiljander was likely mostly wrong in claiming that lightsum, darksum, and XRD are good proxies for temperature prior to 1720. Note that the Little Ice Age is clearly visible in the profile of Chironomid fossils from nearby Lake Hamptrask (Link at “Regarding another question”). No clear-cut changes in any of the Tiljander data series are evident.

    Instead, the Tiljander series may correlate to precipitation patterns. I suspect that they also mark disruptions such as forest fires and blow-downs from windstorms — anything that would make it easier for soil particles to be mobilized by water, make their way into streams, and thus end up on the lake bottom.

  4. IMHO arguments over the truth and validity of the hockey stick are unnecessary.

    Even if the method Mann used had been valid and his conclusion had been true, his paper is still fraudulent in this sense: it’s not replicable in the way science demands. It doesn’t provide “enabling detail” to a researcher seeking to retrace Mann’s steps. It took 5 years for someone (an amateur, as it turned out) to pry all the missing information from Mann and fill in the blanks.

    So the paper is not a work of science.

    Mann passed it off as one.

    That is fraudulent.

    Not in the legal sense, like a Nigerian email scam, but in the intellectual sense. Specifically: pseudoscience.

    ——————————————————————

    Brandon, I need a favor: please tell me if the above argument is sound, IYHO.

    (Nobody’s ever disputed it, but that doesn’t mean I haven’t overlooked some flaw.)

  5. I fixed two typos amac78 alerted me of privately. I’d leave a record for most alterations, but it’s too much trouble when switching one or two letters in a word.

    On the issue of the Tiljander series, I agree with you amac78. They don’t seem that good of proxies to me, and there are potential issues with them. Sadly, that’s true of most proxies which go back that far. In fact, compared to some proxies popular in temperature reconstructions, the Tiljander series are great proxies. I guess it’s easy to look good when people’s choices are between “bad” and “not good.”

    Brad Keyes, that argument could possibly work for Michael Mann’s original hockey stick. He failed to disclose tons of details about what he did, including a number of essential ones. It doesn’t work for the 2008 hockey stick though. Mann did a pretty good job of explaining what he did for it. The trick is just wrapping your head around what he said (or figuring out his horribly written code).

    I wouldn’t rely on an argument like that alone though. While it may be true in theory, most people won’t find it very convincing. Most people aren’t going to say scientists commit fraud when they don’t say what they did. There are too many explanations for why that might happen (e.g. word counts and laziness). That doesn’t mean it’s wrong. It does, however, mean it is unlikely that’s what Mark Steyn meant. If he tries to say it was, people aren’t going to believe him.

    I wouldn’t use that argument to cry “fraud.” I’d just use it to say Mann’s work was “bad science.” I’d then build upon that to cry fraud. There’s a big difference between these two arguments:

    “He didn’t show his work. Fraud!”
    “He didn’t show his work which proved his results were bogus. Fraud!”

  6. >(or figuring out his horribly written code)

    That’s what Martin Vermeer said, until he realized the author was Mann and not McIntyre at which point he said it was actually well written.
    It doesn’t seem that bad to me.

  7. Brandon,

    Thanks for that favor!

    (I see the limitations of my arg now, thank you. Mark Steyn still liked it enough to Retweet so I’m happy.)

    I’d say something closer to:

    ““He refused for 5 years to show his working, which means his paper wasn’t science. Yet he passed it off as science 6 times in the IPCC report. Fraud.”

    (Note the punctuation—shouting “fraud” just plays into their denier caricatures, so I always try to say “fraud” at a normal speaking volume.)

    😀

  8. No prob. I like looking at potential arguments. I’m not sure about your new one though. That’s accusing Michael Mann of fraud, but the phrase used was “fraudulent hockey stick.” A person could respond, “Even if Mann committed fraud by not showing his work, the hockey stick itself wasn’t fraudulent.”

    (I prefer to lower my voice, take a dramatic pause then say the word slowly.)

  9. Hi Brandon!

    Sorry, miscommunication: I meant “that’s what I’d argue if I were me,” not “that’s what Mark Steyn would argue if I were him.”

    As you suggest, it’s tricky (and unethical) for him to “sell” the jury any argument he himself wasn’t sold on when he wrote the words he wrote.

    B

  10. Well, the methodology being poor is a scientific weakness not a coding one. Unnecessary steps is a weakness in programming ability or perhaps algorithm design, but I wouldn’t call it poor coding. I’m looking at, can I use and understand the code. I was able to look at it, and figure out that Tiljander was used upside-down pretty easily. Looks to be decently commented. I would guess my own code is poorer than Mann’s, though probably more efficient.

  11. MikeN, I don’t agree a methodology “being poor is a scientific weakness not a coding one.” I think it can be either or even both. Statistical issues, like the weakness of CPS due to its variance deflation (see screening fallacy), are scientific. Smoothing every step of a reconstruction so you’ve smoothed some data 10+ times seems more like a coding issue of someone who doesn’t know how to do what he wants to do. I struggle to see how we’d call that a scientific issue. I guess we could say he implemented a procedure incorrectly (and in a way which makes no scientific sense)?

    In any event, I think it’s reasonable to refer to it as Mann’s “horribly written code.” He did things like create multidimensional arrays so large my machine with four gigabytes of RAM crashes when an efficiently coded version needs less than 50MB of RAM to run. Even if you can understand what he did in his code,* it’s definitely written badly.

    *I can’t recall if anyone ever finished figuring out the code for Mann’s EIV reconstruction. Do you know?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s