Cherry Picking, Done Stupidly?

I wasn’t intending to write a post tonight, and I’ve had a bit to drink so maybe I’ve missed something. If not, Tamino has done some incredibly stupid cherry-picking.

I just stumbled across a post about a week ago becaues of a response I saw on Watts Up With That? Tamino’s post is made in reference to the notion:

Most who call themselves “skeptics” of global warming would probably say “No global warming since 1998!” Under the name “hiatus” or “pause,” it features prominently in public discussion and even in senate testimony (e.g. from Judith Curry).

To rebut this notion, he creates a series of graphs for various temperature data sets in the form of:

hadcrut4

He describes the image:

We’ll start with the HadCRUT4 data set from the Hadley Centre/Climate Research Unit in the U.K. Taking the data from 1979 through 1997, we’ll compute a linear regression line, then extrapolate that line through to 2013 to construct our “still-warming” prediction. We’ll also compute the standard deviation of the residuals from our linear regression so we can add two lines to the graph, one of which is two standard deviations above our forecast, the other two standard deviations below, in order to delineate the range in which we would expect most of the future data to be.

We’ll also take the final value of the linear regression line (not the slope) as our estimate of what we would expect if we had been given certain knowledge of no statistically significant warming from 1998 through 2013, and we’ll add extra lines, two standard deviations above and below, to mark out the expected range.

I don’t think this test is appropriate for the conclusions Tamino draws, but that’s not what I want to discuss today. What I want to discuss is a far simpler point. Tamino claims he is going to discuss the claim there has been “no global warming since 1998.” To do this, he sets his breakpoint at 1997.

Why? Why would you include data from 1998 if people say there hasn’t been warming since 1998? The phrase “since 1998” is clearly intended to mean, “We saw warming up through 1998 but not after 1998.” The entire point of people picking the year 1998 was it was anomalously warm. Obviously people aren’t wanting to include it in the period they say has had no warming.

I can’t think of a legitimate reason for Tamino’s decision. As far as I can tell, it’s simply idiotic cherry-picking – claiming to test one period by randomly using data from a different period. Maybe I’m missing something though.

Regardless, one might be curious how much of a difference this issue makes. As such, I’ve taken the liberty of making a (very) crude replication of the key elements of Tamino’s graph:

tamino_rep

There are some minor differences, but it clearly shows the same thing Tamino describes:

What actually happened is that, according to the HadCRUT4 data, most of the data are above both forecasts. Twelve of sixteen were hotter than expected even according to the still-warming prediction, and all sixteen were above the no-warming prediction

But what if we didn’t include data from 1998 when “testing” to see if there had been warming “since 1998”? Would we get the same results? No. Using the same process as the image above, I get:

tamino_fixed

In this image, a couple years are below the blue line, and about half the years are below the red line. It’s a dramatic change. It shows Tamino exaggerated his results by randomly choosing to include the hot 1998 in the period he tested to see if there had been no warming “since 1998.”

As far as I can tell, that’s just stupid cherry-picking. It’s stupid because he flat-out said what he was doing. Anyone reading his post can tell he isn’t testing the argument he claimed to be examining. It seems he basically said, “I’m going to examine argument X by testing argument Y.”

Maybe I’m missing something though. Tamino often says he knows a great deal about analyzing data, and he’s published scientific papers heavily involving linear regressions. Why should anyone care if I think Tamino used stupid cherry-picking to exaggerate the results of an already inappropriate test? I’m just a nobody.

Of course, I’m a nobody who understands simple phrases like “since 1998.” That’s one point in my favor.


February 7th, 7:17 AM Update: A post I wrote a few days ago is highly relevant to Tamino’s. As I pointed out there, you can get all sorts of graphs if you’re willing to cherry-pick things like endpoints. To demonstrate, I’ve created a third graph following Tamino’s method:

tamino_mock

The difference between this and his image is enormous. Why? Because I decided to make an arbitrary choice of my own. I began by making the non-arbitrary choice to test the claim there has been no warming since 1998, instead of Tamino’s since 1997. I then decided to make both sides of the break point equal in length. With those two changes, the results are dramatically different.

Are they any better than Tamino’s? No. Are they any worse? No. It’s true I used less data than Tamino did, but he could have used more data if he wanted. The HadCRUT4 data set goes farther back than 1979. The fact other data sets might not begin until 1979 doesn’t mean he can’t use the data we do have before 1979. And if we can arbitrarily exclude 50+ years of data, I can arbitrarily exclude five years of data.

What this shows is Tamino’s method is inappropriate, and it practically begs for cherry-picking.

Advertisements

17 comments

  1. To be clear, Tamino believes he is hot stuff when it comes to doing these sorts of analyses. In his latest post, he <a href="http://tamino.wordpress.com/2014/02/06/concern-trolls/"says:

    When I’ve presented detailed analysis of some data (like sea ice, for instance) based on decades of experience analyzing time series in the physical sciences (including inventing useful new methods), then you come here and disparage my conclusions based on your “view of the data” or “a few minutes looking at the data,” that is an insult. It insults me, it insults the intelligence of my readers, and it sure seems to be based on nothing more than your grossly inflated ego. The stupidity and arrogance with which the ignorant assert their nonsense, could well be considered a far worse insult than anything I’ve ever said about any of the fake “skeptics.”

    It didn’t even take me “a few minutes looking at the data” to conclude Tamino’s inclusion of data from 1998 was stupid. That was my immediate impression upon looking at his graph. It took me a little while to write this post and create the images in it, but otherwise, this was a 30 second deal. And that was without “decades of experience analyzing time series.”

    Apparently that makes this post an insult.

  2. Actually a very reasonable ‘cherry pick’ would be to take the average of the last ‘n’ years as the cut point. That way you get to draw the blue line right down the centre of the later years. I mean, how much fairer could you be but that 🙂

  3. That would make as much sense as what Tamino says. Consider his response to a reader:

    Our little “thought experiment” consisted of predicting what would have happened after 1998, given only the knowledge of what happened prior to 1998.

    In what world does “only the knowledge of what happened prior to 1998” include all the temperature data for 1998?

  4. What a waste of time reading this drivel! The figures themselves are subjective as readings are cherry picked to begin with to substantiate a particular point of view..

  5. As I regard all ‘Linear Trends’ = ‘Tangents to the curve’ = ‘Flat Earth’ I am not sure I can sensibly contribute to this anyway 🙂

  6. Tamino has been producing these meaningless cherry-picked graphs for years. I gave up reading his blog after I saw him doing this kind of thing back in 2008. In those days he was still insisting that global warming was accelerating, and fiddling graphs to try to show it.

    I only came across his latest nonsensical line-drawing exercise when some climate scientists on twitter described it as ‘interesting’ and ‘good’. Foxgoose immediately called it a cherry pick.

  7. “I’d say 90% of the time they have as much validity as claiming the earth is flat!”

    About the only real use for them is INSIDE the range on which they are drawn. Then you can draw some useful conclusions with them. OUTSIDE of that range, such as when they are used to imply increases into the future, then they are most definitely ‘Flat Earth’ thinking :-).

  8. Anyway, I prefer to use continuous functions such as Cascaded Triple Running Means and Savitzky-Golay filters rather than the discreet Linear Trends and LOWESS alternatives.

    An Engineers response to Climate Statisticians/Scientists.

  9. I think that should be “decades of experience statistically analyzing time series.”

    And you know what they say about stats……

  10. I never check his site, but I happened to see this image, and it seemed ridiculous. I couldn’t resist looking for myself. It doesn’t help I wrote a post about a similar issue just a day or two ago.

    Plus, I’ll always have a soft spot for Tamino. One of his cherry picking escapades is what led to me writing my first blog post ever (over at The Blackboard). Who knows? Maybe I’d have never become a blogger if not for him.

    (I guess I am a blogger now? I don’t know. How many posts do you have to write before you are one?)

  11. If I graph the height of my last christmas tree across time, I would see: an upward line until the time that the tree was cut; a sudden drop; and then a line with a slope of zero. Tamino would conclude that my tree is still growing.

  12. Ironically I am somewhat reminded of an analysis done by Pat Michaels a few years ago. He simply took the model from:

    Michaels, Patrick J., and Paul C. Knappenberger. “Natural signals in the MSU lower tropospheric temperature record.” Geophysical research letters 27.18 (2000): 2905-2908.

    And extended it under the assumption that from 1998 on, the trend term had flattened out, and then under the assumption that the trend continued. The coefficients used to derive the fit were unchanged. the model looking better with a continuing trend than with one that completely halted.

    But I should mention that Troy Masters has done some interesting work suggesting that doing this sort of thing (creating a linear regression model to remove “solar” volcanic and ENSO signals) will fail to catch a slow down even if we *know* one is present.

    Also Steve_F at lucia’s blog has used more sophisticated methods that find there is, in fact, a recent change in the rate to slower warming, although not a *stop*.

    Personally I find neither of these satisfactory because I don’t agree philosophically with the notion that ENSO needs to be “removed” from the climate record.

  13. Brandon,
    While 1997 (rather than 1998) is a cherry-pick, I think the more serious issue is the presumption that the OLS fit to the trend is a good predictor. It’s appropriate when the underlying process is one of linear growth plus white noise. But is that a good characterization of the surface temperature series?

    Here’s an experiment which I mentioned in a comment at Bishop Hill:

    Take an ideal (noiseless) plateau-ing temperature history of the sort mooted by Tamino: set the anomaly to 0 in 1977, increase it by 0.01 each year through 1997 (when it reaches 0.2), and then hold steady at 0.2 thereafter. [I started in 1977 only to make the 1997 temperature come out to a nice even figure.] Compute the trend from 1979 to 1997 — you should get 0.01 K/yr slope, and it will lie over the ramping part of the history. The post-1997 values all lie below the extension of this trend line. Just as Tamino writes.

    Now set the temperatures in years 1992 through 1994 (inclusive) to 0, as if something exogenous such as a volcano temporarily depressed temperatures. [Note that in this idealization, there is no long-term effect; temperatures bounce right back to the original trajectory.] Temperatures in 1995 through 1997 are 0.18, 0.19, 0.20 respectively, and the post-1997 temperatures remain at 0.2, The plateau is still present. But what is the new trend line, and where are the plateau years relative to this trend line?

    The charts are here and here. The event reduces both the slope and the intercept of the trendline. Easy to see when there is no noise, harder to discern in the real series.

  14. HaroldW, I agree. As I pointed out in this post, I don’t think this methodology is appropriate. I just think cherry-picking is a more effective accusation than using a bad methodology. Lots of eyes glaze over once you start talking about methodologies, but everyone can understand the simple point, “Tamino intentionally picked those endpoints which made his results look best.”

    That said, the issue you raise is more damning. OLS regressions cannot sensibly be used the way Tamino is using them. Anyone with any experience analyzing time series would know this. A person with “decades of experience analyzing time series” certainly would. Tamino has to know his methodology is utter nonsense, even if he won’t admit it to himself.

    Incidentally, the issue you raise is highly related to the issue of cherry-picking. The entire reason cherry-picking like Tamino’s can work so well is using OLS regressions like this is inappropriate

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s