Checking Work

It’s important to check your work. It helps prevent you from publishing stupid mistakes. It’s also important to check other people’s work. That helps prevent you from believing people’s stupid mistakes. Sadly, in the internet age of near-instantaneous communication, people often do neither.

Andy Skuce of Skeptical Science, who I follow on Twitter, recently retweeted this:

https://twitter.com/SouBundanga/status/455227855484383232

I like to keep abreast of who uses deceptive charts, so I clicked on the link to the Hot Whopper piece. I then scrolled down to the section, “How has Bob Tisdale fudged his chart?” In doing so, I saw the chart which was supposedly fudged:

Reynolds_Tisdale

I didn’t see anything out of place at first, but I read on. Sou (the site’s owner) claimed to plot the same data as Bob Tisdale and said:

Hmm. The chart I plotted from KNMI data is quite different to the one that Bob Tisdale constructed.

The side by side comparison Sou provided seemed especially damning:

Sou_Reynolds_Comparison

I decided to check for myself. I went to the same site and got the same data the two of them had supposedly plotted. Here’s a screen shot of my result:

Reynolds

This is exactly what Bob Tisdale showed. This reinforced the suspicions I got when I saw Sou first present results:

Sou_Tisdale_Replication

That image, like the image from Bob Tisdale, has something not present in the side-by-side comparison: information about what data was used. Sou’s graph says “24S to 24N and 80W to 120E.” Tisdale’s graph says “24S-24N, 120E-80W.” Notice the subtle difference. The order for the second range is inverted.

To confirm my suspicions, I went back to the KNMI site and swapped my longitude values from 120 and -80 to -80 and 120. It gave me this graph:

Reynolds_Sou

Which shows the same data Sou showed. That proves the difference between their results is simply the choice of longitude range. One person used the longitude range to get data for the Pacific Ocean. The other person used the longitude range to get data for everything but the Pacific Ocean.

In other words, one person made an incredibly boneheaded mistake, failed to catch it (if and) when they checked their work, and apparently at least some readers didn’t catch it when they saw the graphs.

Since this topic is about checking work, I’ll leave it to readers to check and see which person made the boneheaded mistake. Was it Sou, or was it Bob Tisdale?

Advertisements

19 comments

  1. Since this topic is about checking work, I’ll leave it to readers to check and see which person made the boneheaded mistake. Was it Sou, or was it Bob Tisdale?

    I normally don’t get involved in scientific questions like this but I having Google Earth I could fire it up and check – probably in olden days people just used to check their drinks globe 🙂

    If the idea is to cover the NS Tropic region of the Pacific, then yeah, I am going to go out in a limb and say it was Sou who has made the boneheaded mistake here 🙂

  2. If you think about it, Sou made a boneheaded mistake here no matter what. There are two possibilities: 1) Sou screwed up his longitude range; 2) Bob Tisdale screwed up his longitude range, and Sou failed to catch the obvious mistake.

    Anyway, one thing which made this issue interesting to me is there is no inherent way to tell which region one is selecting given two longitude values. One could just as easily go east-west as go west-east. Which is “correct” is merely a matter of convention. And like many conventions, it can be tricky to search for it on the internet. That means a person could understandably read both posts, look at maps and still not know who was right.

    I actually made sure I didn’t check to see who was right before finishing this post. I was confident I knew, but geography has always bored me. On most days, I couldn’t even tell you where the 0 value is for longitudes.

    It’s pretty cool when the point of a post is true regardless of which side discussed in it is right.

  3. Sou was alerted to her error by a commenter, and she’s deleted the section which said Bob Tisdale “fudged” his graph. I don’t like that sort of “correction.” If you make a mistake, you should correct the mistake, but you should also let people see the mistake. People deserve to be able to read what has been said.

    In any event, I archived a Google cache of Sou’s post for posterity’s sake, using the same service she used in her post. I think that’s reasonable. Also, I think it’s kind of important. Sou didn’t just make the comparison I discuss in this post. There were a couple other graphs, using four different data sets, which suffered from the same mistake. Plus there half a dozen paragraphs like:

    What Bob seems to have done somehow is shift up the earlier years and/or shifted down the later years. Even then he has odd peaks and troughs in his chart. As far as I know he’s using monthly data like I did, so I cannot explain the reason for the difference. Perhaps a reader can help out.

    SST does not show the same amount of warming as the mean of CMIP5, but it does show around 0.4 degrees of warming over the period, which is not “little” warming, at least not from where I sit. If he’d written instead that there was little warming in the past 12 years or so, I’d agree with him. But over 32 years the sea surface in that region of the Pacific Ocean has warmed perceptibly.

    But you wouldn’t know how much effort Sou put into this argument. Looking at her post now, you wouldn’t even know she felt it was important enough to include in her title (which she has now changed). Instead, all you’d see is:

    How has Bob Tisdale fudged his chart? Answer: He didn’t!
    Update: Lars Karlsson has pointed out that I mapped the opposite area of the ocean. So Bob didn’t fudge the chart at all.

    I should have been more careful – and apologies to Bob.

    I’ll leave this comment and placeholder here as a salutory lesson to myself!

    Sou: 7:50 pm AEDT

    Not only does deleting the original material make the update uninformative, it makes it impossible to tell what her mistake actually was. What “opposite area” is she talking about? We know because we saw what she did before the update, but could a reader tell now? I don’t think so. One may be able to fairly call everything but the Pacific Ocean “the opposite area of the ocean,” but Tisdale only looked at the tropical latitude band. One could easily interpret Sou’s update as indicating she selected the non-tropical portions of the Pacific Ocean.

  4. Yes, I made a boneheaded mistake. It wasn’t the first time and it won’t be the last 😦

    My apologies to Bob in case he reads this and doesn’t read HotWhopper (which I think he does, from time to time).

  5. One could just as easily go east-west as go west-east. Which is “correct” is merely a matter of convention.

    I instinctively used the Earth’s rotation, i.e. if area is delimited by two longitudes in order I would put my finger on the line of the first given longitude, and then let the globe turn in its natural direction on its axis until the next line arrived. Maybe a difference of a tactile/visual way of thinking?

    I’d be surprised if that isn’t the convention but I haven’t looked.

  6. Actually I made a boneheaded mistake and got the Earths rotation wrong there in my mind. I did that the other day when I though UK was ahead of Europe on the clock, so I’m actually following the increasing longitude number as the direction to follow.

  7. “Yes, I made a boneheaded mistake. It wasn’t the first time and it won’t be the last” – Sou

    Great.

    Maybe, there are mistakes in other criticism you have been offering on Bob’s work too? This was a silly one, you make a mistake and it remained because the conclusions arising from the mistake of longitude mix-up was just in line with your pre-conception that Bob Tisdale is “perennially mixed-up”. What about larger mistakes of judging his numerous posts for what they are? Maybe you’ve been making a series of mistakes.

  8. Sou, thanks for correcting your mistake, but you followed it with another. In your 7:46 comment…
    http://blog.hotwhopper.com/2014/04/anthony-watts-is-confused-about-enso.html?showComment=1397382414103#c2989299730283757154
    …you stated, “Oh dear. That explains it. It did seem odd because while Bob does fudge things sometimes, it’s not usually his straight charts.”

    I do not fudge things at any time. People have been verifying my graphs for years…using the correct coordinates. If I fudged things, I would have been drummed out of WUWT years ago.

    Adios, Sou.

  9. “…It did seem odd because while Bob does fudge things sometimes, it’s not usually his straight charts.”

    What a brave person. Can’t resist making passing shots even when in the wrong.

  10. Shub Niggurath says: “What a brave person. Can’t resist making passing shots even when in the wrong.”

    Maybe Sou likes to double down on a blackjack hand after she busts.

  11. I suspect that anyone who’s used KNMI Explorer and has had occasion to take data across the 180 E/W line has made this error once at least. The grid index increases W to E, so the first longitude entered is the West end of the region, which is not necessarily the lower value. One can enter a longitude range of 120-280 which looks odd but has the lower value first. [You could use -240 to -80 as well, I suppose, but that would just be twisted.]

  12. sou, everybody makes boneheaded mistakes at times. A risk of joining a public discussion is that you’ll make one where everyone can see.

    tlitb1t, I’d normally just assume it goes from left to right on a map. That’s the direction most things go. The problem is checking that. I spent a bit of time on Google trying to find a search query which would direct one to the convention. I didn’t find one. It was weird.

    Shub Niggurath, did I call her a he somewhere? I thought I made sure to type “she” each time, but pronouns and handles do sometimes cause mixups.

    Bob Tisdale, glad to. I thought it was hilarious, and I’ve long been bothered by people believing things too easily. This gave a good example for highlighting the problem.

    HaroldW, it’s an easy enough error to make. As I pointed out to tlitb1t, there’s no real indicator as to the right way to input values there. Mixing them up is understandable. However, it’s also quite obvious if you check your work. All you have to do is compare the two graphs. Sou spent a bit of time modifying images to line them up. Had she included those images’ headers (captions?) when doing so, she’d likely have caught the mistake

    Even if not, the first thing you ought to do when your results don’t match someone else’s is make sure you’re examining the same thing as they are.

  13. Brandon,

    A consistent problem I see with Andy Skuce and the SkS crew in general is their willingness to cast aside any hint of skepticism when presented with a claim that aligns with whatever they prefer to believe, but they will go to town on anything they disagree with. I have seen few better examples of confirmation bias.

    For example…. last summer, Andy Skuce reported on a study (here: http://www.skepticalscience.com/BCCarbonTax2.html) that attributed a ~19% reduction in fuel consumption to the introduction of a ~5% carbon tax in British Columbia. As implausible as it may sound, not a single person at SkS seemed to find any cause for skepticism. I pointed out in the comments that leakage might be a factor (i.e. BC residents buying fuel in cheaper jurisdictions). Well, that aroused their skepticism… but it was directed solely at me, rather than at the study.

    For added entertainment value, our friend Tom Curtis got in on the act making some pretty boneheaded mistakes of his own, which I think you’ll find amusing.

  14. Russ R., that was an interesting read. It’s good Tom Curtis figured out his mistakes, but it’s hilarious he made them in the first place. Making boneheaded mistakes is one thing. Nobody can avoid it. What we can avoid is calling people’s ideas BS while failing to perform simple checks on our conclusions.

    Sadly, this issue isn’t limited to one side of the global warming issue. It’s not even limited to the global warming issue. Confirmation bias is a blight that permeates pretty much everything. For an example from the other “side,” just look at the criticism of Cook et al from Richard Tol I’ve discussed on this site. He claimed finding patterns in sorted data showed Cook et al’s paper was flawed because random data shouldn’t have such patterns.

    As far as I know, only one person (Kenneth Fritsch) other than myself pointed out the absurdity of his argument. Most just seemed happy Tol was on their “side” of the argument. I found it baffling. Even people who could normally do serious analysis seemed unwilling to realize finding patterns in sorted data it completely expected. They didn’t even notice one pattern Tol complained about was actually a finding discussed in the paper he criticized.

    It’s worrying. Not only is it troubling for its implications on discussions in general, it makes me wonder if maybe I’m as guilty. Are there beliefs I hold when any real critical analysis of them would show them wrong? I don’t think so, but people have said so. Could I be fooling myself into not seeing their point?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s