Richard Tol and Wild Imaginations

Richard Tol recently claimed the Guardian published what amount to “wild imaginations” and “fibs” regarding his work in an article by John Abraham. He claims to correct a number of these. I’ll investigate his supposed corrections.

The first claimed correction begins in the third paragraph of Tol’s piece:

Professor Abraham repeats Mr Ward’s claim that errors in my 2009 paper in the Journal of Economic Perpectives would have a major impact on its conclusion. I had informed Abraham by email on March 21, 9 days before publication, that this is not the case.

This is what Abraham actually said regarding the 2009 paper:

When you put all of these mistakes together, there was only one of Tol’s studies that showed a significant positive impact from climate change. That study, published in 2002 suggested a 2.3% effect on global wealth with a 1°C temperature rise. Even there however, Tol admitted that many impacts of climate change were omitted which will change the results. When I challenged Professor Tol on this point, he identified a second paper which showed a very small potential positive economic impact of 0.1% as climate change progresses, which I think we can agree is not significant.

You’ll note this paragraph does not say a word about the conclusions of the 2009 paper. Neither did the quote Abraham provided from Bob Ward. Both discussed the data used by the paper, but neither discussed how what they said about the data affected the conclusions. Tol’s “correction” is responding to something that wasn’t said.

Tol continues:

There are 18 peer-reviewed studies of the total economic impact of climate change, and 21 estimates. Two of these estimates show net benefits; 18 show net losses. Professor Abraham dismisses one of the positive estimates with a “which I think we can agree is not significant”. That is not the way significance tests are done.

There were 14 data points in Table 1 of the paper being discussed. The numbers Tol refers to are from more recent work of his, the relevant table of which I’ve shown in an earlier post (link to table here). It’s difficult to see why Tol would discuss his 2009 paper then suddenly, and without any indication, shift to discussing things not in his 2009 paper.

It also doesn’t matter. The shift doesn’t change anything. A more relevant issue is Tol says, “That is not the way significance tests are done.” Abraham never said anything about statistical significance so the issue of significance tests is a red herring. All Abraham said is a “positive economic impact of 0.1%… is not significant.” That’s trivially true. Even if the result somehow passed significance tests (I find it difficult to see how it would), the result is not significant in the way any reader of Abraham’s article would interpret the word. A benefit of 0.1% is unimportant.

Additionally, Tol tacitly concedes factual criticisms of his 2009 paper. His 2009 paper listed four peer-reviewed papers has showing positive impacts from global warming. Bob Ward and John Abraham pointed out two of these were incorrectly used by Tol as they actually showed negative impacts.

The next paragraph says:

Professor Abraham alleges that I refuse to correct errors. This is simply false. I alerted Abraham to this falsehood on March 21. I am not sure why he chose to repeat it on March 30.

While I won’t comment as to who is right on the specific examples being referred to (as I wasn’t able to tell from a quick review), I feel I should point out the argument is true as a general rule. I’ve discussed examples before, such as in this old argument, but a more relevant example is from a piece Tol wrote addressing the same arguments he’s addressing now. In it, he labels Bob Ward’s criticisms of him “fantastical claims.” Of particular interest is:

“there was only one study that showed significant positive effects from global warming”
Two estimates show net positive effects. Mr Ward was aware of this in October 2013. I am not aware of any claim to significance, which is a problematic concept in forecasting.

Tol made the words “one” and “significant” in the quote red for emphasis. This shows he repeats the same arguments even when it’s clear they’re straw men. At this point, nobody could possibly take Abraham comment saying 0.1% is not significant as referring to statistical significance instead of “significant” as used in the common parlance.

In any event, Tol moves onto a topic recently discussed here:

Professor Abraham alleges that material was added to the Final Government Draft (FGD) of IPCC WG2 AR5 that “had not appeared in previous drafts of the report”. In fact, material was moved from one chapter (19) to the another (10), and revised in response to comments. Again, Abraham was informed of these facts on March 21.

As I’ve discussed before, it is inappropriate to say the “material was moved” in reference to the Aggregate Impacts section being referred to. The material Tol claims was moved has no relation to the section appearing in the Final Government Draft. The only thing the two sections have in common are their names.

Additionally, there is absolutely no evidence any comments called for such a rewrite. I examined all available evidence and found nothing to support Tol’s claim. I posted it online so others could examine it for themselves to try to find support for his claim. Nobody found any. I even asked Tol himself for clarification. He repeatedly refused.

Moving on:

Professor Abraham also claims to that the FGD relies heavily on my work. In fact, the previous draft cited 7 studies, including one by me. The final draft cites 18 studies, including two by me. Two out of 18 is smaller than one out of seven.

This is remarkably false. The previous draft of the Aggregate Impacts section cites several dozen studies on a number of different subjects in its text. The final draft cites six studies in its text. Neither matches the numbers given by Tol. It would be difficult to figure out his claim save I’ve seen him cite those values before:

Only there he refers to previous, Second Order Draft section’s figure. When responding to Abraham, he leaves that qualifier off. By doing so, he makes it sound as though he’s referring to the sections in their entirety while he’s actually just referring to the change in what figure was shown with them.

But there’s more. While Tol says the “final draft cites 18 studies,” he covers up the fact the figure and table added in the final draft were taken directly from his own work. It’s true only ~11% of the studies cited in the figure (and table) were his, but 100% of the figure (and table) was his. Also, eight entries had a footnote attached which said:

Results aggregated by (Tol 2013)

Meaning their listed values were created to some extent by Tol himself. That means 10 of the 18 entries in a figure (and table) created by Tol were at least partially created by Tol. And that’s without looking at the text of the section. I’ve previously shown the central conclusion of the section was entirely dependent upon Tol’s own work.

Finally, Tol says:

Finally, Professor Abraham repeats Mr Ward’s lament about lack of transparency. In fact, all data are in the public domain. It does take a bit of multiplication, addition and division to go from disaggregate data to aggregate data, but that should not be beyond someone with a bachelor’s degree in geology.

This claim immediately begins misleading the reader by imply Ward and Abraham have said data was not available. In actuality, their complaint “about lack of transparency” was limited to methodological transparency:

Disappointingly, none of the journals have so far secured an agreement from Professor Tol to make his calculations available, which means that a number of the data included in Chapter 10 of the IPCC report remain unverifiable.

The only concern stated regarding data was that the data might be incorrect. To check that, one must examine the calculations. Having only the data does little good. One can aggregate data in more than one way. Without knowing what process Tol used, it is difficult, if not impossible, to verify his calculations are correct. If one used an even slightly different process than him, the results could be different regardless of whether or not he implemented his methodology correctly.

Moreover, if the process is as trivial as Tol suggests, there is absolutely no reason for him to keep stonewalling requests for an explanation of it. A trivial process would take only a couple minutes to document. The only reason to refuse a simple and reasonable request like this is an unscientific obstinance.

To sum up, I was able to check nearly every statement Richard Tol claimed was a fib existing only in John Abraham and Bob Ward’s “wild imaginations.” In each case, not only were their statements accurate, Tol’s responses were misleading, if not completely false.

The one claim I was unable to check was the claim Tol is unwilling to correct certain errors. Tol points to the fact he has corrected some errors as a rebuttal. While it’s true Tol has acknowledged some errors, it is unclear to me what errors Ward thinks have been left uncorrected. My current impression is Ward’s statement regarding this is at least overstated, but it may or may not have some truth to it.


One comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s