Missing the Obvious

My last post criticized changes made to the recent IPCC report by Richard Tol after the last round of external reviews. There’s a lot of material involved, and I won’t go into it here. Instead, I’d like to show how I was an idiot in my previous discussions of this topic.

Here is a graph from Richard Tol’s 2009 paper. Pay attention to what I highlighted in its caption:

8-13-Tol-Original

Compare that to the graph shown in the IPCC report:

AR5-10-1

Notice this caption does not say what starting temperature was used. The text of the section its in doesn’t either:

Since AR4, four new estimates of the global aggregate impact on human welfare of moderate climate change were published (Bosello et al., 2012; Maddison and Rehdanz, 2011; Roson and van der Mensbrugghe, 2012), including two estimates for warming greater than 3°C. Estimates agree on the size of the impact (small relative to economic growth) but disagree on the sign (Figure 10-1). Climate change may be beneficial for moderate climate change but turn negative for greater warming. Impacts worsen for larger warming, and estimates diverge. The new estimates have slightly widened the uncertainty about the economic impacts of climate.

A reader could assume the IPCC figure referred to temperature changes from today, but that is only an assumption. Only someone who had read other work, like Tol’s 2009 paper, would know this analysis is for what we can expect in the future. And by “know,” I mean, “would be tricked into believing.” Here’s the updated version of that graph and its caption (from here:

8-11-Tol-Updated

Pay close attention to its caption. It says:

estimates of the global economic impact of climate change, expressed as the welfare-equivalent income gain or loss, as a function of the increase in the annual global mean surface air temperature relative to preindustrial times.

“[R]elative to preindustrial times” is quite different than “relative to today.” There’s been ~.85C warming since preindustrial times (see page three). That means the x-axis in his original graph was off by nearly one full degree.

That’s a significant error. The only portion of Tol’s graph which shows increasing values is the first ~1C. If you cut off the first .85 of it, there’s practically no increase. This is what would have fit Tol’s caption:

8-13-Tol-Shifted

Nobody seeing that graph would think global warming is a good thing. Maybe global warming was a good thing in the past, but that graph says it will only make things worse in the future.

Of course, Tol didn’t tell people that. He published a paper saying global warming will be beneficial for the next two degrees of warming. He told the media we’d see such benefits. Everybody understood his work as saying future warming would be a good thing. Here are a couple explicit examples:

As Tol’s diagram quite clearly indicates, the consensus of economic studies finds that global warming would be on net beneficial to human welfare, at least through 2C degrees of warming (and this is relative to the current baseline, not to preindustrial times).

At first, I thought this was just their usual bluster. But then I realised that they are genuinely unaware. Good news is no news, which is why the mainstream media largely ignores all studies showing net benefits of climate change. And academics have not exactly been keen to push such analysis forward. So here follows, for possibly the first time in history, an entire article in the national press on the net benefits of climate change.

To be precise, Prof Tol calculated that climate change would be beneficial up to 2.2˚C of warming from 2009 (when he wrote his paper). This means approximately 3˚C from pre-industrial levels, since about 0.8˚C of warming has happened in the last 150 years. The latest estimates of climate sensitivity suggest that such temperatures may not be reached till the end of the century — if at all. The Intergovernmental Panel on Climate Change, whose reports define the consensis, is sticking to older assumptions, however, which would mean net benefits till about 2080. Either way, it’s a long way off.

Even Richard Tol’s critics generally failed to notice this problem. The blogger Anders wrote about this conclusion, taking for granted:

What I was interested in is if there was any evidence that there could be a net benefit from 2 degrees of warming (relative to pre-industrial levels). In a comment on a recent post, Martin linked to a 2008 paper (actually an ESRI working paper) by Richard Tol called The Economic Impact of Climate Change. The paper is really a review of 13 studies (and 14 estimates) that have considered the economic impacts of climate change. The basic result is shown in the figure below. It shows the percentage change in GDP for increases in global surface temperature (relative to today) of up to 3oC. There is a suggestion that a rise of 1oC (i.e., almost 2oC relative to pre-industrial times) could be beneficial, but anything beyond 2oC (relative to today) seems likely to be detrimental.

A few people did notice the issue, but only after Tol was forced to publish a correction. For four years, everybody was deceived by Richard Tol into believing his work estimated global warming will cause net benefits for a significant amount of time. In reality, his work estimates global warming will pretty much only cause harm from now on.


At least, that’s true if we believe what Richard Tol says about his work now. Mabye we shouldn’t. My brief perusal of the sources he used shows at least some of the estimates are for changes since pre-industrial times. I don’t know that they all are though. It’s possible some may have used modern times as a starting point. If so, Tol’s work is pointless as some unknown number of estimates he used are wrong.

Regardless, it makes the new text of the IPCC report rather interesting. How can it say:

Estimates agree on the size of the impact (small relative to economic
growth) but disagree on the sign (Figure 10-1).

When we now know the only meaningful positive estimate is (basically) for where we are now, not where we’ll be in the future? How can it say:

Climate change may be beneficial for moderate climate change but turn negative for greater warming.

When every version of Tol’s own work apparently says we can only expect things to go down hill?

And beyond all this, if Tol’s correction is right, and those estimates really are relative to pre-industrial, not modern times, why did Richard Tol say otherwise for four years? Why is it when he finally corrected himself, he didn’t even attempt to draw attention to this correction? Why didn’t he do anything to let people who have been relying upon his work know that work was wrong?

And what’s the point of having peer-review, much less having thousands of scientists review the IPCC reports, if they can’t catch such a fundamental, and simple, problem?

Advertisements

22 comments

  1. Having had the recent experience of skimming a number of this papers in the other thread, I can definitely say that Tol 2002 is *not* based on pre-industrial temperatures. Its timeline begins at 0 in 2000, which is inconsistent with that. I’ve closed the windows on the other papers I read, but I didn’t get the impression from any of them that they were talking about increases from preindustrial times. I suspect the caption is in error. The copy of Tol 2009 I have open (from Nordhaus at Yale) states the increase is from “today”.

  2. I also wonder why you quote people who are not Tol to complain about what Tol has said. Surely what Tol actually said in Tol 2009 is most relevant to what Tol 2009 claims? Here’s what he specifically said about the short-term benefits:

    However, this pattern should be interpreted with care. Even if, initially, economic impacts may well be positive, it does not follow that greenhouse gas emissions should be subsidized. The climate responds rather slowly to changes in greenhouse gas emissions. The initial warming can no longer be avoided; it should be viewed as a sunk benefit. The fitted line in Figure 1 suggests that the turning point in terms of economic benefits occurs at about 1.1°C warming (with a standard deviation of 0.7°C). Policy steps to reduce emissions of greenhouse gases in the near future would begin to have a noticeable affect on climate sometime around mid-century—which is to say, at just about the time that any medium-run economic benefits of climate change begin to decline (Hitz and Smith, 2004; Tol, 2002b; Tol, Fankhauser, Richels, and Smith, 2000). In short, even though total economic effects of 1–2°C warming may be positive, incremental impacts beyond that level are likely to be negative. Moreover, if one looks further into the future, the incremental effects look even more negative.

    The glowing we-don’t-need-to-worry lesson that Murphy got from Tol 2009 is not what I drew from the paper. I don’t worry because I don’t believe Tol’s 2009 negative outlook, even if it is less negative than some alarmists would like. Judging from the massive amount of caveats contained in the Tol 2009 text, I don’t think anyone else should find it (or anything else in that research area) as being closer to facts than speculation.

  3. Dale Stephenson, I’m not sure you need to say Tol 2009 states the increase is relative to today when I highlighted it saying exactly that in my post 😛

    Anyway, I’m not sure just where all the estimates are relative to. Rehdanz and Maddison 2005 might be relative to 1995, but I’m not sure. Tol 2002 is definitely relative to ~2000. Bosello 2012 is definitely relative to pre-industrial times. It specifically says:

    Estimates indicate that a temperature increase of 1.92°C compared to pre-industrial levels in 2050 (consistent with the A1B IPCC SRES scenario) could lead to global GDP losses of approximately 0.5% compared to a hypothetical scenario where no climate change is assumed to occur.

    It may be we can just shift that one by ~.85 to the left in the graph, and everything would be fine. It’s hard to know though. With how many problems there are in his data set, the only way to be sure of anything is to look for yourself.

    It’s pretty silly when people have to check a couple dozen references because an author can’t even get his baselines straight.

  4. Dale Stephenson, one random quote from Richard Tol wouldn’t add anything to the discussion. I was discussing how to interpret Tol’s paper. Showing how people interpreted it is a normal way to do that. Showing other things the author of the paper might have said is not. This rhetorical remark of yours:

    Surely what Tol actually said in Tol 2009 is most relevant to what Tol 2009 claims?

    Is just wrong. What the paper says, and how people understood it, are most relevant to what the paper says.

  5. You’re right about Boselli, I missed that completely. That makes the scale of increase a fairly close match to Tol ’02, but unfortunately it takes it off the y axis, since the -0.5% is a result of the post-1850 increase to the future, *not* the +1.0C from “today”. It would’ve been fascinating if it’d shown the 1850-today impact. I’m *very* skeptical that the 1850-now increase resulted in a net negative impact even before CO2 fertilization effects.

    It’s pretty silly having to check references, it’s equally silly when the references themselves don’t make clear what the baseline actually is. Tol’s not lacking for errors, but I don’t think he’s unique in this regard.

    As far as my quote from Tol 2009 goes, I’m a strict constructionist when it comes to papers. What people understand, or think they understand about papers is not relevant. What’s relevant is what the paper *actually* claims. I thought Boselli was +1.82C from current times. I was wrong. Text trumps impression. Murphy thinks Tol’s diagram “clearly indicates” something. Tol states in the text that it’s more an interesting calculation than hard analysis, and peppers the text liberally with caveats like the one I quoted. Murphy is evidence for what Murphy thinks, and if he thinks wrongly I certainly would not jump to the conclusion that Tol somehow deceived him. My quote was relevant because it was from the paper. Murphy’s quote was irrelevant because it was not.

    Now if Tol is misrepresenting his own paper, that’s certainly worth calling out. In this age of science by press conference, that seems way too common to me. Even the difference between abstract and content is sufficient enough for me not to want to comment on any paper with the details hidden behind a paywall.

  6. Dale Stephenson:

    You’re right about Boselli, I missed that completely. That makes the scale of increase a fairly close match to Tol ’02, but unfortunately it takes it off the y axis, since the -0.5% is a result of the post-1850 increase to the future, *not* the +1.0C from “today”.

    Tol 2002 is relative to 1990 (I checked the equations). Nordhaus 1994b is relative to ~1993. Maddison and Rehdanz 2011 is relative to 2008. How can you compare them despite their differences if you can’t compare Bosello 2012?

    And why does it matter if the results are from “today”? Maddison and Rehdanz 2011
    used SRES A2 projections for temperatures. Roson and van der Mensbrugghe 2012 used a new model of their own to project temperatures. Nordhaus 1994a used three entirely different projections (though results from only one were reported by Tol).

    You have about ~20 different temperature paths for ~20 different papers. How can it be okay to compare some to each other, but not others?

  7. By the way, when you quoted the Tol 2009 paper, I didn’t realize you were quoting the paper. I thought you just said you were quoting from Tol in 2009 (as in, a contemporary comment).

    I definitely agree what a paper says is most important. However, nothing in that quote contradicts anything I said, or anything said by those I quoted. I can’t see why I should have quoted it in this post.

  8. Here’s another question. From what I’ve seen, Tol 1995 only did calculations for the United States and Canada. How can damages for one region be compared to global damages?

    And how is that better than comparing data from different time periods?

  9. Brandon, I think you ask an excellent question about how you can compare such different models. Even before you pointed out the different baselines, I was pointing out that the different methods, the different coverage, and the different magnitude made it so very few of these papers were directly comparable. Throw in the differing baselines and timelines and it’s possible that none of these studies are directly comparable at all. I’m not the one who has been referring to Tol ’02 as an outlier or laying stress on a consensus of signs.

    So what do all this mismatched and incomparable papers actually have in common? With one exception (Maddison 2011), they’re all awfully small relative to GDP. And I don’t think any of them dispute that *eventually* the net effect of temperature rise will be increasingly negative.

    The other thing they seem have in common is that the methods are shaky and the conclusions should have error bars from floor to ceiling. If summarizing the entire body of work in three sentences, more weasel words need used, not less.

  10. Brandon, with respect to my Tol 2009 quotation, I did not say you should have quoted it. But I stand by my claim that it is more relevant to what Tol 2009 actually claims. The sense of Murphy is that we don’t need to worry because warming is actually good. My Tol 2009 quotation shows that Tol wrote that the benefits were sunk and we need to worry.

    Here’s my opinion of what you “should have quoted”. When alleging that Tol has deceived people, you should quote Tol, and only Tol. When doing so, you should not ignore other Tol quotes that convey a different sense.

  11. Well, obviously I can’t answer for Richard Tol. However, I looked at the Nordhaus 94 paper (expert elicitation) which you mentioned earlier, and it is not clear about its temperature baseline. E.g. scenario A is described as a 3 K rise by 2090. I interpret this as being 3K from the then-current temperature, for two reasons. First, that’s the “natural” interpretation (in my opinion) in the absence of an explicit baseline. Second, scenario A is described as “in the middle range of the projections made by the IPCC.” Only the first AR was extant in 1994, and its projection for “business as usual” emissions and mid-range sensitivity (ECS=2.5K if I recall correctly) was around 0.3 K/decade, or 3 K/century.

    So for this data point, one can infer the baseline, and it’s not pre-industrial. [As described in the captions to Tol(2009) corrected & updated figures 1&2.] But I wouldn’t be at all surprised if other studies were more ambiguous in this regard, making a definite answer impossible.

    And yes, I don’t know how one can put all the data points on a single graph if their baselines are not reconciled or unknown. But frankly, I think the whole exercise, viewed quantitatively, stinks. If one views the answers as qualitative WAGs, OK. But the values are being treated in the same manner as noisy measurements, and they don’t seem to me (a) to be estimating the same property, and (b) to have any obvious or demonstrable distribution relative to some “true” value.

  12. I think it’s fair to say that the least-squares-fit regression line from Tol 09’s Figure 1 (original or revised) means nothing and the confidence intervals derived from the line also means nothing. It’s good that the corresponding IPCC figure lacks both and just plops them on the x-y axis, though it’d be nice if the text made clear that the points aren’t actually comparable to each other.

  13. While not referring specifically to Tol ’95, Tol ’09 calls out extrapolation as a weakness of enumerative models in general:

    However, the enumerative approach also raises concerns about extrapolation: economic values estimated for other issues are applied to climate change concerns; values estimated for a limited number of locations are extrapolated to the world; and values estimated for the recent past are extrapolated to the remote future.
    —-
    My impression is that Tol 2009 devotes far more space to caveats than findings. This seems appropriate.

  14. Dale Stephenson:

    Here’s my opinion of what you “should have quoted”. When alleging that Tol has deceived people, you should quote Tol, and only Tol. When doing so, you should not ignore other Tol quotes that convey a different sense.

    I quoted Richard Tol’s paper saying something. I then pointed to various people showing it was understood the way I claimed it was. There is no reason I should have stopped at the first, especially not when Richard Tol gave interviews about his work.

    As for the caveats you say I didn’t quote, as far as I can see, they don’t detract from anything said in this post. You haven’t done anything to show they do.

    My impression is that Tol 2009 devotes far more space to caveats than findings. This seems appropriate.

    Even if that were true, he made a figure and presented it with a stated level of certainty which was entirely wrong. The caveats in his paper don’t change that fact.

    I think it’s fair to say that the least-squares-fit regression line from Tol 09’s Figure 1 (original or revised) means nothing and the confidence intervals derived from the line also means nothing. It’s good that the corresponding IPCC figure lacks both and just plops them on the x-y axis, though it’d be nice if the text made clear that the points aren’t actually comparable to each other.

    This comment confuses me. For all the grief you’ve given me for (supposedly) comparing data points that aren’t comparable, all you say when the IPCC does it is “it’d be nice if” they made it clear the points aren’t comparable. You seem to devote a lot of time and energy to downplaying criticisms of what Richard Tol has done while only casually acknowledging problems in the most offhand of ways.

  15. By the way, this is the sort of thing Richard Tol says in public:

    Even if one accepts WG2′s estimate that a “further warming of 2°C could cause loses equivalent to 0.2-2 per cent of world gross domestic product,” that is “about as bad as losing one year of economic growth” in half a century, Tol notes.

    The IPCC’s estimate on this is based entirely upon his work. If he’s giving interviews where he describes his work that way, I don’t see how any caveats in his paper could make my portrayal of his work unfair.

  16. Brandon, I think the caveats in the paper about Tol ’09’s Figure 1 are an important part of the figure’s “stated level of certainty”. What Figure 1 means is defined by what Tol ’09 says about it, and that includes in-text caveats as well as captions. In your posts I see a lot of what you say about the figure and very little about what Tol says about the figure. And you keep bringing it up in connection with the very few sentences from the IPCC report, which does not include, reference, or rely on Figure 1.

    If Tol himself is misrepresenting Figure 1 in the media, by all means criticize him for that. But I’m not going to blame Tol if someone *else* is misrepresenting it.

    Criticizing Tol for the numerous errors in Figure 1 is certainly appropriate and you’ve identified more than one. Not only is it not very meaningful, it’s also sloppy.

    Why do I only say “it’d be nice” if the IPCC pointed out that the studies aren’t actually comparable? Because the IPCC selection you’ve quoted only devote three short sentences to summarizing the studies. If you were Tol and were writing three sentences between the first and fifth sentences, what would you write? Given infinite space, I’d prefer something like Tol ’09, spending lots of time on what these studies are, how they work, what they cover, what they don’t cover, and how they’re totally inadequate for forming policy. But I don’t think that’s remotely a realistic expectation.

  17. As far as Tol’s comment that +2C could cause losses between 0.2% and 2.0%, I don’t see why you state that’s “based entirely on his work”. There’s plenty of non-Tol papers in the table that estimate losses of less than 2% for warming greater than +2C. Why am I supposed to criticize Tol for a statement that is weak (“could”), true, and apparently representative?

  18. Dale Stephenson:

    Brandon, I think the caveats in the paper about Tol ’09’s Figure 1 are an important part of the figure’s “stated level of certainty”. What Figure 1 means is defined by what Tol ’09 says about it, and that includes in-text caveats as well as captions. In your posts I see a lot of what you say about the figure and very little about what Tol says about the figure. And you keep bringing it up in connection with the very few sentences from the IPCC report, which does not include, reference, or rely on Figure 1.

    Three things. First, it is never okay to present a misleading figure with the justification, “The text explains it.” I’d hope we don’t need to revisit that argument after having it with the “hide the decline” saga.

    Second, you haven’t shown any caveats in the paper which have a meaningful impact on the figure in question. The only relevant caveat I’ve seen you quote deals solely with the regression line, and it barely caveats the reader about it. It doesn’t do anything to caution about the nature of the regression or the incompatibility of the estimates. (As with the IPCC report, I don’t see you criticizing Tol 2009 for comparing these points.)

    Third, the IPCC report compares the data points. That it doesn’t show the regression line from Tol 2009 is irrelevant. Whatever implicit comparison it does is equally problematic. That the reader has to guess at the basis of what it says doesn’t help its case.

    Why do I only say “it’d be nice” if the IPCC pointed out that the studies aren’t actually comparable? Because the IPCC selection you’ve quoted only devote three short sentences to summarizing the studies. If you were Tol and were writing three sentences between the first and fifth sentences, what would you write?

    What…? Richard Tol added a completely new section, and you say we shouldn’t be very upset it is misleading because he kept it short? He didn’t have to write what he wrote at all. He could have written more. He is the one who chose to write so little. How can you use his choice not to include more information as justification for him not including more information?!

    If his work needed that much space to explain its caveats (which don’t begin to cover the extent of its problems), it probably shouldn’t have been included at all. He certainly should not have written entirely new text just so he could promote his own work.

    You’re acting as though him providing misleading information just to promote himself is somehow better than him just not providing any information.

    As far as Tol’s comment that +2C could cause losses between 0.2% and 2.0%, I don’t see why you state that’s “based entirely on his work”. There’s plenty of non-Tol papers in the table that estimate losses of less than 2% for warming greater than +2C. Why am I supposed to criticize Tol for a statement that is weak (“could”), true, and apparently representative?

    This is a red herring. It doesn’t matter how many papers in Tol’s figure had his finger on them. He made the figure. The calculations were done via his work. If you remove the collation process he used, you cannot get the figures used in the SPM.

    It doesn’t matter what data there is. If you don’t have a step where you process the data (i.e. Tol’s work), you can’t have answers drawn from the data.

  19. Brandon, you wrote:

    Three things. First, it is never okay to present a misleading figure with the justification, “The text explains it.” I’d hope we don’t need to revisit that argument after having it with the “hide the decline” saga.

    Hide the decline involved actually deleting data to make a graph look better. While I don’t think the regression line was appropriate, I don’t think there’s any evidence that the data was intentionally manipulated to make the line look better. (There is evidence that some of the data is wrong, of course.)

    Second, you haven’t shown any caveats in the paper which have a meaningful impact on the figure in question. The only relevant caveat I’ve seen you quote deals solely with the regression line, and it barely caveats the reader about it. It doesn’t do anything to caution about the nature of the regression or the incompatibility of the estimates. (As with the IPCC report, I don’t see you criticizing Tol 2009 for comparing these points.)

    You don’t see me criticizing Tol for plotting the points, which in the absence of the regression line does not constitute any sort of weighting. I find the figure useful when used in conjunction with the table, which identifies not only the impact and magnitude but also the age, type, coverage, and author. I do think it’s appropriate to plot the points, I think it’s reasonable to make general observations like “hey, they’re all pretty small” or “hey, they’re mostly negative” or “hey, there’s not many evaluations at +1.0C”. I just don’t think it’s appropriate to treat them as data and calculate a regression line. Tol ’09 does that. IPCC doesn’t. (Criticism of Tol for incorrectly plotting the points is still appropriate.)

    Second, you haven’t shown any caveats in the paper which have a meaningful impact on the figure in question. The only relevant caveat I’ve seen you quote deals solely with the regression line, and it barely caveats the reader about it. It doesn’t do anything to caution about the nature of the regression or the incompatibility of the estimates. (As with the IPCC report, I don’t see you criticizing Tol 2009 for comparing these points.)

    You don’t think the repeated caveats about the quality, coverage, assumptions, independence, uncertainty, methods, and validation of the estimates being plotted have a meaningful impact on the figure in question? I disagree strongly on that point. The conclusions section to Tol 2009 don’t mention the figure, or even the possibility of positive impacts from small temperature rises. It *does* mention that what they have isn’t very good.

    I don’t think it’s possible to do a survey article without comparing estimates *at all*. The Findings and Implications mostly refers to the table rather than the figure, here’s the discussion relevant to Figure 1:

    Given that the studies in Table 1 use different methods, it is striking that the estimates are in broad agreement on a number of points—indeed, the uncertainty analysis displayed in Figure 1 reveals that no estimate is an obvious outlier.

    The rest of that paragraph is caveats about assumptions and observations about different columns in Table 1.

    The second finding is the one relevant to the “possible benefits” claim, and invokes figure 1 for support, not source.

    A second finding is that some estimates, by Hope (2006), Mendelsohn, Morrison, Schlesinger, and Andronov (2000), Mendelsohn, Schlesinger, and Williams (2000), and myself (Tol, 2002b), point to initial benefits of a modest increase in temperature, followed by losses as temperatures increase further. Figure 1 illustrates the pattern. There are no estimates for a warming above 3°C, although climate change may well go beyond that (as discussed below). All studies published after 1995 have regions with net gains and net losses due to global warming, while earlier studies only find net losses.

    The horizontal axis of Figure 1 shows the increase in average global temperature. The vertical index shows the central estimate of welfare impact. The central line shows a best-fit parabolic line from an ordinary least squares regression. Of course, it is something of a stretch to interpret the results of these different studies as if they were a time series of how climate change will affect the economy over time, and so this graph should be interpreted more as an interesting calculation than as hard analysis. But the pattern of modest economic gains due to climate change, followed by substantial losses, appears also in the few studies that report impacts over time (Mendelsohn, Morrison, Schlesinger, and Andronova, 2000; Mendelsohn, Schlesinger, and Williams, 2000; Nordhaus and Boyer, 2000; Tol, 2002b; also, compare Figure 19-4 in Smith et al., 2001).

    My reading of the two Mendelsohn papers supports this contention, but I don’t see it in Nordhaus, and I haven’t read Hope. The part about regions with net gains holds true for the post-2009 papers I’ve read, and I found a reference in Tol 02 that suggests Tol ’95 projects gains at +1.0C, though, it projects negative for all regions at +2.5C.

    After a further paragraph about why net positive benefits may occur (basically because the benefited regions have bigger economies than the harmed regions), another paragraph mentions Figure 1 while arguing we should ignore the benefits:

    However, this pattern should be interpreted with care. Even if, initially, economic impacts may well be positive, it does not follow that greenhouse gas emissions should be subsidized. The climate responds rather slowly to changes in greenhouse gas emissions. The initial warming can no longer be avoided; it should be viewed as a sunk benefit. The fitted line in Figure 1 suggests that the turning point in terms of economic benefits occurs at about 1.1°C warming (with a standard deviation of 0.7°C). Policy steps to reduce emissions of greenhouse gases in the near future would begin to have a noticeable affect on climate sometime around mid-century—which is to say, at just about the time that any medium-run economic benefits of climate change begin to decline (Hitz and Smith, 2004; Tol, 2002b; Tol, Fankhauser, Richels, and Smith, 2000). In short, even though total economic effects of 1–2°C warming may be positive, incremental impacts beyond that level are likely to be negative. Moreover, if one looks further into the future, the incremental effects look even more negative.

    This is the original article. The fitted line obviously is different in the updated figure, but I don’t know if this we-should-worry-anyway paragraph was reworked.

    Figure 1 gets mentioned in finding five:

    Figure 1 has three alternative estimates of the uncertainty around the central estimates. First, it shows the sample statistics. However, these may be misleading for the reasons outlined above; note that there are only two estimates each for a 1.0°C and a 3.0°C global warming. Second, I re-estimated the parabola 14 times with one observation omitted each time. This exercise shows that the shape of the curve in Figure 1 does not depend on any single observation. At the same time, the four estimates for a 1.0°C or 3.0°C warming each have a substantial (but not significant) effect on the parameters of the parabola. Third, five studies report standard deviations or confidence intervals. Confidence intervals imply standard deviations, but because the reported intervals are asymmetric I derived two standard deviations, one for negative deviations from the mean, and one for positive deviations. I assumed that the standard deviation grows linearly with the temperature and fitted a line to each of the two sets of five “observed” “standard deviations.” The result is the asymmetric confidence interval shown in Figure 1. This probably best reflects the considerable uncertainty about the economic impact of climate change and that negative surprises are more likely than positive ones.

    In short, the level of uncertainty here is large, and probably understated—especially in terms of failing to capture downside risks. The policy implication is that reduction of greenhouse gas emissions should err on the ambitious side.

    There’s one final mention later on the paper:

    Another big unknown is the effect of climate change in the very long term. Most static analyses examine the effects of doubling the concentration of atmospheric CO2; most studies looking at effects of climate change over time stop at 2100. Of course, climate change will not suddenly halt in 2100. In fact, most estimates suggest that the negative effects of climate change are growing, and even accelerating, in the years up to 2100 (as suggested by Figure 1). It may be that some of the most substantial benefits of addressing climate change occur after 2100, but studies of climate change have not looked seriously at possible patterns of emissions and atmospheric concentrations of carbon after 2100, the potential physical effects on climate, or the monetary value of those impacts. One may argue that impacts beyond 2100 are irrelevant because of time discounting, but this argument would not hold if the effects grow faster than the discount rate—because of the large uncertainty, this outcome cannot be excluded.

    That’s the *entirety* of Figure 1 mentions, and I don’t remotely get a “figure one proves global warming is good” vibe from it. You say “Everybody understood his work as saying future warming would be a good thing”. I think drawing that conclusion from the entirety of Tol ’09 is absurd, and even claiming that as the take-home message from the few paragraphs discussing figure one is, in Tol’s words regarding the regression line, “something of a stretch”.

    Third, the IPCC report compares the data points. That it doesn’t show the regression line from Tol 2009 is irrelevant. Whatever implicit comparison it does is equally problematic. That the reader has to guess at the basis of what it says doesn’t help its case.

    That it doesn’t show the regression line from Tol 2009 means that criticism of Tol 2009’s regression line is irrelevant to the IPCC report. I don’t think plotting the points is inherently bad (plotting them incorrectly is still a problem), and I think making weak, vague observations based on them is appropriate. Whether the weak, vague observations Tol did make are the most informative remarks to make to sum up the state of the estimates in three sentences is a different question. I think it could be better, though now that I’ve read the fifth sentence and the following paragraph I’m less concerned about confusing “estimates” with “new estimates”, the lack of discussion of new estimate impact (fifth sentence) and the lack of caveats (second paragraph). However, I think your criticism is bang-on here:

    What…? Richard Tol added a completely new section, and you say we shouldn’t be very upset it is misleading because he kept it short? He didn’t have to write what he wrote at all. He could have written more. He is the one who chose to write so little. How can you use his choice not to include more information as justification for him not including more information?!

    You’re absolute right here. Tol could have chosen to write more and did not. He could have laid things out in greater detail and did not. And most crucially, he did not carry over a figure from the previous work that had exactly what I wanted to see, a comparison of damage paths! That would’ve made a lovely supplement to the information that’s there (I think 10-1 would still have value, because of the presence of estimates from different methodologies). There is definitely more that could be said than was said, and I think more caveats certainly would be appropriate. However, I think any extended discussion certainly *should* include the possibility of net benefit from small warmings. It should also include the consensus of net benefit for some regions from warming and caveats should be clear that the uncertainty goes in both directions, not just on the alarmist side.

    Further, I think you make a good case (elsewhere) that Tol trimmed this area down specifically because of caveats that he saw as an attack on his field. To be fair, Tol may be right when he complains that such caveats are not applied elsewhere in the document. Aggregate impacts isn’t alone in needing massive caveats. In fact, it needs massive caveats partially because its inputs are so often coming from elsewhere in climate science.

    If his work needed that much space to explain its caveats (which don’t begin to cover the extent of its problems), it probably shouldn’t have been included at all. He certainly should not have written entirely new text just so he could promote his own work.

    This is where we part company, I think. If it were just Tol’s work that needs massive caveats, then it shouldn’t be included at all. But it’s everybody’s work that needs massive caveats. That’s what makes these estimates inapplicable for anything other than vague generalizations, they aren’t good enough for firm conclusions. But since they’re the best we have, they need to be summarized in some fashion, and even plotted on the same graph.

    And I don’t think the massive caveats for this particular little field are at all unique in climate science. If the other areas in the document give short shrift to their weaknesses, it actually would be misleading-but-true to include it for Tol’s field only. If we reduced the IPCC documents to the fields where the impacts are well understood and well qualified, I think the SPM could be reduced to saying “Temperature may go up.”

    But I still think he should’ve kept those two figures.

    On the subject of what Tol said, I asked:

    As far as Tol’s comment that +2C could cause losses between 0.2% and 2.0%, I don’t see why you state that’s “based entirely on his work”. There’s plenty of non-Tol papers in the table that estimate losses of less than 2% for warming greater than +2C. Why am I supposed to criticize Tol for a statement that is weak (“could”), true, and apparently representative?

    You replied:

    This is a red herring. It doesn’t matter how many papers in Tol’s figure had his finger on them. He made the figure. The calculations were done via his work. If you remove the collation process he used, you cannot get the figures used in the SPM.

    If you replace the collation process he used with whatever collation process you care to invent, I think you’ll get a graph something like what you find in the SPM. But how is that relevant to Tol’s comment which, as quoted, did not reference the SPM figure. In fact, the figure has no data points AT ALL at +2.0C. Now given the data points beyond +2.0C, I think the “could” statement is certainly justified, and dropping out the estimates that Tol touched leaves a bunch of estimates that are certainly consistent with that estimate. Further, if you go back and look at the sadly deleted figure 19.7 that compares the time damage series, at about +2.8C it very much looks like all the models but CRED are in that -0.2% to -2.0% range. Did Tol make that figure? If Tol is just promoting his work, why wouldn’t he just give the FUND range, which at that point still includes the possibility of positive impact?

    I think you should consider the possibility that you are no longer capable of discussing Tol or his work objectively.

  20. Oops. I didn’t notice this last comment before. A very late response:

    Hide the decline involved actually deleting data to make a graph look better. While I don’t think the regression line was appropriate, I don’t think there’s any evidence that the data was intentionally manipulated to make the line look better. (There is evidence that some of the data is wrong, of course.)

    This is true, but irrelevant. The reason I referenced this is during discussions of the issue, a number of people defended the graph by saying the deletion was discussed in the text. This was roundly rejected as presenting a misleading graph is wrong, even if “the text explains it.”

    You don’t think the repeated caveats about the quality, coverage, assumptions, independence, uncertainty, methods, and validation of the estimates being plotted have a meaningful impact on the figure in question? I disagree strongly on that point. The conclusions section to Tol 2009 don’t mention the figure, or even the possibility of positive impacts from small temperature rises. It *does* mention that what they have isn’t very good.

    If there are any relevant caveats, I haven’t seen you quote them. I also haven’t spent much time looking for them as I’m more interested in the subsequent work using the same approach (such as Tol 2013, a near direct predecessor to the IPCC section), work which largely lacks any such caveats.

    I don’t think it’s possible to do a survey article without comparing estimates *at all*. The Findings and Implications mostly refers to the table rather than the figure, here’s the discussion relevant to Figure 1:

    First off, you quote text which is simply wrong because Richard Tol somehow messed up his data. Second, I don’t see how you can justify supporting text which refers to a “broad agreement” in data while criticizing me for saying the data disagrees. I’m not seeing any particularly meaningful caveats here.

    The second finding is the one relevant to the “possible benefits” claim, and invokes figure 1 for support, not source.

    Which acknowledges it is being used to support an argument, something you’ve agreed it cannot do. Without that regression, what you quoted has nothing of value.

    After a further paragraph about why net positive benefits may occur (basically because the benefited regions have bigger economies than the harmed regions), another paragraph mentions Figure 1 while arguing we should ignore the benefits:

    That’s not actually what that paragraph says. It says those benefits are inevitable while damages are not, but it doesn’t say we should ignore the benefits.

    Figure 1 gets mentioned in finding five:

    A finding which is completely wrong due to Tol’s strange data errors.

    That’s the *entirety* of Figure 1 mentions, and I don’t remotely get a “figure one proves global warming is good” vibe from it. You say “Everybody understood his work as saying future warming would be a good thing”. I think drawing that conclusion from the entirety of Tol ’09 is absurd, and even claiming that as the take-home message from the few paragraphs discussing figure one is, in Tol’s words regarding the regression line, “something of a stretch”.

    You quoted him referring to inevitable benefits of global warming. How then, can you say the paper doesn’t say future warming will be a good thing?

    However, I think any extended discussion certainly *should* include the possibility of net benefit from small warmings. It should also include the consensus of net benefit for some regions from warming and caveats should be clear that the uncertainty goes in both directions, not just on the alarmist side.

    I’m fine with that, so long as one doesn’t imply the evidence is in favor of their being net benefits for moderate warming. That is, unless one can actually show such rather than just saying some studies suggest it will.

    This is where we part company, I think. If it were just Tol’s work that needs massive caveats, then it shouldn’t be included at all. But it’s everybody’s work that needs massive caveats. That’s what makes these estimates inapplicable for anything other than vague generalizations, they aren’t good enough for firm conclusions. But since they’re the best we have, they need to be summarized in some fashion, and even plotted on the same graph.

    The fact we lack decent work in no way means we must include bad work. There are plenty of other ways this section could have been written. For instance, they could have done what they do in most sections: summarize what the papers show. It’s not like the IPCC generally relies on (horribly low-quality) meta-studies to summarize fields. Instead, it looks at the studies and discusses what they say.

    (Nevermind the other section where Tol made significant revisions entirely to undermine work he didn’t agree with even though there was nothing wrong with the text)

    If you replace the collation process he used with whatever collation process you care to invent, I think you’ll get a graph something like what you find in the SPM. But how is that relevant to Tol’s comment which, as quoted, did not reference the SPM figure.

    Er… what? I think you misunderstood what I said. When I said “the figures used in the SPM,” I was referring to the numerical values of -0.2% to -2.0%. There was no graphical representation of Tol’s findings in the SPM.

    Did Tol make that figure? If Tol is just promoting his work, why wouldn’t he just give the FUND range, which at that point still includes the possibility of positive impact?

    I think you should consider the possibility that you are no longer capable of discussing Tol or his work objectively.

    We have a numerical range you defend the inclusion of despite no source for it being presented. You argue we can’t know it came from Tol’s work despite the fact he is the clear originator of it, having added the same values (with a different reference point) to his section at the same point he made significant entirely relying on his own work and views.

    I don’t place much stock in your judgment of my objectivity. I’d suggest you consider your own advice. I’m not sure you’ve raised a single new point in any of these discussions which was critical of Richard Tol or his work. Even when you’ve acknowledged criticisms, you’ve almost always done so grudgingly.

    I don’t think it should be hard to believe I’d be just as critical of anyone who had done this sort of thing. I’ve been harder on people who did far less wrong.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s