Richard Tol Abuses the IPCC Process

There’s a spat between Richard Tol and Bob Ward I haven’t paid any attention to. It seemed boring until I saw a tweet today from Tol which claimed Ward admitted to lying. I rarely hear anyone make such an admission. Naturally, I was curious.

It turned out Tol’s claim was misleading. Ward hadn’t admitted to lying. He said some things, and Tol interpreted them, in their totality, as Ward admitting to lying. Whatever. It was boring, but it wasn’t a complete loss. I discovered an interesting abuse of the IPCC process which Tol defends. It’s not surprising he defends it given he’s apparently the one responsible for it.

This is what Tol quoted Ward as saying about the IPCC and Tol’s response:

“A section had been inserted on ‘Aggregate impacts’”
In fact, that section was moved from Chapter 19 to Chapter 10. As far as I am aware, Mr Ward did not raise this concern with the IPCC. He was informed no later than 2 April 2014 that the text was moved rather than added.

This is fuller context of what Ward said:

I was stunned when I looked at Chapter 10 on ‘Key Economic Sectors and Services’ on which Professor Tol was one of the Coordinating Lead Authors. A section had been inserted on ‘Aggregate impacts’ which was based almost entirely on Professor Tol’s 2013 paper. The Chapter also included a new table and graph which were based on Figure 1 and Table 1 from his 2013 paper.

None of this material had been included in the Second Order Draft of the report (a copy of which was also leaked to a blog for climate change ‘sceptics’ that had been made available to reviewers, including me.

Tol’s response implies the section Ward took issue with had been moved but otherwise left relatively unchanged. This could not be further from the truth. I’ll demonstrate. The text for the section Ward complains about (10.9.2, Final Version) was:

10.9.2. Aggregate Impacts

Since AR4, four new estimates of the global aggregate impact on human welfare of moderate climate change were published (Bosello et al., 2012; Maddison and Rehdanz, 2011; Roson and van der Mensbrugghe, 2012), including two estimates for warming greater than 3°C. Estimates agree on the size of the impact (small relative to economic growth) but disagree on the sign (Figure 10-1). Climate change may be beneficial for moderate climate change but turn negative for greater warming. Impacts worsen for larger warming, and estimates diverge. The new estimates have slightly widened the uncertainty about the economic impacts of climate.

Welfare impacts have been estimated with different methods, ranging from expert elicitation to econometric studies and simulation models. Different studies include different aspects of the impacts of climate change, but no estimate is complete; most experts speculate that excluded impacts are on balance negative (Füssel, 2010; Tol, 2008; Yohe, 2008). Estimates across the studies reflect different assumptions about intersectoral, interregional and intertemporal interactions, about adaptation, and about the monetary values of impacts. Aggregate estimates of costs mask significant differences in impacts across sectors, regions, countries and populations. Relative to their income, economic impacts are higher for poorer people.

After this is Figure 10-1:

AR5-10-1

Strangely, despite being entirely about aggregate impacts, this section fails to state what they are. It fails to say anything about what previous IPCC reports concluded on the subject. It fails to explain what has changed or why. It fails to make any effort to quantify uncertainty. It merely describes the conclusions of a handful of papers published in the last few years. In totality, it is two paragraphs spanning half a page with only six references and one figure.

The Second Order Draft of this report did not have a section titled “Aggregate Impacts” in Chapter 10. Richard Tol claims a section from another chapter (19) was merely moved to Chapter 10. The section which was supposedly moved was 19.6.3.5 (available here). The first paragraph of this section is:

19.6.3.5 Aggregate Impacts

The RFC pertaining to aggregate impacts includes risks to society or ecosystems that are aggregated globally into a single metric, such as monetary damages, lives affected, or lives lost, although most aggregations in the literature are carried out in monetary terms. Estimates of the aggregate, economy-wide risks of climate change since AR4 continue to exhibit a low level of agreement. Studies at the sectoral level have been refined with new data and models, and have assessed new sectors.

Already, it is obvious the two sections are different. This paragraph explains what aggregate impacts are and refers to the conclusions of the previous IPCC report regarding them. The differences become more pronounced two paragraphs later when a discussion of species extinctions begins:

AR4 stated with medium confidence that approximately 20-30% of the plant and animal species assessed to date are at increasing risk of extinction as global mean temperatures exceed a warming of 2-3°C above pre-industrial levels (Fischlin et al., 2007). Since AR4 the literature has expanded in many ways with respect to climate change impacts on biodiversity. Firstly, new literature has added to the evidence of increased extinction risk, for example to species in unique and threatened systems (see 19.6.3.2) and in tropical, polar and mountain ecosystems (see 19.6.3.2). Secondly, there are many more detailed studies quantifying extinction risks, of which several include also studies on possible adaptive measures to support conservation (e.g. Hunter et al.2010; Amstrup et al., 2010; Pearman et al., 2011; Lenoir et al., 2008; Balint et al., 2011; Barnosky et al., 2012; Ledit et al., 2012; Norberg et al., 2012; Bellard et al., 2012)….

That paragraph goes on, the next paragraph discusses effects on ecosystems, and a later paragraph discusses the effects of sea level rise. None of these issues were given any attention in the section Bob Ward complained about. Clearly, the section was not merely moved as Tol claims.

However, it is possible part of the section was moved while other parts were discarded (or moved elsewhere). To examine this possibility, we can scroll past a page of text on environmental effects to reach the first discussion of anything covered by the section Ward complained about. At that point, it says:

Assessments of economy-wide consequences of climate change report results either as total damages or as marginal damages, the latter represented by the social cost of carbon (SCC). Estimates of global aggregate impacts from integrated assessment models (Figure 19-7) exhibit a low level of agreement (Ackerman et al., 2011; Anthoff and Tol, 2010b; Bosello et al., 2012; Hope, 2013a, 2013b; Nordhaus, 2008, 2010; Roson and Mensbrugghe, 2012; Waldhoff et al., 2013). Sectoral breakdowns also exhibit a low level of agreement (Figure 19-8) (Anthoff and Tol, 2010b; Anthoff et al., 2013; Nordhaus 2007, 2008; Roson and Mensbrugghe, 2012). There is very high confidence that aggregate estimates of costs mask significant differences in impacts across sectors, regions, countries and populations. For this reason, as well as the existence of only a few studies with sectoral detail that employ alternative development pathways, it is difficult to detect a monotonic relationship between vulnerability and aggregate risks at the global scale. In some locations and amongst some groups of people with high exposure and high vulnerability, net costs per capita will be significantly larger than the global average (Anthoff et al., 2009; Nordhaus, 2011; Warren, 2011).

This paragraph discusses the same topics the section Bob Ward complained about covered. Specifically, it discusses global impacts:

Estimates of global aggregate impacts from integrated assessment models (Figure 19-7) exhibit a low level of agreement

While citing nearly a dozen references. This says there is a low level of agreement on global aggregates. The section Ward complained about says:

Estimates agree on the size of the impact (small relative to economic growth) but disagree on the sign (Figure 10-1). Climate change may be beneficial for moderate climate change but turn negative for greater warming.

While citing only three references. This section states new estimates “agree on the size of” global impacts, a claim not found in the original. It also removes any comment regarding a “low level of agreement.” Finally, it adds the entirely new claim that global warming “may be beneficial for moderate climate change.”

The Second Order Draft discussed uncertainties at length, including multiple paragraphs I haven’t quoted. It goes so far in discussing uncertainties as to say:

it is difficult to detect a monotonic relationship between vulnerability and aggregate risks at the global scale

Yet the Final Version of the report says almost nothing about them. The Second Order Draft section was 12 paragraphs spanning two pages, citing dozens of references. The final version was two paragraphs spanning half a page, citing only six references.

Additionally, as Bob Ward pointed out, the only figure from the Final Version section (Figure 10-1, shown above), was entirely new. Also, as Ward says, it is derived from Richard Tol’s work. We can even see it discussed on Tol’s blog. It is nothing like the two figures included in the original section:

AR5-SOD-19-7

AR5-SOD-19-8

The new section in the final version of the IPCC AR5 is dramatically different from any section in previous drafts of the report (even ignoring non-economic topics). It failed to define the subject it was discussing. It failed to describe the findings of previous reports. It failed to explain what had changed between reports. It removed paragraphs of text discussing uncertainties. It removed dozens of references. It removed two figures. It introduced at least one new claim and an entirely new figure.

Bob Ward was right to be stunned. There was no reason such significant changes should have been made between the Second Order Draft and the Final Version. Not only did they change the focus of the section to promote the work of a Contributing Lead Author of the chapter (Richard Tol), they removed a great deal of information from the report.

Richard Tol’s defense on this point is to say “that section was moved from Chapter 19 to Chapter 10.” This claim is true only in the sense the title of the section was moved from one chapter to another. Practically nothing else was. Tol is aware of this not only because he was a Contributing Lead Author of the new section, but he was also strongly opposed to the original version, submitting a comment which said:

The opening paragraphs of 19.6.3.5 are not about aggregate impacts at all. These are followed by paragraphs stating that you really should not trust these studies. This is most odd. You do not write in the same way about other literatures, even if those papers can be picked apart just as easily or even more so. It is your job to assess the literature, rather than attack it. (Tol, Richard S.J., Vrije Universiteit Amsterdam)

Richard Tol was highly critical of the original section, complaining it focused too much on uncertainties in the literature. Given that, he obviously doesn’t believe the section was merely moved to a different chapter. He is fully aware the section was rewritten to be almost completely different. He is fully aware the new version heavily promoted his own work.

In other words, Richard Tol is fully aware Bob Ward’s reaction to the inclusion of this section in the IPCC report was correct. Everyone should be stunned. By all appearances, this was nothing but a blatant abuse of power by Richard Tol to remove material he disliked while promoting his own work.

Advertisements

24 comments

  1. Funny, you are bemoaning exactly what has been done repeatedly the other way around! Only at a clip of 20 to 1. HOWEVER, I don’t like this kind of machination, but the phrase: “What’s good for the goose is good for the gander.” comes to mind.

  2. Timothy Sorenson, I’m well aware of the history of this being done. I actually pointed it out to several people in e-mails discussing this post. As I said to them, it doesn’t matter if you like the “message.” If it was wrong for people to do it in the past, it’s wrong for anyone to do it now. Wrong it wrong. It doesn’t matter who does it or why.

    The main reason I even bothered to write this post is I’ve been critical of the process which allows this to happen in the past. I’ve criticized the IPCC on the last two assessment reports because of it. This example shows those past criticisms are still valid. The IPCC still has terrible procedures which allow chapter authors to make dramatic, unjustified changes to promote their own views and work. This diminishes the trustworthiness of the report. It also shows claims the IPCC process has improved are wrong, thus showing the IPCC is not going to fix the problems that have existed for at least ten years.

    The weird thing is this example is a great example of why the IPCC assessment reports can’t be relied upon, yet few people seem interested in it. One person I talked to indicated they aren’t interested in finding out if I’m right. Another person defended Richard Tol’s actions. If this had been done by Michael Mann, I bet both would have congratulated me on making a great discovery.

  3. It’s been brought to my attention a couple links don’t seem to work. They apparently don’t work if you click on them, but if you copy the link and paste it in, it should work. I don’t know why that’d happen, but it seems to have something to do with Bishop Hill’s blog (where the documents are hosted). If anyone knows how to fix it, I will. Otherwise, I guess you’ll have to copy and paste the URLs.

  4. There we go. I uploaded copies of the documents whose links were causing problems and changed the links to point to the files on this site. All the links should be fine now. Let me know if there are any more problems with them.

  5. Link to Bishop Hill broken:

    Author of the new section, but he was also strongly opposed to the original version, submitting a comment which said:

    Okay, what is the first issue?

    Sections moved on a draft. Everyone should be shocked, why? Did IPCC attempt to correct AR5 based on the IAC recommendations? Is Richard Tol attempting to fix the grocery list of typos and graphing errors? A document change without a sign-off? After 25 years?

    If Bob missed the notification of the change, I can not even bother to care considering how much of his attention has be devoted to kicking rocks and dust. He has a short attention span.

    What would you like to address first to clarify your horror?

  6. intrepid_wanders, there should no longer be any links to Bishop Hill. I uploaded copies of the documents to this site and linked to them instead. For some reason, clicking on the links to Bishop Hill didn’t work even though the links would work if you copied and pasted them. I think that must be due to some quirk of how Bishop Hill’s server handles the requests, but I’m not sure. Either way, I believe all the links are working now.

    Okay, what is the first issue?

    Sections moved on a draft. Everyone should be shocked, why?

    This is incredibly misleading. The majority of this post is spent explaining how Tol did not merely move a section like he claimed, but actually rewrote it and its conclusions. The only way you could claim the section simply “moved” is to ignore practically everything I wrote in this post.

    Did IPCC attempt to correct AR5 based on the IAC recommendations? Is Richard Tol attempting to fix the grocery list of typos and graphing errors? A document change without a sign-off? After 25 years?

    I can’t see any connection between these questions and anything I wrote in this post. Perhaps because of that, I don’t know what you’re asking. If you want answers, you’ll have to explain what you’re asking.

    If Bob missed the notification of the change, I can not even bother to care considering how much of his attention has be devoted to kicking rocks and dust. He has a short attention span.

    There was no notification of the change. Not only were people not notified of it, when confronted about the fact he had significantly rewritten the section, Richard Tol dissembled by saying “the text was moved rather than added.” The reality is the text was not moved. It had not existed prior to the final version of the report. Nobody signed up to review the report could have known about it.

    What would you like to address first to clarify your horror?

    I’d like you to explain why you said untrue things in response to my accurate statements without even acknowledging what I said. If you think I’ve gotten anything wrong, you should quote what I said and explain how it was wrong. You should not merely make unsubstantiated claims that contradict my well-referenced post.

  7. This is incredibly misleading. The majority of this post is spent explaining how Tol did not merely move a section like he claimed, but actually rewrote it and its conclusions. The only way you could claim the section simply “moved” is to ignore practically everything I wrote in this post.

    If the IPCC does not have these basic document control procedures, it might as well be SkS administrated by Bob C.. All changes must be counter-signed by somebody with oversight. For Tol to move something from Ch12. to Ch10. without a process would be quite a feat. My biggest concern is that you might not have a document control theory down well enough to make these accusations.

    I can’t see any connection between these questions and anything I wrote in this post. Perhaps because of that, I don’t know what you’re asking. If you want answers, you’ll have to explain what you’re asking.

    AND

    There was no notification of the change. Not only were people not notified of it, when confronted about the fact he had significantly rewritten the section, Richard Tol dissembled by saying “the text was moved rather than added.” The reality is the text was not moved. It had not existed prior to the final version of the report. Nobody signed up to review the report could have known about it.

    Tol stated that Ward would have been notified of a change no later than April 2nd. If the IPCC change control system is that weak, it does not reflect on Tol. If Ward is busy with other things to keep his eye on the ball, nobody else is going to help him (except you 🙂 ).

    I’d like you to explain why you said untrue things in response to my accurate statements without even acknowledging what I said. If you think I’ve gotten anything wrong, you should quote what I said and explain how it was wrong. You should not merely make unsubstantiated claims that contradict my well-referenced post.

    What untrue thing. Was I mistaken in understanding that Ward complained about numerous things and then discovered something more interesting with the model projections that 1°C *could* affect Tourism, Skiing and other superfluous entertainment activities?

  8. You are quite right Brandon, if Dr. Tol took out data and conclusions and then re-wrote the section to push his own work then he’s as wrong as the others who “make sure it won’t get in even if we have to change the peer review process”. In many ways the IPCC reports do a monumental job in shifting through the scientific literature, but the process is so opaque that all too many times papers are ignored that don’t suit the political views of the scientists involved – in fact they’re easy to ignore.

    That serial mis-informer Bob Ward a paid PR man for global warming picked up on it is no surprise.

    Maybe we should put the IPCC to bed by giving it the new remit of trying to understand what causes climate change in the round rather than how humans are causing it. As for the chapters on adaptation, mitigation and effects they’re what you might call “soft” science, or even “voodoo” science in that in order to have any value one has to assume the authors can foretell the future. As no one, as far as I’m aware, has involved the engineering profession in figuring out how to adapt, or mitigate, we are left with the WWF and Greenpeace doing it. Bizarre.

  9. The majority now know that man-made climate change is happening. Stop the bickering and accusations and apply your intellect to damage limitation please.

  10. interprid_wanders:

    If the IPCC does not have these basic document control procedures, it might as well be SkS administrated by Bob C.. All changes must be counter-signed by somebody with oversight. For Tol to move something from Ch12. to Ch10. without a process would be quite a feat. My biggest concern is that you might not have a document control theory down well enough to make these accusations.

    That would be Chapter 19, not Chapter 10. Also, I’ve never suggested he moved the s ection without anyone else signing off on it. I presume he convinced the authors of Chapter 19 the discussion of aggregate impacts fit better in his chapter. I’d have no problem with that. I don’t think anyone would.

    The issue of is Whether or not he told people he’d completely rewrite the section is a different story. I’d like to think he didn’t go around telling people that, but I have no idea.

    Tol stated that Ward would have been notified of a change no later than April 2nd. If the IPCC change control system is that weak, it does not reflect on Tol. If Ward is busy with other things to keep his eye on the ball, nobody else is going to help him (except you 🙂 ).

    Richard Tol did not state this. What he said is Bob Ward “was informed no later than 2 April 2014 the text was moved rather than added.” That is, after Ward discovered this issue and complained, he was told the text had been moved. Not only does this not say what you claim it says, it was, for all intents and purposes, completely untrue.

    As for claiming this “does not reflect on Tol,” I have no idea why you think it wouldn’t. It certainly reflects upon the IPCC. I’ve said so. I’ve said this shows the IPCC process is still screwed up. However, it would certainly reflect upon Tol if he chose to exploit a screwed up process to benefit himself.

    What untrue thing. Was I mistaken in understanding that Ward complained about numerous things and then discovered something more interesting with the model projections that 1°C *could* affect Tourism, Skiing and other superfluous entertainment activities?

    I haven’t said anything about whatever you’re talking about, so I have no idea. You keep talking about things nobody is discussing, and I have no idea why.

    What you originally said that is untrue is that the section was moved. It wasn’t in any meaningful way. It’s true the section title was moved from Chapter 19 to Chapter 10, but everything else about the section was rewritten. None of the text or figures are the same.

  11. geronimo:

    In many ways the IPCC reports do a monumental job in shifting through the scientific literature, but the process is so opaque that all too many times papers are ignored that don’t suit the political views of the scientists involved – in fact they’re easy to ignore.

    Definitely. The amount of work done for these reports is impressive. The problem is the process is horribly designed. The people writing chapters can pick and choose what they want to include, and they can just ignore any reviewer who points problems out. That’s silly.

    My favorite example is still the mistake about Himilayan glaciers melting. A lot of people try to play it off as a minor typo, mixing up 2035 and 2350. That’s disingenuous. The real problem isn’t even the mistake. It’s how the the mistake got by. Not only was it taken from grey literature without following proper procedure, the mistake was obvious enough reviewers pointed it out, highlighting the fact it contradicted other parts of the same section. It was ignored.

    A good report on it can be found here. It shows even the most obvious problems can pass the IPCC review process.

    That serial mis-informer Bob Ward a paid PR man for global warming picked up on it is no surprise.

    I’m sure I had heard Bob Ward’s name before, but even now, I can’t think of anything I know I about him. I don’t know whether he’s as bad as you claim or not. I don’t really care either. I’m content to just say he found an important issue we should discuss.

    Maybe we should put the IPCC to bed by giving it the new remit of trying to understand what causes climate change in the round rather than how humans are causing it.

    If you had ended this sentence after the eigth word, I’d agree with you 😛

  12. Is it me, or is that smiley face really creepy? Anyway, I wanted to point out Richard Tol claims my portrayal is inaccurate. On Twitter, he said:

    I’ve provided a link to the reviewer comments on the chapter the section was in. I can find none which support his claim. Am I missing some? Similarly, while he claims the section was “extensively re-reviewed,” it was written only after the Second Order Draft had been reviewed. According to IPCC procedures, there’s no round of expert review after that. All there is is governments reviewing the Summary for Policy Makers. As far as I can tell, the only people who could have reviewed this new section are the authors of the chapter and possibly people higher up in the IPCC chain. Am I missing something? I’ve asked Tol for clarification.

    Also Mike, everyone agrees the climate is changing. The questions are regarding what changes there will be, and how we should handle them. That’s what the section this post covered is about. Examining how conclusions are reached is important.

  13. Actually a pretty tidy documentation procedure:

    http://www.ipcc.ch/pdf/ipcc-principles/ipcc-principles-appendix-a-final.pdf

    4.5 Acceptance of Reports …The content of the authored chapters is the responsibility of the Lead Authors, subject to Working Group or Panel acceptance. Changes (other than grammatical or minor editorial changes) made after acceptance by the Working Group or the Panel shall be those necessary to ensure consistency with the Summary for Policymakers or the Overview Chapter. These changes shall be identified by the Lead Authors in writing and made available to the Panel at the time it is asked to accept the Summary for Policymakers, in case of reports prepared by the Task Force on National Greenhouse Gas Inventories by the end of the Session of the Panel which adopts/accepts the report.

    So, not only is the content the responsibility of the Lead Authors, major changes would need to be submitted to the Session of the Panel for review. Tol and the other Lead Authors would have control of Ch10 but not Ch19.

    5. REVIEW EDITORS

    Function:
    Review Editors will assist the Working Group/Task Force Bureaux in identifying reviewers for
    the expert review process, ensure that all substantive expert and government review
    comments are afforded appropriate consideration, advise lead authors on how to handle
    contentious/controversial issues and ensure genuine controversies are reflected adequately
    in the text of the Report.

    One would expect that any changes and wording would have to go through the Review Editors and conveyed to the appropriate Expert Reviewer.

    I suggest that you attempt to find your irregularity based in the SOP of the IPCC, other than that, I do not know what to say.

  14. intrepid_wanders, if and when the IPCC makes such documentation available, I’ll examine it to see if the changes were reported fairly and correctly. Until then, I’ll examine what we have. And when I do, I find Richard Tol rewrote material to favor his views while removing much content and information. I also find he did so after all expert review had already been done, meaning there was no review of the material outside of IPCC staff.

    If you challenge such conclusions, please explain what you think I got wrong and how.

  15. I’m cross-posting a comment from Bishop Hill. Richard Tol gave some “information” in a comment there, and it appears that “information” is false. I don’t know how to link to a specific comment at that site, but you can find it on this page at Apr 19, 2014 at 6:49 PM. My response to it is as follows:

    Alright. I’m home so I can respond with more detail. Richard Tol claims:

    This bit of Chapter 19 was substantially rewritten and became nonsensical. For instance, a figure was included to show the impacts of climate change on different economic sectors according to DICE, a one-sector model.

    The figure in question is 19-8, and if you listen to Tol here, you’d think the figure only includes information from the DICE model. In actuality, it lists three different models, DICE 2007, ENVISAGE and FUND 3.8.

    The next thing to note is Tol claims DICE is a one-sector model, thus it is nonsensical to claim the model shows damages across various sectors. This is a strange claim. If DICE couldn’t show damages across multiple sectors, how did the IPCC get values from DICE for damages across multiple sectors? Unless Tol wishes to claim the data used in the Figure 19-8 was fabricated, I see no way for him to make this argument.

    As best I can tell, Tol’s argument is simply misleading people via semantics. IT’s thinking of sectors in models as “boxes.” A one-sector model would be a one-box model. One-box models can have many outputs. A one box climatic model could have temperature, pressure and volume as output. Similarly, a one box economic model could have damages to agriculture, energy supply, coastal property, labor force, ecosystems and many more thing as outputs.

    At least, that’s the best interpretation I could come up with. Regardless, one box/sector models can have submodels, and if we check documentation for the DICE model (such as here), we see mentions of many different sectors (other documentation gives more specific listings of such sectors). That, combined with the fact the IPCC had data from the DICE model on damages for various economic sectors should be enough for people to say Tol’s claim is wrong.

    The comments on the Second Order Draft called for a complete revision, so that’s what was done.

    I asked Tol what comments the changes were supposedly made in response to. He hasn’t answered. However, I’ve read every review comment on the chapter, and I’m confident none “called for a complete revision” of this section. There were a few general comments suggesting tightening up language/condensing material within the chapter to save space, but that’s about it.

    Comments specific to the section are #770-#794. Three are from Tol himself, and they obviously don’t count as showing what other reviewers thought. Seventeen were editorial comments or generic suggestions about making things more cohesive (making things mesh with other chapters). One asked for more discussion of uncertainty. Another suggested one paragraph be moved to another chapter. Another suggested a paragraph about biodiversity/species extinction was redundant.

    That leaves comments #770 and #772. Commenter @770 said he likes the section but suggests some text might be condensed (but finds no specific recommendations). #772 is the only comment one could argue calls for any significant revision. Specifically, it does suggest economic and ecological damages be separated. That single suggestion is the only suggestion that supports Tol’s claim in the slightest.

    But even comment #772 doesn’t support Tol’s claim. #772 called for the section to be separated into two components, which it was. It did not, however, call for the material discussing economic damages be rewritten. Had Tol merely moved the discussion of economic damages to a different chapter (thus separating economic/ecological issues), I wouldn’t have commented. That’d be a reasonable thing to do, and one reviewer did suggest it’d be done.

    However, as far as I can see, none of the expert reviewers “called for a complete revision.” If Tol or anyone else thinks otherwise, they ought to cite the comment numbers which support their claim. As it stands, it appears he has simply made this up.

  16. Brandon, first a nit-pick regarding the text of your post. You claimed that Richard Tol is/was a “Contributing Lead Author”; however, in the official pantheon of the IPCC writing hierarchy there is no such role. This may well have been the only thing that Bob Ward – a PR hack who has never been known for truth in posting or tweeting – actually got right: Tol is/was (one of two) “Coordinating Lead Authors” of WGII’s Chapter 10.

    Because I’ve always considered myself to be graphically and iconically challenged, I will readily admit that I pay little attention – or give much credence – to the pretty picture a graph might convey. This being the case, I could be mistaken – and I do stand to be corrected – but my impression from your post is that much of your concern regarding Tol’s alleged ‘rewrite citing his own work’ hangs not so much on the difference in the text as it originally appeared in Chapter 19 vs that which appears in Chapter 10, but on the graph which derives from Tol’s own work.

    I don’t dispute that there is considerable difference in the text that appeared in the SOD (Chapter 19) and that which appears in the transfer to the “final” version in Chapter 10. But I’m not entirely sure that this would have been Tol’s doing. Nor do I see in the text a preponderance of citations to Tol’s own work.

    Let us not forget Muir Russell’s “conclusion” that neither Jones nor Briffa could possibly have influenced the final text of the chapters of AR 4 for which they were Coordinating Lead Authors because decisions are a “team” effort; not to mention the rather incongruous but evidently longstanding “page limits” for a document that purports to “inform” decisions/policies on which the future of the planet will depend.

    It seems to me that perhaps some consideration should be given to the – IMHO, little known and rarely publicized – role of the IPCC Bureau, i.e. that of “Editorial Board” [Pls. see IPCC’s makeover result: Business as usual]. To my mind, it is not beyond the realm of possibility that whatever text remains extant in that which was moved from Chapter 19 to Chapter 10 could well be (at the very least partially) the work of this “Editorial Board”, rather than an independent “rewrite” by Richard Tol.

    But, as noted above, I could be entirely mistaken 😉

  17. Hilary Ostrov, thanks for the correction on terminology. I saw “Contributing Authors” then somehow read “Coordinating Lead Authors” as “Contributing Lead Authors.” It’s a silly mistake. I’ll try to use the correct terminology from now on, but I’m not going to edit my post this long after it went up. I’ll let the mistake stand. People can laugh at me for it.

    On the issue of text vs. graphical changes, you’re right the graphics are where Tol’s work gets heavy promotion (by removing two graphics and replacing it with one of his own). However, the issue here is not merely one of textual citations. There’s also the issue of viewpoints. The most obvious example is the final version of the section says:

    Climate change may be beneficial for moderate climate change but turn negative for greater warming.

    This is a position Richard Tol has repeatedly advanced in his papers and public outreach, and it is one absent from the original section. The closest reference to it directs one to Figure 10-1 (Tol’s figure). That means this claim is new, belongs to him and relies upon his own work. Changes like this mean even without heavily citing his work in the text, the section’s tenor has been changed to match his viewpoints pretty much perfectly.

    Additionally, there’s an odd issue with this figure. The sentence directing the reader to it claims the four estimates on economic impacts made since AR4 “agree on the size of the impact… but disagree on the sign (Figure 10-1).” Figure 10-1 seems to contradict this claim. It shows four diamonds (indicating new estimates), but all four are negative. That means they all agree on the sign, meaning they all agree the impact will be negative.

    For better information, we can look at Table 10.B.1 which Figure 10-1 references for its data. The four new impact estimates listed in it are -11.5, -0.5, -1.8, -4.6. This shows the text is wrong to say the new estimates disagree on sign. It also makes it impossible for me to see how the text can claim the new estimates “agree on the size of the impact.” -4.6 is far larger than two of the values, and -11.5 is the most extreme outlier of all the data points.

    I think I know how at least part of that happened, and if I’m right, it’s tied directly to a mistake Tol made in his papers. If so, that makes it difficult to claim anyone else is responsible for these changes.

    That said, I acknowledge that without access to communication/underlying documentation, I cannot know for certain Tol made these changes. I tried to indicate this by using phrasings like “he’s apparently” responsible. I might have failed to make it clear though.

  18. It turns out I was wrong about that figure. I misread these two sentences:

    Since AR4, four new estimates of the global aggregate impact on human welfare of moderate climate change were published (Bosello et al., 2012; Maddison and Rehdanz, 2011; Roson and van der Mensbrugghe, 2012), including two estimates for warming greater than 3°C. Estimates agree on the size of the impact (small relative to economic growth) but disagree on the sign (Figure 10-1).

    I thought these sentences referred to the same thing. As in, I thought the second sentence referred to the “four new estimates” mentioned in the sentence before. I no longer think so. The next sentence refers to estimates, not “these estimates.” That means the paragraph brings up four new estimates then says nothing about what those estimates are. It instead discusses all 20 estimates used in Figure 10-1. That’s a strange way to write a paragraph. You have to be paying close attention to catch what was done.

    Having caught onto what was done, it is easy to check to see why the section could say that. There are 20 impact estimates in Table 10.B.1 (and thus Figure 10-1). Seventeen are negative. One is 0. Two are positive. They are:

    (Mendelsohn et al. 2000)a: 0.1
    (Tol 2002): 2.3

    This should give everyone pause. The section explicitly refers to four new estimates, then it immediately discusses conclusions which depend upon papers from more than 10 years ago. It’s difficult to see why the IPCC would still rely upon decade old results to draw its conclusions. There are ten estimates listed newer than those two, all of which are negative. It’s only by going back 10+ years we can find any estimates which are positive.

    As though that’s not bad enough, footnote a is striking:

    a Results aggregated by (Tol 2013).

    One of the two positive estimates is from a 12 year old paper Richard Tol wrote. The other is a value Tol calculated based upon someone else’s work. That means the section was moved to the chapter Tol helped head, underwent a major revision then suddenly made a claim that rests entirely upon Tol and work done over a decade ago. And naturally, this claim is one Tol has made time and time again, both in his published papers and public relation efforts.

    I’d say Bob Ward was right to be stunned.

  19. Brandon,

    since yesterday, I’ve taken a look at these documents. I agree with some of your points, but disagree on others. I am not in a position to comment on facts and details of Tol v Ward, as I haven’t parsed the underlying documents (and do not plan to do so right now), but I do wish to comment on IPCC procedures and Tol’s compliance with these procedures.

    Obviously, I’ve been very critical of IPCC procedures in which Lead Authors can, seemingly on their own, make change assessments in the Final Draft from the assessment as submitted to external reviewers in the Second Draft. Such latitude means that assertions in the Final Draft have not necessarily received the thorough review that IPCC routinely claims for its process. On this point, I believe that we are in complete agreement.

    In your article, you discuss the evolution of the text on economic impacts that was in section 7.6.3.5 of Chapter 19 of the Second Draft and say that the changes were “By all appearances, … nothing but a blatant abuse of power by Richard Tol to remove material he disliked while promoting his own work.”

    In the Final Draft, discussion of economic impacts was entirely removed from section 7.6.3.5 of Chapter 19. Instead, discussion of this topic was transferred to Chapter 10 (where Tol was a Coordinating Lead Author), where the assessment was completely re-written, including a reduction in length. In respect to the transfer of responsibility for this section from Chapter 19 to Chapter 10, I do not believe that Tol, as Chapter 10 Coordinating Lead Author, had the authority to change or delete material from Chapter 19. So the changes to Chapter 19 could not be an “abuse of power” by Tol, let alone a blatant one. My surmise is that the decision to move the discussion from Chapter 19 to Chapter 10 was probably made by one of the Co-Chairs of WG2. Perhaps Tol suggested the move, perhaps not. But that aspect isn’t an “abuse of power”.

    As you observe, the movement to Chapter 10 was not limited to a cut-and-paste; instead, Chapter 10 authors re-wrote the section for the Final Draft. Thus, the material in the final assessment had never been submitted for external review. This happens all the time with IPCC and has been a long-standing point of contention. In my own case, Briffa changed the assessment of the Hockey Stick dispute in the Final Draft from that submitted to external reviewers in the Second Draft – a change that I disputed, especially when the Climategate correspondence showed that the changes had almost certainly been made in surreptitious correspondence with Eugene Wahl. As Hilary pointed out, Muir Russell considered these events and (for what it’s worth) concluded that Briffa did not commit any abuse in making changes to the Final Draft.

    I entirely agree with your expectation of consistency as an antidote to hypocrisy. If it’s wrong for Briffa to unilaterally change assessments in the Final Draft, then the same applies to Tol. And vice versa. On the other hand, I don’t see any evidence that Tol failed to comply with IPCC procedures or “abused” IPCC procedures. It seems odd and unsatisfactory that assessments in the Final Draft can be different from those presented to reviewers, but Tol’s case is hardly unusual.

    Returning to Bob Ward: Bob Ward says that he was “stunned” when he read the WG2 Final assessment of economic impacts. Well, so I was when I read the AR4 assessment of the Hockey Stick. If Ward’s conclusion is that such assessments are worthless, I agree. (This doesn’t settle either side of the underlying dispute – it only means that no weight can be attributed to the assessment.)

  20. Steve McIntyre, I’m not sure we disagree like you think. I don’t believe the decision to move material was an abuse of power. I don’t think I’ve ever said it was. It’s possible I’ve slipped up somewhere, but on multiple occasions, I’ve said the move itself doesn’t bother me. For instance, over at Bishop Hill, I said:

    Had Tol merely moved the discussion of economic damages to a different chapter (thus separating economic/ecological issues), I wouldn’t have commented. That’d be a reasonable thing to do, and one reviewer did suggest it’d be done.

    I’m sure Tol couldn’t make such a decision on his own, so maybe I shouldn’t have said “Tol” there, but it shows my concern isn’t with the move itself. My concern is with the significant revisions, an entirely new section and multiple new figures and tables which serve only to promote Richard Tol and his views. Additionally, I’m concerned by Tol’s untrue responses to this issue. For just one example, when Bob Ward brought this up, Tol said:

    In fact, that section was moved from Chapter 19 to Chapter 10. As far as I am aware, Mr Ward did not raise this concern with the IPCC. He was informed no later than 2 April 2014 that the text was moved rather than added.

    Tol downplayed what happened by claiming the text was merely moved. It wasn’t, something he has since acknowledged. Tol giving false information would be bad enough, but he tried to portray Ward as being dishonest based upon that false information. That is horribly bad.

    Anyway, you say:

    As Hilary pointed out, Muir Russell considered these events and (for what it’s worth) concluded that Briffa did not commit any abuse in making changes to the Final Draft.

    I entirely agree with your expectation of consistency as an antidote to hypocrisy. If it’s wrong for Briffa to unilaterally change assessments in the Final Draft, then the same applies to Tol. And vice versa. On the other hand, I don’t see any evidence that Tol failed to comply with IPCC procedures or “abused” IPCC procedures. It seems odd and unsatisfactory that assessments in the Final Draft can be different from those presented to reviewers, but Tol’s case is hardly unusual.

    I can accept the possibility none of this reflects actual abuse of the IPCC process. I don’t know what the exact rules are for what changes CLAs are allowed to make and when (assuming such rules even exist). I’m simply going off what the IPCC says in public. It claims its reports are thoroughly reviewed by external reviewers. That clearly didn’t happen here. As such, I say the process they claim to use has been abused.

    The other possibility is the IPCC makes false statements about its process. I think that’s even worse than what I describe. When confronted with two interpretations, I always try to assume upon the most generous of the two.

    In other words, I say the IPCC process is abused because the only other interpretation I can come up with is the IPCC process is worthless.

  21. Brandon, you say:

    The other possibility is the IPCC makes false statements about its process. I think that’s even worse than what I describe. When confronted with two interpretations, I always try to assume upon the most generous of the two.

    Sometimes, it’s impossible to make such a decision without being less generous to one of the parties to the dispute.
    In this particular case, by choosing the interpretation that is most generous to IPCC, you’ve chosen the interpretation that is least generous to Tol. I think that you have seriously under-estimated the prevalence of unilateral Lead Author changes in the Final Draft – a practice that you and I may object to, but which nonetheless exists independently of Tol. Nor should you take public statements by IPCC about their process at face value. There has been substantial criticism over the years about the ability of Lead Authors to make unilateral changes and it seems unreasonable to me to single Tol out as an “abuser”. This does not imply that I agree with his changes or views or that I think that his edits were praiseworthy because they supported a less alarmist narrative.

    I agree with you that there has to be consistency. However consistency means that if Bob Ward has a valid objection to Tol’s changes under IPCC policies, then there are equally valid objections to numerous other final draft changes in assessment, including my objection to Briffa’s changes.

    In fact, let me tell you about an even more bizarre incident involving WG2. According to one source who had been a Lead Author with AR4 WG2 (and this information came to me several years ago), in the final version of AR4 WG2, the IPCC TSU made changes to several chapters without notifying or obtaining consent from chapter Coordinating Lead Authors. They inserted references to the Stern Report, a document that had been released too late to meet WG2 deadlines, but which supported a more alarmist narrative.

  22. Steve McIntyre, I disagree with this:

    by choosing the interpretation that is most generous to IPCC, you’ve chosen the interpretation that is least generous to Tol.

    Even when a bad practice is tacitly accepted, it can still be abused. You’ll see this in food businesses where employees aren’t supposed to get free food, but everyone has a little snack from time to time. Nobody minds that, but then someone comes along and takes three full meals a day. The people who had been snacking get upset, saying the new guy abused the process.

    The size and overtness of Richard Tol’s changes are enough to justify calling what he did an abuse, even if the general practice were accepted. But beyond that, you say:

    I think that you have seriously under-estimated the prevalence of unilateral Lead Author changes in the Final Draft – a practice that you and I may object to, but which nonetheless exists independently of Tol. Nor should you take public statements by IPCC about their process at face value. There has been substantial criticism over the years about the ability of Lead Authors to make unilateral changes and it seems unreasonable to me to single Tol out as an “abuser”.

    Why? People responsible for entire chapters of the IPCC report know fully well what the IPCC’s public portrayal of its review process is. That means they know fully well what they’re doing when they subvert it. Why shouldn’t we single out people we know willfully subvert the process as “abusers”?

    People are responsible for their actions. Maybe all CLAs have the same opportunity to make these sorts of changes, but most don’t do it. Those who do deserve to be singled out. If they don’t like being labeled “abusers,” maybe they should have thought about that before abusing the review process.

    I agree with you that there has to be consistency. However consistency means that if Bob Ward has a valid objection to Tol’s changes under IPCC policies, then there are equally valid objections to numerous other final draft changes in assessment, including my objection to Briffa’s changes.

    I’d happily say Keith Briffa abused/subverted the IPCC process. I have before. The things I said about what he did are exactly in line with what I say about what Richard Tol did. When talking to people about what Tol did, I drew parallels to what Briffa did. I said what Tol did ought to be used as proof what people said about AR4 is still true in AR5.

    I’m all for being consistent. I just don’t know why talking about one example should require I talk about another.

  23. “I just don’t know why talking about one example should require I talk about another”

    The process of selecting which examples to talk about goes to the issue of whether you have a bias against Richard Tol.

    This is not to say that your points are incorrect – just that by focusing on his transgressions without applying the same diligence to the same transgressions of others indicates a bias against Tol for whatever reason.

  24. Joe Goodacre, that would be a fair comment if I did focus on Richard Tol’s “transgressions without applying the same diligence to the same transgressions of others.” I haven’t. I’ve been critical of abuses of the IPCC process in both of the previous IPCC reports. Several times when talking about what Tol did in this report, I referenced the examples Steve McIntyre has brought up. In fact, one of the main things I’ve argued is people should pay attention to what Tol did because it shows the problems they’ve talked about in the past are still present.

    I struggle to see how anyone could view this as a bias against Richard Tol. It’s true I’ve written far more about his abuses than others’, but that’s hardly surprising given I didn’t have a blog when the others were discovered. Moreover, lots of people immediately discussed the other examples. When there are lots of people discussing something, there is less need for one person to say much about it.

    I can’t think of any example of inappropriate behavior tied to the IPCC I’ve defended, downplayed or even just ignored. I think there are few people who can say the same.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s