Arbitrary Aggregations

Should the IPCC be allowed to change values based entirely upon arbitrary, and undisclosed, assumptions? I don’t think so. My recent post discussing undisclosed changes to the latest IPCC report discussed how several changes were made to values which had the note, “Results aggregated by Tol (2013).” A question I raised at the time is, how can the IPCC change values yet claim to have gotten them from a source which lists the original values?

Today I’m going to pursue that question. Before I begin though, I want to discuss a little history. The results in question were collected by Richard Tol and published in a number of papers. These results were used by Tol to claim global warming will be beneficial for up to ~2C of warming, a claim which got quite a bit of air in the media. This claim was eventually challenged by Bob Ward who pointed out a number of errors in the values listed by Tol (see here for a list).

There were a series of back and forths as Ward tried to get various errors corrected. During this time, Ward explains he:

drew the attention of each editor to the problems. I also noted that many of the data that had been plotted by Professor Tol were aggregations that he had made of other authors’ research findings. For instance, 8 of the 17 data used in the 2013 paper had been aggregated by Professor Tol. I suggested to the editors that, given the other mistakes, Professor Tol should make available the details of his calculations so that they might also be verified.

Tol refused to provide his calculations, mocking anyone who asked to see them. For instance, when one called for Tol to show his work, Tol responded:

Finally, Professor Abraham repeats Mr Ward’s lament about lack of transparency. In fact, all data are in the public domain. It does take a bit of multiplication, addition and division to go from disaggregate data to aggregate data, but that should not be beyond someone with a bachelor’s degree in geology.

That’s where things stood for quite a while. Then, Tol’s work got slipped into the latest IPCC report. Ward complained about various errors in that work and Tol not having published his calculations. In what appears to have been a response to Ward’s complaint, the IPCC changed many of the values and published a set of calculations.

That’s where we come in. We now have Richard Tol’s calculations, and everything should be resolved. But it’s not. You see, those calculations do not match the results published by Richard Tol. Here is the IPCC’s table of values:

10-17-Figure10-Table

Here is the table published in the Tol (2013) paper it claims to take those values from:

10-17-Tol-corrected_table

There are a number of differences. The two I’ll be focusing on today are the Hope 2006 and Maddison and Rehdanz 2011 entries. Before moving on, I want to point out the change in the Hope 2006 from 0.9 to -0.9 was published by Tol in a correction to his paper. The range was also changed to -0.2 to -2.7. The new change introduced by calculations uploaded by the IPCC is one end of the range went from -0.2 to 0.0.

That’s not a big change, but the fact one can get different results at different times shows the results are somewhat arbitrary. That shows Tol should have published his calculations from the start rather than mocking the people who wanted to check them. More importantly though, it raises the question of, how did Tol generate his results?

To answer this, we can turn to Hope 2006. To estimate damages from global warming, Hope estimated damages to the European Union. He then multiplied the %damage by different scaling factors for different parts of the globe to get a %impact for each area. We can multiply that by each area’s total GDP to get an estimate of the total impact for that area. Richard Tol did so, using the equation:

Impact  = GDP * Scaling Factor * %Damage to the EU

Having done that, all one needs to do is add up the results for each area and divide by the total GDP. Here is it laid out:

Area		          Damage to EU	GDP 	Scaling Factor (mean)	Impact
European Union			1.23%	13705	        1.00	          168.57
USA					13095	        0.25	          40.27
OECD except EU and USA			7358	        0.25	          22.63
Former Soviet Union and Eastern Europe	1919	       -0.35	         -8.26
China and Centrally Planned Asia	2453	        0.20	          6.03
India and Southeast Asia		1958	        2.50	          60.19
Africa and the Middle East		2579	        1.83	          58.04
Latin America				2836	        1.83	          63.83
Sum:				        45900.99		          411.29
				
			%Economic Damage:	0.9

It seems pretty simple and straightforward. It almost seems like there should be no room for disagreement. Only, there’s a problem. Where did I get those GDP figures from? Tol lists them as from “GDP (World Bank)” for the year 2005. Why 2005? I’m not sure. You might think it’s because 2005 is the year before Hope 2006 was published, but that’s not why. Tol uses the exact same data for his calculations with Plambeck and Hope 1996.

Why does Tol use the exact same economic data from 2005 for calculations for 1996 and 2006 estimates? I don’t know. What I do know is it’s not what either of those papers used. I don’t think I need to explain how we know Plambeck and Hope 1996 didn’t use this data. I will explain for Hope 2006 though. You see, Hope 2006 published the parameters it used for its model in an appendix. One table in that appendix is:

10-21-Hope-GDP

There’s a column for GDP plain as day. The values in it are notably different from Tol’s. Here are the two side by side, with the ratio between them displayed:

Hope	Tol	Ratio
8760	13705	0.64
9640	13095	0.74
6570	7358	0.89
2630	1919	1.37
5260	2453	2.14
4380	1958	2.24
3070	2579	1.19
3500	2836	1.23

The differences are not insignificant. Obviously we should plug them into the previous equation and see what’s different. When we do, we find the central value has only changed from -.9% to -1%. That’s small. On the other hand, the previous range of 0 to -2.7% changes to 0 to -3.3%. That’s an increase of ~20%, caused solely by using the data the authors of the paper used.

Why did Richard Tol use 2005 GDP data from one particular source? We may never know. Why did he use it instead of the data the authors provided? I wouldn’t care to guess Why did he use it in calculations for similarly calculations in the Plambeck and Hope 1996 paper, published 10 years before that data existed? I shudder to imagine.

At this point, all I know is by arbitrarily chosing that particular data set rather than the data used by the authors themselves, Tol changed the results. And had he chosen a different data set, he would have changed the results in a different way.


The second difference I want to focus on has a much larger effect. We can see the importance of this one by looking at Tol’s data plotted by the IPCC:

One of the data points is a clear outlier. The point at the bottom of the graph, the point for Maddison and Rehdanz (2011), is nothing like any of the other values. As it is such an extreme outlier, it makes sense to investigate why it is so different. In doing so, the first thing we should note is this value was originally given as -11.5. In the final version of the IPCC report, it was given as -12.4. Both values are listed as having been aggregated by Tol (2013) even though Tol (2013) only lists -11.5.

The next thing we should note is once again, the calculations provided by the IPCC use World Bank 2005 data. However, this time the calculations do not use the World Bank’s GDP data. Instead, they only use World Bank’s population data. The column for this is multipled by a column titled “GDP/cap (PPP) (CIA World Fact Book).”

It’s strange to see GDP per capita, or GDP/Population, multiplied by Population. It seems a backwards way of simply calculating the GDP for an area. That’s not the case here though. The key here is what is in the parentheses: PPP. PPP stands for “purchasing power parity.” This is a different type of GDP than is normally used, what is called “nominal GDP.”

GDP the total measure of the value of everything a country makes. Nominal GDP, which is what most people are exposed to, is simply that value in terms of the worldwide economy. PPP is that value in relative terms. Everyone knows when you travel to a different area, you’ll find some products there are cheaper. PPP tries to account for things like that. Nominal GDP does not.

It’s important to stress neither measure of GDP is “right.” It is fine to use either measurement. It is just not fine to conflate the two. The difference between nominal and PPP GDP is far greater in third world countries than it is in first world countries. That means anything which affects developing nations more than developed nations, such as global warming is said to, will appear worse by PPP standards than by nominal standards.

You can probably guess what that means. There are 20 point in the figure published by the IPCC. 19 are estimates of damage to nominal GDP. One is an estimate of PPP GDP. That one is an extreme outlier. How much do you want to bet it is an outlier because it uses a different measurement of GDP?

It’s at this point I’d normally show you the calculations as published by the IPCC alongside the calculations with nominal GDP. It’s easy to find the data. The World Bank has a web page with the nominal GDP of each country for each year. The problem is Maddison and Rehdanz examined 85 different countries. That’s too many to post. Instead, I’ll just post a screenshot of the spreadsheet with the IPCC calculation and my modified result:

10-21-M&R-Nominal

That’s right. The value the IPCC changed from -11.5% to -12.4% falls to -4.7% if we use the same data said to have been used in all of the other calculations attributed to Tol (2013).

I get Maddison and Rehdanz used PPP GDP in their paper, but that’s no excuse. Either the aggregation calculations should have used nominal GDP, or the results shouldn’t have been included. It’s wrong to conflate two different things (nominal vs PPP GDP) in the same figure.

And it’s stupid to do so when it does nothing but introduce an extreme outlier.

Advertisements

2 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s