Hey guys. I believe the performance issues of the site for my TCP Reanalysis project is on should be fixed now. With that taken care of, I figure now would be a good time to discuss the sorts of results this project can generate.
For one example, we can see raters rated two papers in opposite ways. Here is the summary of endorsement ratings for two papers (lower value indicating stronger endorsement of the “consensus”):
id 1 2 3 4 5 7 0 1 3 1 0 38 0 2 4 1 0
And quantification ratings:
id 1 2 3 7 0 4 1 38 0 7 0
As you can see, the raters don’t agree. Both papers were rated as endorsing and rejecting the greenhouse effect (with one also being rated as minimizing the effect). To see which rating is right, we can look at the abstracts. Number seven, Aerosol Size Confines Climate Response To Volcanic Super-eruptions, says:
Extremely large volcanic eruptions have been linked to global climate change, biotic turnover, and, for the Younger Toba Tuff (YTT) eruption 74,000 years ago, near-extinction of modern humans. One of the largest uncertainties of the climate effects involves evolution and growth of aerosol particles. A huge atmospheric concentration of sulfate causes higher collision rates, larger particle sizes, and rapid fall out, which in turn greatly affects radiative feedbacks. We address this key process by incorporating the effects of aerosol microphysical processes into an Earth System Model. The temperature response is shorter (9-10 years) and three times weaker (-3.5 K at maximum globally) than estimated before, although cooling could still have reached -12 K in some midlatitude continental regions after one year. The smaller response, plus its geographic patchiness, suggests that most biota may have escaped threshold extinction pressures from the eruption.
As far as I can see, the abstract neither endorses nor rejects the greenhouse effect. I don’t see any reference to it. To check my intepretation, here are three comments left by raters:
Temperature changes linked to volcanic eruptions.
It’s all about volcanos
Modeling study. If the Earth system is less responsive to aerosols, then climate sensitivity may be lower also.
I agree with each of those comments, but I don’t believe any of them make a case for rating the paper as anything other than neutral. I’m sure the paper itself takes a position, but I don’t think the abstract gives us enough information to be sure what position that is.
Abstract 38 says:
Conspicuous global stable carbon isotope excursions that are recorded in marine sedimentary rocks of Phanerozoic age and were associated with major extinctions have generally paralleled global stable oxygen isotope excursions. All of these phenomena are therefore likely to share a common origin through global climate change. Exceptional patterns for carbon isotope excursions resulted from massive carbon burial during warm intervals of widespread marine anoxic conditions. The many carbon isotope excursions that parallel those for oxygen isotopes can to a large degree be accounted for by the Q10 pattern of respiration for bacteria: As temperature changed along continental margins, where similar to 90% of marine carbon burial occurs today, rates of remineralization of isotopically light carbon must have changed exponentially. This would have reduced organic carbon burial during global warming and increased it during global cooling. Also contributing to the delta(13)C excursions have been release and uptake of methane by clathrates, the positive correlation between temperature and degree of fractionation of carbon isotopes by phytoplankton at temperatures below similar to 15 degrees, and increased phytoplankton productivity during ice-house conditions. The Q10 pattern for bacteria and climate-related changes in clathrate volume represent positive feedbacks for climate change.
One rater left this comment:
Assumes significant warming from CO2 and that increased CO2 and CH4 during warm periods creates a positive feedback.
Unfortunately, the rater who rated the abstract as rejecting the consensus didn’t leave an explanation. At the moment, I’m inclined to agree with the comment above. I could be wrong about either abstract though. With this system, we can look at the disagreements for ourselves and discuss them.
We can look at other things as well. For example, here are three different raters who have notably different rating patterns (rater ids removed):
1 2 3 4 6 5 2 0 0 0 15 0 6 6 8 0
The differences might change as the sample size increases, but this shows how easy it can be to check for systematic differences. If there was more data available right now, I’d even be able to make graphs showing the influence of each rater.
And all that can be done with little effort. With more effort, it could even be made automated and integrated into a web page. I think that’d be pretty cool.
In other news, I’m considering changing the sample set available for rating later today. I’m thinking it’d be more interesting if I only offered papers rated by Cook et al as taking a position. That would prevent people from having to read so many papers which say nothing relevant. What do you guys think?
Also, if I do that, should I leave this set available too? I could leave this up as a demo set for people to play with/practice on/discuss standards with. It might be useful to have something like that when designing guidelines for this.