As you may know, John Cook and others of Skeptical Science website published a paper finding a 97% consensus on global warming. They then refused to release data related to their work. I eventually came into possession of that data, as well as other material. When I contacted John Cook to discuss what should be done with that data, he refused any sort of dialogue. Instead, he had the University of Queensland make absurd legal threats (including threatening to sue me if I told people they had threatened to sue me) then broke off all communication.
Because of that and other reasons, I decided to release the material I came across. I explained my reasons here. I won’t repeat myself. Instead, I am going to just discuss the material I’m releasing. You can find that material here.
When you click on the link, you’ll be directed to a “mirror” of what I stumbled upon. Not only will you be able to see the data I found, you will be able to see it in the same form I found it (save some technical limitations). You’ll even find the same display settings I saw.
Upon accessing the mirror, you will find approximately 40 links. Some are uninteresting. Some don’t even have any material in them. That was true when I accessed them. The links I find most interesting are these two.
The first link shows the datestamps of ratings performed for this paper. This link is interesting as John Cook and associates explicitly denied having recorded timestamps, saying:
Timestamps for the ratings were not collected, and the information would be irrelevant.
While Cook responded to a criticism from Professor Richard Tol by telling a representative of the University Queensland the journal which published his paper (Environmental Research Letters):
said I didn’t have to include time stamp info but I’m probably going to anyway, just to show Tol’s fatigue theory is all rubbish.
Timestamps and datestamps are different things. It’s impossible to tell which of John Cook’s statements were true. Still, it’s interesting to see datestamps were recorded.
The second link makes these datestamps more interesting. It shows the rater ID# for various ratings. This allows us to tell who performed each rating, and how. Combining the data of the two links allows us to see some individuals rated over 200 abstracts on individual days. Cook et al may feel that is “irrelevant,” but others are certainly free to disagree. Additionally, that second link is what allowed me to create this image:
Showing how the various raters rated the abstracts. Each column shows a value an abstract could be given. Each color represents an individual rater. The vertical alignment shows how often (proportionally) each individual rater selected a particular value. The size of the circles indicate how many times (total) each individual rater chose a given value.
It’s a bit complicated, but the basic point is the more raters agreed about how things should be rated, the more the circles would align. Where the circles do not align, there was a systematic difference in opinion. Such a systematic difference would necessarily introduce bias in the results. The effect of that bias would be directly proportional to the amount of ratings done by the biased individual – represented by the size of the circle.
These biases are especially interesting as all raters were Skeptical Science representatives. They are all advocates for the global warming movement. If we know there were sysetmatic disagreements amongst people who share the same cause, we can only imagine what disagreements there would be if neutral raters had been used.
Another link I especially recommend examining is this one. It will take you to an index of 21 data files, one for each year from 1991-2011. These files include much of the same data linked to above, but they also include comments left by raters during the rating process. Unfortunately, raters could modify their comments at any time, and only the latest version is available. It still gives us insight into why raters selected the values they selected.
In addition to the material I’m now releasing, you may find it worth examining a copy of the Skeptical Science forum released by a hacker while this project was underway. I’ve made an easily accessible version of it available online here. It even includes a subforum specifically created for this paper (direct link here).
You may also find the ethics approval for the paper interesting. It’s available here. Of particular note, this ethics approval was submitted only after the Skeptical Science group had finished their ratings. It does not cover the bulk of their paper. Instead, it only covers John Cook’s requests to individual scientists to rate their own papers. The Skeptical Science group ratings were done without any oversight or ethical approval.
As a final note, I believe the legal threats made regarding my posession and release of this material are baseless. I do not expect any legal action to be taken. I certainly do not expect to be arrested like John Cook and the University of Queensland threatened.
I could be wrong though. Frivolous lawsuits do happen. Law enforcement agencies do sometimes persecute people without cause. At a number of people’s request, I’ve created a legal fund to address such concerns. You can donate to it, or to simply offer support against bullying designed to prevent people from examining data for Cook et al’s work, here:
Any money not needed for legal purposes will be spent according to the wishes of those who donate.