Scientific Literacy and the Big Bang

Did you know the universe began with a huge explosion? If not, you’re an idiot. So says Dan M. Kahan. He has sought to measure people’s “Ordinary Science Intelligence” by asking questions like:

According to astronomers, the universe began with a huge explosion. (True/false)

And expecting people to give the idiotic answer of: True.

It’s amazing how many things are wrong with this question. As anyone who knows anything about the Big Bang theory can tell you, there was no explosion. The Big Bang theory says the entire universe existed in a single point of infinite density approximately 14 billion years ago.* The universe then expanded from this single point and began growing in size. Eventually, it reached the size it is today (though it is still expanding).

There was no explosion involved. Even if there had been an explosion, there certainly couldn’t have been a huge explosion. The entire universe was compressed into a single, infinitely small point. It couldn’t possibly have been smaller. Even if there was an explosion, that explosion would have had to be incredibly tiny. It would have been so small the naked eye couldn’t see it (assuming there was an eye outside the universe to observe the Big Bang).

And then there’s the somewhat trivial error of saying “astronomers” all support the idea of the Big Bang. The Big Bang is only one cosmological model. Astronomers and physicists have put forth other models. Scientists don’t know which (if any) is right.

The only reason to believe the Big Bang theory involved a “huge explosion” is the name itself. “Big” translates into “huge.” “Bang” translates into “explosion.” That’s it. If you think like that, you show scientific intelligence. If you don’t, instead thinking about what the Big Bang theory actually says, you show low intelligence.

Fortunately, Kahan decided to exclude this item from his measure of scientific intelligence. According to him:

analysis shows that the Indicators’ Evolution and Bigbang items are both biased with respect to individuals who display a relatively high degree of religiosity (Figure 3, top two panels). As the form of science comprehension measured by OSI_2.0 increased, the probability of a correct response to the items increased substantially more for relatively non-religious individuals than for relatively religious ones.

You can see Figure 3 below:

8-27-Kahan-Religion

The paragraph I quoted just above refers to a variant of the question I quoted at the beginning. The difference is immaterial. As you can see, in both variants of the question, religious views make one less like to give an answer showing the respondent has no idea what the Big Bang theory states. The more educated and less religious you are, the less likely you are to understand the Big Bang theory. We could assume scientifically literate religious folk better understand the Big Bang because of how it relates to their religion.

Of course, we might be wrong. These people may give the right answer for the wrong reason – because it is contrary to their religious views. We can’t tell. All we can really tell is Kahan’s question is incredibly stupid, and his analysis of his results shows he is in a poor position to judge people’s knowledge of science.

And it’s not just that one question. His question about evolution was poorly phrased:

“According to the theory of evolution, human beings, as we know them today, developed from earlier species of animals.” (True/false)

A human being does not evolve. A species does not develop. The immediate predecessor to the human species was several species of humanoids, humanoids which were only “animals” in the sense that humans are also animals. None of this is wrong, per se, but it does make for a very awkward question.

Some of his other questions are worse than awkward. For instance, he used one question about global warming to indicate political influences on scientific views:

8-27-Kahan-Warming

But look at that question:

“[Is the earth] getting warmer (a) mostly because of human activity such as burning fossil fuelsor (b) mostly because of natural patterns in the earth’s environment?”

The primary temperature indexes used in the global warming debate indicate there hasn’t been any noticeable warming in over a deacde. Given that, how can we ask people what is causing the planet to get warmer? The question is built upon a false premise. There is no right answer.


I can forgive poorly phrased questions. I can forgive questions which are wrong as stated, but can kind of be understood given their implications.

I can’t, however, forgive questions which show a complete and total lack of knowledge of fundamental facts of subjects being discussed. There is no excuse for Kahan’s question about the Big Bang. There is no excuse for his commentary about how people responded to it. And given the fact he cited several other papers in reference to it, there’s probably no excuse for other people making the same stupid mistakes he made.

Five minute swith Google would have been enough to teach Kahan better. He could have just looked on Wikipedia. It wouldn’t have been hard. Anyone who made even the slightest effort to understand the Big Bang theory would have known better than to ask the stupid question he asked.

If he never bothered to put the slightest effort into understanding a topic he asked thousands of people about and wrote paragraphs discussing, how can he be in a position to tell us who is and is not scientifically intelligent?

*Technically, the Big Bang model does not discuss the point at which the entire universe existed in a single point of infinite density (singularity). It describes what happened immediately after that point.

Advertisements

12 comments

  1. Are you not misreading?

    The evolution question is also *not* in the scale. Indeed, the scale is used to *critique* both the Bigbang & Evolution items, neither of which was drafted by me. The critique was exactly the one you are offering: that the items (standard ‘science literacy’ ones) are *invalid* b/c they confound knowledge w/ religious identity.

    Same for the global warming question. It’s a standard item in public opinion polls. My point is that it *isn’t* valid; it clearly isn’t measuring science comprehension, at least as that disposition is measured w/ other items that don’t have this same defect of confounding political outlooks & knowledge.

    There might be a problem w/ my scale, but I actually think we are in agreement on the very points you raise.

    Interested readers can, of course, see for themselves: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2466715

  2. ==> “Kahan believes it is okay to measure people’s scientific intelligence by taking absolutely incorrect answers as “right.” .

    I think that having his mind-boggled has left Brandon a bit confused on your view of the validity of the science literacy scales.

  3. I don’t think it’s a good idea to frame science as a collection of beliefs. Teach a child to regurgitate factoids and they are considered scientifically literate?

    Just for fun, here’s Eric Lerner critiquing the big bang theory.

    Interestingly the big bang theory was first proposed by a Jesuit priest. Some have suggested he was trying to reconcile his scientific understanding with his religious beliefs. “Let there be light!” 🙂

  4. dmk38, you suggest I am misreading because:

    the scale is used to *critique* both the Bigbang & Evolution items, neither of which was drafted by me. The critique was exactly the one you are offering: that the items (standard ‘science literacy’ ones) are *invalid* b/c they confound knowledge w/ religious identity.

    But this isn’t true. I never said anything of the sort. In fact, at your site, I specifically suggested the opposite was possible. You just claimed to say “exactly” what I said even though your position is quite possibly the exact opposite of mine.

    To be clear, I’ve said you have no evidence this “bias” is due to religious identity. An alternative interpretation is religious people are more interested in the Big Bang theory. If so, we could expect them to better understand it, meaning we could expect them not to pick a stupid answer.

    My point is that it *isn’t* valid; it clearly isn’t measuring science comprehension, at least as that disposition is measured w/ other items that don’t have this same defect of confounding political outlooks & knowledge.

    And as I’ve said before, you have provided no evidence for your claim the item “isn’t measuring science comprehension.” All you’ve done is show an inconsistency and assumed that inconsistency is because of a particular effect.

    There might be a problem w/ my scale, but I actually think we are in agreement on the very points you raise.

    Uh… no. You specifically labeled agreement with that question as the “correct answer.” That’s specifically saying the answer is right, not just the one which you believe happens to measure some latent variable.

  5. Brandon, I think you are confused. All I can say is that people should read the paper if they are interested to figure out what sort of analysis was being done in connection with the items in question. (again: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2466715 )

    I agree entirely with JanesNV’s point, too — a valuable test measures the capacity to acquire knowledge & use it, not the assimilation of some canonical list of “facts”

  6. dmk38, given you directly misrepresented my position, falsely claiming that I am arguing the Big Bang and Evolution items measure religious identity, I find it strange you simply say you think I am confused. You read into my post something completely absent from it. If anyone is confused, I’d suggest it is you.

    I encourage people to read the material in question. And if you want to leave the discussion with an empty response like saying I am just confused, you can. However, I think everyone can agree you were wrong to say your:

    critique was exactly the one you are offering: that the items (standard ‘science literacy’ ones) are *invalid* b/c they confound knowledge w/ religious identity.

    You can participate as much or as little as you’d like. However, I cannot understand why you’d read into my writings things unlike anything I’ve ever said then refuse to acknowledge your mistake. That only inhibits communication and understanding.

    I suppose you could say I’m confused about that.

  7. JamesNV, I agree. That’s one of the many concerns I have with scales like this. Suppose, for instance, you find one group is less “scientifically literate” than another according to this scale. Does that actually mean what it seems to mean? Maybe not. Maybe the less “scientifically literate” group actually understands the science better, thus it gives more accurate answers. In that case, they’re being ranked lower simply because they are able to recognize poorly crafted questions as being poorly crafted.

    As for the Big Bang, I’ve actually been involved in many discussions of whether we should accept it or some other cosmological model (or both) because I used to talk a fair bit with people studying to be high energy physicists and astronomers. Most said the Big Bang model fits the data we have, but we have little evidence directly supporting any single model so new evidence may direct us to different models. One of the biggest questions is just whether or not there actually was a singularity at any point.

    But according to Dan M. Kahan’s document, that’s not the “correct answer.” If you say that, you’re not giving “a correct response.”

  8. Dan –

    I suggest that you look at the first sentence of Brandon’s post:

    Did you know the universe began with a huge explosion? If not, you’re an idiot. So says Dan M. Kahan.

    Even though he could never show that you said what he said you said, he will insist that you said it nonetheless. It doesn’t matter how you clarify what you actually said, or god forbid what you meant or what you think or what you believe, Brandon is of the opinion that if he thinks that you said something, even if you didn’t say it, you did say it.

    It goes along with a basic confusion on his part that his opinions (such as his mischaracterizations of what someone said) = fact.

  9. False dichotomy.

    JamesNV writes (and DK endorses) “I don’t think it’s a good idea to frame science as a collection of beliefs. Teach a child to regurgitate factoids and they are considered scientifically literate?”

    Yes, exactly. Certain things must be accepted, such as 2+2=4. WHY it is this value is explored in college and is mostly a philosophical question of naming convention.

    In the case of science, basic scientific literacy IS rote memorization of specific facts pertaining to light, gravity, inertia, energy. It doesn’t matter WHY Earth’s gravity is to be memorized as providing an accelleration of 9.81 meters per second per second and even my description is more pedantic than I would expect of a high school student.

    Having a brilliant mind (which I do) is by itself not sufficient to describe any kind of “literacy”. Literacy is literature, literal, knowledge, factoids.

    How you *use* all that is quite another matter. I believe people are better served having a *wide* base of knowledge such that you get some synergy from all these factoids. Occasionally society does need someone brilliant in one specialty (and necessarily ignorant everywhere else) to define these factoids in the first place.

    My own experience in this came when I decided I could write science fiction. But I’m pedantic and didn’t want to be an idiot not having orbital mechanics correct. So I studied the physics of gravity, orbits and things like that. For each “factoid” quite a lot of research, study, theory and testing exists — each of which has its own foundation of factoids.

    So, yeah, let’s just ask some scientific questions and see who gets the “consensus” answers and that’s as good as it is going to get.

  10. In sympathy to DK I would suggest the following story where a test taker (me) is smarter than the test giver (someone in the Navy).

    The test question was “how many characters are on a print drum?” (We’re talking mainframe line printer that used a print drum).

    I raised my hand and asked, “The question is ambiguous. Do you want the number of characters across the page, or around the circumference of the drum, or the product of the two? Shall I consider the space (there is actually a space on this particular drum, not just the computer’s decision to not print anything there, I think the idea is to stabilize the power supply by having every hammer actuate on every rotation of the drum even if there’s nothing there).

    The instructor, exasperated by the question (and clearly not knowing), asked the class, “Is there anyone here that is confused by the question?” and the class all assured him they were sure what it meant.

    But I persisted: “But she is quite certain you mean the 63 printable characters around the circumference while he believes you mean the 120 characters along the axis of the drum. One of them is going to get it wrong even though he or she is actually correct.”

    In the end they threw out the question because nobody knew the “correct” answer! Clearly I knew all possible answers, or at least three of them plus maybe some I also didn’t think of (does the serial number stamped on the end of the drum count as “characters on the drum”?)

    P.S. I have one of those drums salvaged during replacement. I’m intending some day to roll sugar cookies with it or maybe use it as a nerdy flower vase (there’s a shaft hole all the way through it longwise with a slot for a Woodruff key).

  11. Anyway, the short version is that “literacy” simply means what you know and says nothing about your thinking skills.

    Therefore a science literacy test *is* simply an inventory of what you know and it will be simply a regurgitation of what you’ve been told, or read.

    The divergence of result clearly shows that many people understand how to take the test but do not personally subscribe to some of the theories, even well established theories.

    For instance, “evolution” is a word that describes several theories that all agree more or less as to result but disagree as to particulars. So there is no single “theory of evolution” and a pedantic person (me, for instance) might wonder which one you mean.

  12. Pedants unite!

    The universe exploded.

    Well, expanded really fast, probably at the speed of light since of course it was in the act of defining exactly what that was.

    It is huge now since nothing known is larger.

    So, yes, I would have accepted “huge explosion” but it sounds like something an elementary school child would say.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s