Revisiting the Cosmic Log Poll One More Time

imageA reader writes:

I want to revisit the online poll from Cosmic log. I find it very interesting.

Back in early 2000, according to Slate, a “weekly poll on the Web site of the Democratic National Committee asked visitors: ‘As the nation approaches a new millennium, what are the most important priorities facing our next president? Saving Social Security, strengthening Medicare and paying down the debt or implementing George W. Bush’s $1.7 trillion risky tax scheme that overwhelmingly benefits the wealthy?’ Thanks to an organized Republican effort, more than two-thirds of the respondents favored Bush’s tax cuts, prompting an embarrassed DNC to remove the poll from its site.”

That is one of the reason why online polls are inaccurate. They are easy to tamper with. It happens all the time. Fortunately, this can be usually be detected because it invariably makes it into blogs, Facebook and Twitter. Google and other search engines find messages to this effect when it happens. There is no evidence that this happened with the Shroud of Turin poll at Cosmic Log. There were over 20,000 respondents and there is no way that any attempt to stuff the ballot box would have gone unnoticed.

But there are other problems. As Slate makes clear, these polls only reflect the audience of a particular website on which the poll is conducted. The DNC poll, had it not been tampered with, primarily reflected the views of Democrats who took their politics seriously enough to visit the website.

And Slate discussed technical problems, as well. For instance, they mention “the ‘primacy effect’ and the ‘recency effect,’ meaning that the first and last choices are more likely to be chosen.”

And then there is that fact that an accompanying story may influence the poll results.

So what should we expect for a poll within a popular science blog operated by news site like MSNBC given the order of the questions and the skeptical tone of the story. Not what we see. Only 15% think it is fake while 38% think it is real.

But what about the 43%. Well, it is interesting. And I think the story in Cosmic Log was not so much skeptical as balanced. Nonetheless, yes, I do find it surprising.

2 thoughts on “Revisiting the Cosmic Log Poll One More Time”

  1. My own personal view is that online polls are completely invalid and any attempt to extract meaning from them is an exercise in frustration. Polls done by reputable firms have a hard time getting it right and they take great pains to do that over and above what online polls do. At best online polls reflect only who decided to respond to them out of the particular audience that visited at the time they were posted. Blogs etc draw types of people and that alone can skew the results because certain types may be far from representative of the target audience.

    If the author wants an opinion poll from a certain audience he would do best to commission such a poll from a reputable firm.

Comments are closed.