Saturday, December 8, 2007

Poll Interpretation Hall of Infamy - Margin of Error: today's winner is CNN

We've been using political polls for what, 70 years? Barely a day goes by during the now almost endless political season without one poll or another being announced. But journalists still do not understand what the margin of error means.

For ex., this morning my local news program introduced a CNN report with the statement that the economy was now more important to voters than Iraq. The CNN reporter repeated that, rather breathlessly, as though a major shift, Richter Scale 7 at least, had occurred. She interviewed several "ordinary" people, all of whom explained their fears about the economy.

Then I saw the poll results (for which we should indeed be grateful): 29% ranked the economy first; 28% ranked Iraq second. Not exactly a major difference. And, luckily for me, the chart also indicated the margin of error: 4.5%.

Now, this is pretty basic statistics, sort of Stat 101, first week. The margin of error represents the sampling error, the degree to which the reported percentages may be wrong due to the size of the sample and other polling characteristics.

In short, the relative difference in importance between the economy and Iraq is impossible to determine from this particular sample.

It wouldn't be so bad if this were a one-off reporting error. But it isn't. For decades now, journalists at all levels, continue to cite major shifts of one kind or another based on polls in which the changes are all within the margin of error. What is even worse is that professional pollsters seem to have given up trying to get these thick-headed reporters to understand the MOE and blithely second all the idiotic statements about the significance of this or that "move" in the polling data.