Errors Associated with "Instant" and Overnight Polls
One issue where news values and good polling methods clash is the media's appetite for "instant" polls which provide an immediate reaction to dramatic events such as the recent announcement of vice-presidential candidates, or the impact of presidential debates.
A key question for poll watchers, and the media who report polls, should always be "How many days was the survey in the field?" In general, the quality of a sample improves the longer the survey is in the field. Surveys conducted on one evening, or even over two days, have more sampling biases - due to non-response and non-availability - than surveys which are in the field for three, four or five days.
All surveys fail to interview many people who are not available when the survey is conducted, because they are on vacation, on a business trip, visiting, shopping, eating out or just too busy to take the call. That is why the most reliable telephone surveys make three, four or more calls, on different days, to try to complete an interview. Obviously this is not possible for polls which are conducted overnight or over a few hours, and their response rates are generally much lower.
Given the very real possibility that those who are not interviewed, because they are not available, have even slightly different opinions than those who are interviewed, overnight polls, with their very low response rates, are much more likely to have substantial biases than polls with multiple call-backs over several days.
Furthermore, because more people are at home on weekends, surveys that include both weekend and weekday interviewing are better than those that are only conducted on weekdays.
All reports of instant and overnight polls should mention the likelihood of increased errors because of their much lower response rates.
Also, bear in mind that percentages from one night polls for such things as presidential preference, or approval, or other characteristics cannot be compared to percentages from longer polls. The one night polls only represent the people at home that night. Furthermore, the reaction of those at home to a debate or a speech or some other event may only be meaningful for those who watched the event. Longer polls more nearly represent all people age 18 and over. Therefore, measurement of change in a characteristic or preference should be avoided.
For more information about this and other polling issues, contact the NCPP Polling Review Board Members.