Reporting International Surveys
March 25, 2003
These days there are many more opinion surveys of other countries, particularly countries in the middle-east. It is important to understand the opinions of our global neighbors and to compare and contrast their opinions, when appropriate, with those of the American public. Reporting these international studies often makes for interesting journalism.
However, we want to offer a caution. The reporting of these studies requires special attention to how they were conducted. We have noticed a number of recent studies where the sampling is not representative of the country. Several studies did their sampling in only a single city, sometimes two cities. Some samples are further limited to select groups of residents, typically middle class people. Public opinion in these studies, when measured this way, may not be a fair representation of public opinion in these countries.
The press releases from these surveys should point out the limits of the sampling. What they should say is something like, "this survey only represents the [MIDDLE CLASS] people living in [CITY]," or something similar. If you do not see a methods statement ask for it!
Surveys are usually done this way as a matter of pragmatism and because of cost considerations. They may be useful even with their limitations. If they are reported, we caution not to generalize beyond that part of the country represented by the survey. Both the organizations releasing survey results and those reporting on them should make note of the limitations on the interpretation. There should be no public reporting of surveys that suggest greater reliability than the methodology permits nor that suggests that the results can be generalized beyond the specific sample interviewed.