Skip navigation.
 
 SEARCH
NCPP.ORG
THE WEB
Welcome 

Adventure In Baku: Exit-Polling Azerbaijan

  • : Function ereg() is deprecated in /home/ncpporg/public_html/includes/file.inc on line 649.
  • : Function ereg() is deprecated in /home/ncpporg/public_html/includes/file.inc on line 649.
  • : Function ereg() is deprecated in /home/ncpporg/public_html/includes/file.inc on line 649.

From 1967 to 1990, Warren Mitofsky was executive director of the CBS News election and survey unit, and was an executive producer of its election night broadcasts. He conducted the first exit polls for CBS in 1967, and developed the projection and analysis system used successfully by CBS and Voter News Service. He started the CBS News/New York Times Poll in 1975 and directed it for CBS for its first 15 years.

He is the founder and president of Mitofsky International (MI).  MI is a survey research company whose primary business is conducting exit polls for major elections around the world. It does this work exclusively for news organizations. Mitofsky has directed exit polls and quick counts since 1967 for almost 3,000 electoral contests in United States, Mexico, Russia and the Philippines. Its election research clients in the United States have included ABC, CBS, CNN, NBC, Wall Street Journal, New York Times, Washington Post, Los Angeles Times and Time; international clients include Televisa and the National Chamber for Radio and Television Broadcasting (Mexico), RAI (Italy), ZDF (Germany), Fuji (Japan), NTV and RTR (Russia) and Austrian and Finnish television.

Mitofsky was president of both the American Association for Public Opinion Research (AAPOR) and the National Council on Public Polls (NCPP).  He received AAPOR's Lifetime Achievement Award in 1999. He is a Fellow of the American Statistical Association and serves on the boards of the Roper Center and the NY State Committee on Open Government. In 1995 he was a fellow at Harvard's Kennedy School of Government. Later that year he was the Howard R. Marsh Visiting Professor at the University of Michigan.

Adventure In Baku: Exit-Polling Azerbaijan
By Warren Mitofsky and Joe Lenski

A Public Opinion Pros Special Report

In June of 2005, the two of us were approached by a representative of Renaissance Associates to do an exit poll for the Azerbaijan parliamentary elections in November. We soon learned that the two previous elections in Azerbaijan since the breakup of the Soviet Union were widely believed to have been fraudulent. For this election, transparency and international acceptance of the results were the announced goal of the Azeri government. From the highest levels, it was made clear that exit polls would be part of the process to ensure the appearance of free and fair elections. In a decree issued May 11, 2005, President Ilham Aliyev ordered election officials to facilitate the “process of exit poll[ing] that will be carried out by public experts, [and] not to interfere in an illegal manner in course of that process.”

During our initial visit to Azerbaijan in July we were assured that our reputations for honest results were what our sponsors wanted. Well, that’s what they got. We are just not sure anymore that they really meant what they said.

Recent elections in Ukraine, Georgia, and Kyrgyzstan resulted in regime change as a consequence of corrupt vote counts that exit polling helped to challenge.

In last year’s Ukrainian election for president, neither the ruling party’s candidate, Prime Minister Viktor Yanukovych, nor his chief opponent, Viktor Yushchenko, achieved the required majority in the first round of voting. Yanukovych first claimed he had won, but then backed off after a protest.

In the run-off, Yanukovych, who had been publicly endorsed by Russian President Vladimir Putin, again claimed victory in the official vote count, but U.S. and E.U. observers reported many signs of fraud. Exit polls sponsored by western groups pointed to Yushchenko’s having won. Our own analysis of those exit polls shows their results were not quite conclusive, but that is another story. In any event, that election was set aside by the country’s highest court after a massive and prolonged uprising in the streets of the capital, Kiev. Ultimately, the run-off election was re-run, and Yushchenko won the presidency.

Similarly, in 2003, President Eduard Shevardnadze’s party in Georgia, after an official count of the parliamentary election named it the winner, was ousted by a rebellion in the streets of Tiblisi after two out of three exit poll showed the opposition on top. For good measure, angry mobs stormed the presidential palace and drove Shevardnadze himself out of office.  

The events in Ukraine and Georgia definitely had an influence on the sponsorship of exit polls in Azerbaijan for its Milli Mejlis, or parliamentary, elections on November 6, 2005. In the end, there were three exit polls for this small country. The Agency for International Development (USAID), which is an arm of the U.S. State Department, sponsored an exit poll in 65 of the 125 districts. PA Consulting Group of Madison, Wisconsin, was the lead organization. There was another exit poll by an Estonia firm, Saar Poll, sponsored ostensibly by the Center for Regional Development in Azerbaijan. We believe the same people who sponsored our exit poll also were involved with the Saar Poll. At the time, it made no sense to us why our clients would want two exit polls, but the reason became clear following the election.

And then there was our exit poll, conducted by Mitofsky International, Edison Media Research, and CESSI, Ltd. Edison/Mitofsky did the 2004 exit polls for U.S. news media, and CESSI, a Moscow firm, has worked with Mitofsky on all the Russian exit polls since 1993. We thought we were the only exit poll covering all 125 districts, but the Saar Poll claims it did as well.

One may wonder, why all the exit polls? In a word, OIL. Azerbaijan is an oil-rich country by the Caspian Sea, with a brand new pipeline to Turkey and the Mediterranean Sea. There is lots of interest in that oil, and in having Azerbaijan’s government stable and friendly to the Western countries. Azerbaijan, which is strategically located near both Afghanistan and Iraq and borders Iran, also is friendly to U.S. military bases.

Exit polls, as we have seen, have been used more than once to discredit elections in emerging democracies where ruling parties were expected to falsify the vote. They can also be used to validate outcomes where there has been an honest count. An exit poll in Afghanistan, for example, that was funded by Washington lent credibility to Ahmed Karzai’s claim to a majority in the country’s recent presidential election. Given the announced intent of the Azeri government to conduct an honest election this time, the opportunity to give it credence was presumably a factor in the funding of the AID poll. It was also the avowed purpose of our own clients.

We met the sponsors of our exit poll for the first time in Baku, the capital city of Azerbaijan, in July to finalize arrangements. They seemed genuinely interested in democracy and had worked with other nongovernmental organizations (NGOs) to further its aim in Azerbaijan. We agreed to do the exit polls once our clients agreed that we, not they, could release the results of our exit poll to the media shortly after the voting was over. They made this pledge, but in the end they did not live up to it. Renaissance Associates, the company that hired us, was a Swiss company, run by a Bulgarian.

Yes, we had doubts about the source of the money. We had some thoughts that the money was coming from a government. Whose government it was coming from was uncertain, though the one in Baku was an obvious possibility. In our naiveté we thought it would make no difference, as long as we could do our work unimpeded, and until election day there was no interference. Once the polls had closed and it was time to release the results, the story would be different.

On that July trip, we hired three local organizations to work with us on election day. We divided the interviewing between the Association for Civil Society Development in Azerbaijan (ACSDA) and Sociological Research and Socio-Economic Forecasting Center (QAFQAZ). ACSDA did the interviewing in ninety election okrugs, or parliamentary districts, and QAFQAZ did the work in the other thirty-five. ACSDA had done polling for our client and QAFQAZ worked for years with Vladimir Andreenkov, our Russian partner at CESSI.

The country has more than five thousand polling stations, or about forty per okrug. We selected probability samples of either ten or twenty polling locations per district, depending on how close the vote in that district was expected to be. In the end, we had interviewers stationed at over fourteen hundred polling locations throughout Azerbaijan­as many locations as we had covered in the U.S. presidential election exit poll in 2004. Most districts were not expected to be close. For the last highly suspect parliamentary election, only two districts had even a moderately close election. The rest had been landslides, according to the official vote returns.

We used current registration for the sample selection and stratified each district on the size of the polling locations. The past election results were not much help for either the stratification or for making ratio estimates. We did not have much confidence in the past results where they were obtainable. In a number of polling stations they were not available, having been voided by the election commission due to irregularities.

We were concerned about the accuracy of the exit poll votes that we would receive. An exit poll done by Penn, Schoen, and Berland for the Venezuela recall vote on President Hugo Chavez in 2004, which showed a landslide for dumping Chavez when a landslide backed him in the official count, was called into question, among other reasons, when it turned out that the interviewers were tied to anti-Chavez activists.

To protect ourselves against that contingency, we hired another Azeri survey company, SIAR Social and Marketing Research Center, to monitor the interviewing. SIAR had no responsibility for the conduct of the interviews. Its only role was to send monitors to a subsample of our precincts to assure us that the interviewing was being done according to our procedures. There were 105 monitors covering more than one-third of all of our sample locations. Each was assigned a subsample of the polling locations in the exit poll. All the districts where the race was expected to be close were included in the monitoring. We received reports from the SIAR monitors throughout election day so we could catch problems with interviewers at the polling places. We also used the reports from the monitors to help us judge the reliability of the data we were receiving.

The story SIAR told us on election day about our interviewers was the start of our uncertainty about some of the work being done for us by ACSDA. In a key district, interviewers at thirteen of the twenty precincts could not be located at midday at their sample polling places. There and elsewhere in Baku we received different vote reports from both the interviewers and their supervisors. The supervisors’ reports in many districts were questionable. We learned of the problem almost immediately, as the computer system would not accept a second report from a precinct without a telephone supervisor interceding and canceling the first. Apparently, the interviewers had not been let in on the attempted deception. Once the problem was discovered, the interviewers were re-contacted to verify their vote reports. We did not project winners for the worst of these districts.

We had other problems with ACSDA. Questionnaires were to be delivered to our headquarters on election night from some of the districts in and around Baku, Azerbaijan’s capital. We intended to verify the hand-counted vote reports that had been phoned in as an additional check on the veracity of the work done by the interviewers. We also planned to key the answers to the seven questions on the exit poll questionnaire. Mysteriously, many questionnaires never showed up. When we inquired about their whereabouts, we were told they were delivered to some other site and could not be retrieved. Again, this happened in a number of the Baku districts, all organized by ACSDA.

Our condition for conducting the exit polls was that we would release our results to the media soon after poll closing. We said the sponsor could see our results before we posted them, but we would choose what, where, and when to report. So much for good intentions! We were prepared to report the results in two tables on our website at poll closing time. There were links to this site on the Mitofsky International and Edison Media Research websites. Our partner, Vladimir Andreenkov, had a link on the Russian version of his website, and ACSDA had one its website.

The first table showed the name and party of the winners. Projections of winners were made only when the margin between the top candidates was greater than the sampling error by a factor large enough to provide at most a 1 percent chance of an error. All districts without winners were listed as “too close to call,” and the names of the top two candidates were given. The second table was a summary of seats won by each party.

At poll closing time, our sponsor asked to see the tables before we posted them. They agreed to the party summary table being posted, but not the district-by-district winners. They wanted to have a “private” conversation, we were told, without us present. They did not say with whom they would have this conversation. They would get back to us. We were told that posting the results by district that night could result in civil unrest.

They finally got back to us Monday afternoon­twenty hours later. At that time, they pointed out conflicts between our results, the Saar exit poll and the results from the election commission. As we were not sure what to make of the conflicts, we initially published our district results without naming winners in those uncertain districts. We have since published the estimated percentages for the top three candidates for all but two districts. Winners have been called, where the results met our criteria. We omitted the percentages from districts 19 and 36. The dual reporting by interviewers and supervisors makes any result in those two districts too uncertain.

We also published an analysis of the vote for the country and for various regions. It shows only minor differences in the voting patterns by age, gender, education or occupation. Overall, independent candidates got 39 percent of the vote. The government-party, or YAP, candidates had 32 percent, and the Freedom Block, a coalition of three opposition parties, received 14 percent. The rest was scattered among a number of parties. Three-quarters of the voters told us they expected their votes to be counted fairly. The only real differences are in the regional support for the YAP. It is weakest of all in the capital, Baku, and neighboring regions to the center and northeast of the country.

How did the results from the three exit polls and those of the Central Election Commission of Azerbaijan compare? The media generally gave credibility to the USAID exit poll even though it was funded by the U.S. government, an open supporter of stability and the current Azerbaijan government. The United States also urged fair elections on President Alyiev. No one questioned the integrity of PA Consulting, the lead organization conducting the USAID poll.

A comparison of the results from all three exit polls and the election commission shows relative agreement about winners in 112 districts, and disparities in 10 of the other 13. The USAID and Mitofsky/Edison/CESSI exit polls show many similar results. The only difficulty with this comparison is that USAID only polled in 65 of the 125 districts. Their random selection of districts missed five key districts (9, 20, 21, 31, and 36) where opposition leaders were expected to offer strong challenges. Our exit poll had opposition candidates winning two of these districts­9 and 31­which differed with the original election commission count. In District 20 we agreed with the official vote count showing that the opposition candidate came in a strong second. District 21 was a close three-way race in our exit poll, while the official count showed a clear winner. In District 36 our results were based upon contradictory data, making it hard to reach a clear conclusion.  

The Saar Poll seems to have sampled a different universe of voters. In ten districts they reported votes for candidates who were not on the ballot. They also had an exit poll for district 122, which, we were told by the election commission, would not have any voting at the polling places. Saar’s most disturbing reporting was their winner projections in all but one of the districts where we differed with the election commission. That included District 9, where the election commission overturned its initial declaration of a winner. Somehow, Saar got the same winner as the election commission even when our exit poll, and sometimes the USAID exit poll, showed a different candidate.

One should never go through an experience like this without taking away something for the future. The number one lesson here is that public polling is difficult to do for organizations other than the media and for organizations that have a long history of publication of survey results, regardless of the direction of the findings. This criterion is met in the United States by foundations that sponsor polls, many government agencies, and private companies. However, if one chooses to work, as we did, for organizations with no known record for open availability of the survey findings, caveat emptor. What we had thought was a clear understanding of our plan to publish at poll closing was the start of a negotiating position for our sponsors. We had to struggle to make our report public. In the end we succeeded, but it would have been much timelier had it been released when the polls closed.

The monitors we hired to check the work of the interviewers were an invaluable addition to our peace of mind about what we were reporting. Without them we might have been misled by the bogus reporting from ACSDA supervisors in key districts.

What is unclear to us is why we were hired to do exit polls. We thought the Azerbaijan elections would be fair and transparent. Otherwise, no one would hire us, knowing we would identify questionable vote-counting. The people who hired us must have thought the election would be fair. At least, that is what we would like to believe.

Warren Mitofsky is president of Mitofsky International. Joe Lenski is executive vice president and cofounder of Edison Media Research.