correction to my earlier post:
2100+ sample size should yield an error rate of about 2%, 1200-1500
around 3%, smaller, 4% or more. Error rate also depends on other
factors, such as how many choices are given the voter--therefore it is
possible for the presence of third or fourth party candidates, etc.
could make the error rate larger if they attract enough support.
Cliff
Clifford A. Jones, J.D., M.Phil., Ph.D.
University of Florida Levin College of Law
Center for Governmental Responsibility
Spessard L. Holland Law Center
230 Bruton-Geer Hall
P.O. Box 117629
Gainesville, FL 32611-7629
Tel. (352) 392-2237
Fax (352) 392-1457
Email: jonesca@law.ufl.edu
Home:
2230 N.W. 24th Avenue
Gainesville, FL 32605
Tel. (352) 367-9992
Fax (352) 367-9456
Cell (352) 870-9992
German (Europe) cell phone: +49 [0]173 3708512
"Michael McDonald" <mmcdon@gmu.edu> 11/3/2004 4:00:13 PM >>>
Having just arrived home from working the night at the exit poll
service
(Edison-Mitofsky), perhaps I can shed some light on this issue. First,
it
was clear to us in the mid-afternoon that the exit polls were
over-sampling
women, which was producing a pro-Democratic bias in the sample.
Corrective
measures were taken, and we alerted our subscribers well before the
polls
closed that the exit polls were showing the bias, which delayed clear
Kerry
calls in states like PA and NH. Perhaps you heard the phrase "not
enough
information" throughout the night, as we were waiting for actual vote
returns to confirm the exit polls.
How the exit polls are conducted, how they are weighted, and how they
are
used by the networks on election night go well beyond simple sampling
methodology. If a race is outside the margin of error AND it comports
with
the best previous guess of the election outcome AND it comports with
previous vote shares for various offices in the sample precincts, then
we
are able to call a race immediately when the polls close. If not, a
combination of exit poll results, past election results, and actual
election
returns are used to make calls as data come in from the Associated
Press.
The system worked as it should, and in no case did we make a call this
election that later had to be reversed.
The only issue was a delay in getting the calls out for a few states
that
might have been called sooner if we had more confidence in the exit
polls.
I think we can all agree it is more important to get it right than get
it
quickly, and in some cases we held back calls even longer because we
weren't
certain that something funny was going on with the data, like it did in
FL
in 2000. In PA, for example, the Philly suburbs were late in reporting
so
we didn't want to make a call without some hard numbers from those
areas to
confirm our exit polling. For states like FL and OH this time, that
were
close, there was not enough information in the exit polls to make a
call in
those states when the polls closed, so we were going to have to wait
for the
raw numbers came in anyway.
The real problem with the exit polls is that everybody wants to know
the
results before the polls close. Once the first wave of polls went out
to
the subscribers, it took about a half hour for our polls (and some
fake
polls) to make it onto the internet. Warren Mitofsky continually
warns
everyone not to trust the early exit polls because they are what they
are...polls, and they are subject to error (not just sampling error).
Too
bad more people don't appreciate this point.
=======================
Dr. Michael P. McDonald
Visiting Fellow, Brookings Institution
Assistant Professor, George Mason University
elections.gmu.edu
mmcdon@gmu.edu
703-993-4191
-----Original Message-----
From: owner-election-law_gl@majordomo.lls.edu
[mailto:owner-election-law_gl@majordomo.lls.edu]On Behalf Of Lash
LaRue
Sent: Wednesday, November 03, 2004 1:47 PM
To: VOLOKH@law.ucla.edu
Cc: election-law@majordomo.lls.edu
Subject: Re: Margins of error and sample sizes in exit polls
Eugene, ....... Exit polls are a problem in two ways, only one of
which is "purely mathematical."
The math problem is the random variation that one gets in taking
samples. If one samples any population repeatedly, the several samples
will differ among themselves in ways that are rather predictable. The
standard Gaussian bell curve generates the theory of how these
variations are patterned. Of course, one must make some assumptions
about the nature of the population that is being sampled before any of
this logic can get going, and a second "of course" is that the
population of voters does not fit these assumptions perfectly.
However,
the fit is close enough and this part of the problem is mathematically
well behaved. (Perhaps I should say that "it is close enough for
government work.")
The problem that is not purely mathematical (and this problem is the
serious problem) is that exits polls do not use random samples.
Depending on where one is in the country, people vary in their
willingness to respond to the pollsters. So those who do respond are
"self-selected" and thus a non-random sample; they are not typical of
the larger population of those who voted. At the worst, which does not
always happen, one gets the equivalent of a "Literary Digest" poll.
Those who are in the business try to compensate by comparing the
results of the exit polls with the actual numbers reported from early
precincts. If the margins in the exit polls are large enough (see Sen.
Clinton's run in NY), this caution is not needed. But if things are
close, this caution is absolutely necessary. At any rate, one
discovers
by this empirical comparison how to "discount" the polls. The debacle
of
the Florida exit polls in 2000 was caused by a simple error in
calculation. The precinct data that was used as a benchmark to judge
the
accuracy of the polls was erroneous, since addition errors were made
in
reporting these early totals.
By the way, the "tilt" in exit polls differs from election to
election,
region to region, in ways that are not well understood.
I went to a presentation at the American Statistical Association on
this matter after the 2000 election. One of the speakers, an employee
of
Voter News Service (the group that had egg on their collective faces),
began with two power point slides that read: first slide, "If you make
your living using probability theory"; second slide, "you are taking a
chance."
I hope this answers the question, ............ Lash /
Lewis Henry LaRue
Washington and Lee University
School of Law
Lexington, VA 24450-0303
email: laruel@wlu.edu
phone: 540-458-8513
fax: 540-458-8488
"Volokh, Eugene" <VOLOKH@law.ucla.edu> 11/3/2004 12:19:42 PM >>>
What are the typical margins of error (that is to say, the
purely mathematical ones) and sample sizes in exit polls? Thanks,
Eugene