[EL] Uh oh, Rick...

[Michael McDonald]

I’d like to replicate the analyses to have a better criticism, but at first blush I am not as confident that the authors do a convincing job of showing that the people identifying in the survey as non-citizens are actually non-citizens who voted.
 
Those who favor voter id should welcome my critique, because on p.152 the authors claim that photo id requirements are ineffective to stop non-citizen voting.
 
The authors’ evidence rests on two surveys, the 2008 and 2010 CCES surveys. The sample sizes of these surveys are 32,800 and 55,400. In 2008, 11 respondents identified as non-citizens who said they registered to vote and were matched to a voter list as being registered. In 2010 there were zero respondents in this category (Table 1, p.152).  There is no match to verify if these individuals are really non-citizens or had fat thumbs when they pressed buttons on their computer, or even understood the questions (the CCES is an internet survey).  To get higher numbers, the authors either use self-reports of non-citizens who self-reported being registered and were not matched to the voter files or reported not being registered and were matched with the voter files. I have many reservations about matching procedures and (from a legal perspective) would want independent confirmation from any of these three matching types that these individuals were in fact non-citizens who were registered to vote, especially since we are talking about small numbers of matches that could be a consequence of statistical flukes or other problems with the matching process.
 
The authors do not report if any of the 11 noncitizens who were validated as registered in fact voted (again granting that the matching algorithm didn’t produce a false positive and that the respondents didn’t misreport their citizenship status). They instead use their larger definition to find 48 non-citizens voted and 291 did not. As a percentage of the sample, even these numbers are exceedingly small and survey researchers generally would have less confidence in such small numbers. This is where next the heavy use of weighting that Vladimir references comes in.
 
Why I want to replicate their findings comes from their Appendix analyses, where the authors attempt to convince the reader that the non-citizen voters are really non-citizen voters. There’s a sleight of hand at work here that strikes me as cherry picking of data. In Table A3 (p.156), the authors examine 8 issue areas to convince us that the non-citizens are different than citizens. In Table A4 (p.156) the authors attempt to show through a similar analysis that noncitizen voters are the same as noncitizen non-voters. Here, they present only 3 issue areas because there were zero non-citizen voters who answered affirmatively to 5 issues they choose not to present. They conclude on the 3 issues the non-citizen non-voters are the same as non-citizen voters, but that is obviously not true for the 5 other issue areas that they choose not to report. For this glaring cherry picking of evidence, I’m even more highly skeptical and want to do a full blown replication to see where else there may be issues with their methods.
 
============
Dr. Michael P. McDonald
Associate Professor
University of Florida
Department of Political Science
234 Anderson Hall
P.O. Box 117325
Gainesville, FL 32611
 
phone:   352-273-2371 (office)
e-mail:  dr.michael.p.mcdonald at gmail.com                
web:      <http://www.electproject.org/> www.ElectProject.org 
twitter: @ElectProject
 
From: law-election-bounces at department-lists.uci.edu [mailto:law-election-bounces at department-lists.uci.edu] On Behalf Of Kogan, Vladimir
Sent: Friday, October 24, 2014 7:11 PM
To: Rick Hasen; Steve Hoersting
Cc: law-election at UCI.edu
Subject: Re: [EL] Uh oh, Rick...
 
I would be very careful about drawing broader conclusion about the incidence of non-citizen voting based on this study. The authors do a convincing job of showing that (1) the people who identified as being noncitizens in the survey are actually noncitizens and (2) those who say they voted actually voted.
 
I’m less convinced that we can generalize from this sample. The data is from the Cooperative Congressional Elections Studies survey, which is based on a non-representative opt-in panel from YouGov/Polimetrix. YouGov has a methodology for making their samples look like a random sample, and they have an excellent track record of predicting actual election outcomes. But I would be much more cautious about drawing conclusions about the representativeness of this sub-sub-population. As the authors themselves note, the educational levels among the noncitizens in their sample are much higher than the average among all noncitizens in the U.S. They try to use survey weights to get at this, but this only works as well as the demographics you’re using to construct the weights. Unless you think the noncitizens who sign-up to be in the YouGov panel are representative of noncitizens who do not, I’m not sure how much this teaches us about the aggregate rate of non-citizen voting. 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://webshare.law.ucla.edu/Listservs/law-election/attachments/20141024/8ce8a8e4/attachment.html>