[EL] Uh oh, Rick...

[Michael McDonald]

Some more thoughts:
 
The authors really want to have it both ways in expanding the number of noncitizen voters/registrants. There are three types of matches they use in their definition:
 
1. noncitizen respondents who say they are registered and have a voter file match
2. noncitizen respondents who say they are not registered and have a voter file match
3. noncitizen respondents who say they are registered but do not have a voter file match
 
#2 and #3 are at odds with one another. In #2, the authors assume the respondents are not truthful (for whatever reason) and the voter file is accurate. In #3, the authors assume the respondents are truthful but the matching procedure is flawed. You really can’t have it both ways. Either you trust the respondents or the matching procedure. If you think there are errors in self-reports and/or the matching procedure, then it is important to quantify the bias and magnitude of the errors.
 
It also strikes me that both the 2008 and 2010 surveys have registered voters as the sample frame. In 2008, 11 noncitizens who said they were registered were matched to the voter file (category #1). In 2010, even though the sample size increased by 22,800 respondents, there were zero respondents in category #1. Why did the number do down to zero? Did the matching algorithm improve so that there were fewer false positives of noncitizens matched to the voter file? Also note the combined #2 and #3 declined from 67 (19.8% of self-reported noncitizens) in 2008 to 76 (15.6%) in 2010. To the first, we might add the 11 in category #1 since they are in the authors’ universe. It seems like the change would be outside the sampling error, and might be due to improved matching algorithms finding fewer false positives.
 
One might say that election administration improved between 2008 and 2010 to knock more noncitizens off the voter rolls, which explains why there were fewer noncitizen registrants in 2010 compared to 2008. There is not a lot of evidence of that. Story after story of allegations of massive noncitizen voting end in a whimper when errors are found in the matching algorithms that uncovered the alleged noncitizen voting. It stands to reason if election officials have trouble with matching their voter files to identify noncitizens, Catalist might have similar problems when matching people who fit the profile of noncitizens (e.g., Latino names that are matched between the CCES and the voter file using fuzzy logic). 
 
I want to correct something I wrote previously, there are 6 issue areas, not 8, analyzed in Appendix Table a3 (actually five since the authors drop one of the areas without comment). Since Table 3 reports responses for all noncitizens, and no noncitizen voters were in three of the issue areas, I can fill in the missing issue areas in Table a4
 
Fine Businesses
Noncitizen non-voters 35.3%
Noncitizen voters 0%
Increase guest workers
Noncitizen non-voters 47.1%
Noncitizen voters 0%
 
If one were to run a Chi-squared test to determine if noncitizen voters and nonvoters are different on the issues, then these areas should be included in the test. They are not.
 
============
Dr. Michael P. McDonald
Associate Professor
University of Florida
Department of Political Science
234 Anderson Hall
P.O. Box 117325
Gainesville, FL 32611
 
phone:   352-273-2371 (office)
e-mail:  dr.michael.p.mcdonald at gmail.com                
web:      <http://www.electproject.org/> www.ElectProject.org 
twitter: @ElectProject
 
From: law-election-bounces at department-lists.uci.edu [mailto:law-election-bounces at department-lists.uci.edu] On Behalf Of Michael McDonald
Sent: Friday, October 24, 2014 8:05 PM
To: 'law-election at UCI.edu'
Subject: Re: [EL] Uh oh, Rick...
 
I’d like to replicate the analyses to have a better criticism, but at first blush I am not as confident that the authors do a convincing job of showing that the people identifying in the survey as non-citizens are actually non-citizens who voted.
 
Those who favor voter id should welcome my critique, because on p.152 the authors claim that photo id requirements are ineffective to stop non-citizen voting.
 
The authors’ evidence rests on two surveys, the 2008 and 2010 CCES surveys. The sample sizes of these surveys are 32,800 and 55,400. In 2008, 11 respondents identified as non-citizens who said they registered to vote and were matched to a voter list as being registered. In 2010 there were zero respondents in this category (Table 1, p.152).  There is no match to verify if these individuals are really non-citizens or had fat thumbs when they pressed buttons on their computer, or even understood the questions (the CCES is an internet survey).  To get higher numbers, the authors either use self-reports of non-citizens who self-reported being registered and were not matched to the voter files or reported not being registered and were matched with the voter files. I have many reservations about matching procedures and (from a legal perspective) would want independent confirmation from any of these three matching types that these individuals were in fact non-citizens who were registered to vote, especially since we are talking about small numbers of matches that could be a consequence of statistical flukes or other problems with the matching process.
 
The authors do not report if any of the 11 noncitizens who were validated as registered in fact voted (again granting that the matching algorithm didn’t produce a false positive and that the respondents didn’t misreport their citizenship status). They instead use their larger definition to find 48 non-citizens voted and 291 did not. As a percentage of the sample, even these numbers are exceedingly small and survey researchers generally would have less confidence in such small numbers. This is where next the heavy use of weighting that Vladimir references comes in.
 
Why I want to replicate their findings comes from their Appendix analyses, where the authors attempt to convince the reader that the non-citizen voters are really non-citizen voters. There’s a sleight of hand at work here that strikes me as cherry picking of data. In Table A3 (p.156), the authors examine 8 issue areas to convince us that the non-citizens are different than citizens. In Table A4 (p.156) the authors attempt to show through a similar analysis that noncitizen voters are the same as noncitizen non-voters. Here, they present only 3 issue areas because there were zero non-citizen voters who answered affirmatively to 5 issues they choose not to present. They conclude on the 3 issues the non-citizen non-voters are the same as non-citizen voters, but that is obviously not true for the 5 other issue areas that they choose not to report. For this glaring cherry picking of evidence, I’m even more highly skeptical and want to do a full blown replication to see where else there may be issues with their methods.
 
============
Dr. Michael P. McDonald
Associate Professor
University of Florida
Department of Political Science
234 Anderson Hall
P.O. Box 117325
Gainesville, FL 32611
 
phone:   352-273-2371 (office)
e-mail:  dr.michael.p.mcdonald at gmail.com <mailto:dr.michael.p.mcdonald at gmail.com>                 
web:      <http://www.electproject.org/> www.ElectProject.org 
twitter: @ElectProject
 
From: law-election-bounces at department-lists.uci.edu <mailto:law-election-bounces at department-lists.uci.edu>  [mailto:law-election-bounces at department-lists.uci.edu] On Behalf Of Kogan, Vladimir
Sent: Friday, October 24, 2014 7:11 PM
To: Rick Hasen; Steve Hoersting
Cc: law-election at UCI.edu <mailto:law-election at UCI.edu> 
Subject: Re: [EL] Uh oh, Rick...
 
I would be very careful about drawing broader conclusion about the incidence of non-citizen voting based on this study. The authors do a convincing job of showing that (1) the people who identified as being noncitizens in the survey are actually noncitizens and (2) those who say they voted actually voted.
 
I’m less convinced that we can generalize from this sample. The data is from the Cooperative Congressional Elections Studies survey, which is based on a non-representative opt-in panel from YouGov/Polimetrix. YouGov has a methodology for making their samples look like a random sample, and they have an excellent track record of predicting actual election outcomes. But I would be much more cautious about drawing conclusions about the representativeness of this sub-sub-population. As the authors themselves note, the educational levels among the noncitizens in their sample are much higher than the average among all noncitizens in the U.S. They try to use survey weights to get at this, but this only works as well as the demographics you’re using to construct the weights. Unless you think the noncitizens who sign-up to be in the YouGov panel are representative of noncitizens who do not, I’m not sure how much this teaches us about the aggregate rate of non-citizen voting. 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://webshare.law.ucla.edu/Listservs/law-election/attachments/20141024/24172f7a/attachment.html>