[EL] Uh oh, Rick...
Richman, Jesse T.
JRichman at odu.edu
Tue Oct 28 04:50:38 PDT 2014
Dear Law-Election List,
Michael also raised a question about our decision to omit the crosstab of citizen status and race for 2010 from the paper. This crosstab appears below. As you can see (if the formatting comes through ok anyhow), the same basic pattern appears. Immigrant non-citizens are the least white of any group, and the most Hispanic.
There are some arguable oddities though. For instance, there are two non-citizens who indicated they are Native American. One possibility is that these are immigrants from Central America or Canada who are descendants of the first peoples. For such individuals this is arguably the appropriate response. Another possibility is that these individuals are U.S. citizens who selected the wrong citizenship status. In any case, neither of the respondents who claimed to be Native American said they voted.
Race * Citizen Crosstabulation
Citizen
Total
Immigrant Citizen
Immigrant non-citizen
First generation
Second generation
Third generation
Race
White
Count
1027
170
2813
12407
26318
42735
% within Citizen
41.3%
34.8%
61.6%
88.1%
78.3%
77.4%
Black
Count
352
84
235
218
4927
5816
% within Citizen
14.1%
17.2%
5.1%
1.5%
14.7%
10.5%
Hispanic
Count
657
150
967
633
758
3165
% within Citizen
26.4%
30.7%
21.2%
4.5%
2.3%
5.7%
Asian
Count
292
65
212
77
37
683
% within Citizen
11.7%
13.3%
4.6%
0.5%
0.1%
1.2%
Native American
Count
3
2
21
76
329
431
% within Citizen
0.1%
0.4%
0.5%
0.5%
1.0%
0.8%
Mixed
Count
50
4
147
201
473
875
% within Citizen
2.0%
0.8%
3.2%
1.4%
1.4%
1.6%
Other
Count
92
12
161
460
748
1473
% within Citizen
3.7%
2.5%
3.5%
3.3%
2.2%
2.7%
Middle Eastern
Count
15
2
14
15
10
56
% within Citizen
0.6%
0.4%
0.3%
0.1%
0.0%
0.1%
Total
Count
2488
489
4570
14087
33600
55234
% within Citizen
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
I also want to let you know that I have posted some additional documentation and replication materials on my website. The zip file can be accessed at:
www.odu.edu/~jrichman/ReplicationFilesforNonCitizenVotingPaper.zip<http://www.odu.edu/~jrichman/ReplicationFilesforNonCitizenVotingPaper.zip>
Thank you to everyone on the list for your patience with my long series of posts last night and this morning in response to the questions you have raised about my research.
Best Regards,
Jesse Richman
Jesse Richman
Associate Professor of Political Science and International Studies
Director, Social Science Research Center
Old Dominion University
BAL 7028
Norfolk VA 23529
757-683-3853
www.odu.edu/~jrichman<http://www.odu.edu/~jrichman>
From: law-election-bounces at department-lists.uci.edu [mailto:law-election-bounces at department-lists.uci.edu] On Behalf Of Richman, Jesse T.
Sent: Tuesday, October 28, 2014 2:58 AM
To: law-election at department-lists.uci.edu
Subject: [EL] Uh oh, Rick...
Michael,
Thank you for your comments on my article. I'd like to clarify a couple of things now relative to some of the concerns and issues you raised. I'll try to get more responses up soon.
Michael McDonald <dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>> wrote:
>So, I tried doing the Appendix statistical >tests myself from the data presented in the
>paper last night. I needed some numbers >reported in the main body of the paper and I came
>to the realization that the numbers of >noncitizen voters are inconsistently reported in
>the paper. To be fair, this could be a >consequence of missing data for a demographic
>category or issue area. Still, we don't >usually see high rates of missing data on race
>and ethnicity, so it seems strange to me >that Table 1 would report 2008 noncitizen voters
>equal to 84 and Table A.2 would have only >48 (those numbers are not transposed by me). >Replication is needed to uncover what is >going on here.
Concerning the difference between Table 1 and the appendix. Table 1 is about estimating voter registration not voter turnout. Thus, the gap. There are more non-citizens who say they are registered (or have verified registration, or both), than there are non-citizens who vote. That's why the appendix has a smaller number of non-citizen voters.
Michael McDonald <dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>> wrote:
>One might say that election administration improved between 2008 and 2010 to knock more
>noncitizens off the voter rolls, which explains why there were fewer noncitizen
>registrants in 2010 compared to 2008. There is not a lot of evidence of that. Story after
>story of allegations of massive noncitizen voting end in a whimper when errors are found
>in the matching algorithms that uncovered the alleged noncitizen voting. It stands to
>reason if election officials have trouble with matching their voter files to identify
>noncitizens, Catalist might have similar problems when matching people who fit the profile
>of noncitizens (e.g., Latino names that are matched between the CCES and the voter file
>using fuzzy logic).
It's a good question. Actually this difference is almost entirely due to the fact that we didn't have access to a validated registration variable for 2010. The portion of non-citizens self-reporting that they were registered to vote did not decline substantially between 2008 and 2010. What's driving the higher figure for 2008 in the first row of Table 1 is that we have the addition of a number of non-citizens who said they were not registered, but had a voter registration match from Catalist.
Michael McDonald <dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>> wrote:
>Why I want to replicate their findings comes from their Appendix analyses, where the
>authors attempt to convince the reader that the non-citizen voters are really non-citizen
>voters. There's a sleight of hand at work here that strikes me as cherry picking of data.
>In Table A3 (p.156), the authors examine 8 issue areas to convince us that the non-
>citizens are different than citizens. In Table A4 (p.156) the authors attempt to show
>through a similar analysis that noncitizen voters are the same as noncitizen non-voters.
>Here, they present only 3 issue areas because there were zero non-citizen voters who
>answered affirmatively to 5 issues they choose not to present. They conclude on the 3
>issues the non-citizen non-voters are the same as non-citizen voters, but that is
>obviously not true for the 5 other issue areas that they choose not to report. For this glaring cherry picking of
>evidence, I'm even more highly skeptical and want to do a full blown replication to see where else there may be
>issues with their methods.
I fear we were not clear enough about why we didn't analyze the difference in attitudes between citizens and non-citizens for the two issue areas (thank you for correcting yourself from 8 to 5) for which we didn't look at the difference in attitudes between non-citizen voters and non-voters. The issue is that there were absolutely NO non-citizen voters who were asked those questions. The questions were ones that didn't get asked of the whole common content sample. And luck was against us because they were (unfortunately) not asked of any of the non-citizens who said they were voters. With no non citizen voters, the percentage supporting (or opposing) any immigration attitude obviously cannot be calculated. 0/0 does not equal zero, or one, it's undefined. My daughter likes this video: https://www.khanacademy.org/math/algebra2/functions_and_graphs/undefined_indeterminate/v/why-zero-divided-by-zero-is-undefined-indeterminatehttps://www.khanacademy.org/math/algebra2/functions_and_graphs/undefined_indeterminate/v/why-zero-divided-by-zero-is-undefined-indeterminate Hopefully a validated vote measure will be made available for 2010 (we didn't have it at the time the analyses were done for the article). And with such a measure, perhaps it will become possible to estimate the attitudes of non-citizen voters for this question.
It was good to see you at the conference at American this summer. And congratulations again on the move from George Mason. You left the VA higher education system just ahead of the budget cuts the rest of us are now getting to enjoy! I remain deeply impressed by the key contribution you have made to the field by revising our understanding of the appropriate denominator for the estimate of voter turnout, and use it every semester in my methods class as an example of the importance of carefully thinking through what our concepts mean as we aim to create valid measures.
Thank you once again for your comments and critiques. Although I've got a ton of different things to respond to right now, I'll do my best to come back around to some of your other thoughts about the article. And I hope you do get a chance to do a replication. The more people who work on this issue, the better our ability as a field to come up with accurate estimates.
Best Regards,
Jesse Richman
Associate Professor of Political Science
Old Dominion University
>Fair enough, except that is not how the analysis is presented. There is no discussion of >any potential biases in the alternative definitions of noncitizen voters. Everyone who is >matched or say they voted are included in the analysis, and then the authors plow ahead. >Indeed, the "upward bound" seems to be the preferred measure when assessing overall >effects, with no discussion of a lower bound, which might be from the authors' perspective >the 5 noncitizens who said the voted and were vote validated (with the caveat about >respondents' errors and matching errors). It should be obvious to even people without a >statistics background you can't reliably statistically analyze 5 of anything.
>If the errors of respondents incorrectly answering the citizenship or voting questions or >being falsely matched as voted are random, then that would also explain how noncitizen >voters and nonvoters are statistically indistinguishable (which is the authors' Appendix >argument that their scoring of noncitizen voters is correct). Still, if the tests show >noncitizen nonvoters and voters are statistically different, that would undercut the >authors' argument without their framework. So, I tried doing the Appendix statistical >tests myself from the data presented in the paper last night. I needed some numbers >reported in the main body of the paper and I came to the realization that the numbers of >noncitizen voters are inconsistently reported in the paper. To be fair, this could be a >consequence of missing data for a demographic category or issue area. Still, we don't >usually see high rates of missing data on race and ethnicity, so it seems strange to me >that Table 1 would report 2008 noncitizen voters equal to 84 and Table A.2 would have only >48 (those numbers are not transposed by me). Replication is needed to uncover what is >going on here.
A reason why the tests cannot distinguish noncitizen nonvoters and voters is that there are so few observations that the "margin of error" on some subgroups is so large that statistical tests will have difficulty distinguishing between the characteristics of noncitizen voters and nonvoters.
There is also further apparent cherry picking of data, for example, the authors analyze the demographic characteristics of noncitizen voters and nonvoters on the 2008 CCES but not the 2010 CCES. Was the 2010 analysis inconsistent with the authors' argument, and thus excluded? Again, replication will tell. It seems obvious both years should have been reported, perhaps even pooled together, especially considering the small number of observations.
============
Dr. Michael P. McDonald
Associate Professor
University of Florida
Department of Political Science
234 Anderson Hall
P.O. Box 117325
Gainesville, FL 32611
phone: 352-273-2371 (office)
e-mail: dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>
web: <http://www.electproject.org/> www.ElectProject.org<http://www.ElectProject.org>
twitter: @ElectProject
From: Jonathan Adler [mailto:jha5 at case.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>]
Sent: Saturday, October 25, 2014 10:27 AM
To: Michael McDonald
Cc: law-election at UCI.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>
Subject: Re: [EL] Uh oh, Rick...
I have not focused on this study, but wouldn't one reason to include both categories 2 and 3 be to develop an upper-bound, with the understanding that the actual number of non-citizen voters is almost certainly lower?
Just a thought.
On Fri, Oct 24, 2014 at 9:46 PM, Michael McDonald <dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>> > wrote:
Some more thoughts:
The authors really want to have it both ways in expanding the number of noncitizen voters/registrants. There are three types of matches they use in their definition:
1. noncitizen respondents who say they are registered and have a voter file match
2. noncitizen respondents who say they are not registered and have a voter file match
3. noncitizen respondents who say they are registered but do not have a voter file match
#2 and #3 are at odds with one another. In #2, the authors assume the respondents are not truthful (for whatever reason) and the voter file is accurate. In #3, the authors assume the respondents are truthful but the matching procedure is flawed. You really can't have it both ways. Either you trust the respondents or the matching procedure. If you think there are errors in self-reports and/or the matching procedure, then it is important to quantify the bias and magnitude of the errors.
It also strikes me that both the 2008 and 2010 surveys have registered voters as the sample frame. In 2008, 11 noncitizens who said they were registered were matched to the voter file (category #1). In 2010, even though the sample size increased by 22,800 respondents, there were zero respondents in category #1. Why did the number do down to zero? Did the matching algorithm improve so that there were fewer false positives of noncitizens matched to the voter file? Also note the combined #2 and #3 declined from 67 (19.8% of self-reported noncitizens) in 2008 to 76 (15.6%) in 2010. To the first, we might add the 11 in category #1 since they are in the authors' universe. It seems like the change would be outside the sampling error, and might be due to improved matching algorithms finding fewer false positives.
One might say that election administration improved between 2008 and 2010 to knock more noncitizens off the voter rolls, which explains why there were fewer noncitizen registrants in 2010 compared to 2008. There is not a lot of evidence of that. Story after story of allegations of massive noncitizen voting end in a whimper when errors are found in the matching algorithms that uncovered the alleged noncitizen voting. It stands to reason if election officials have trouble with matching their voter files to identify noncitizens, Catalist might have similar problems when matching people who fit the profile of noncitizens (e.g., Latino names that are matched between the CCES and the voter file using fuzzy logic).
I want to correct something I wrote previously, there are 6 issue areas, not 8, analyzed in Appendix Table a3 (actually five since the authors drop one of the areas without comment). Since Table 3 reports responses for all noncitizens, and no noncitizen voters were in three of the issue areas, I can fill in the missing issue areas in Table a4
Fine Businesses
Noncitizen non-voters 35.3%
Noncitizen voters 0%
Increase guest workers
Noncitizen non-voters 47.1%
Noncitizen voters 0%
If one were to run a Chi-squared test to determine if noncitizen voters and nonvoters are different on the issues, then these areas should be included in the test. They are not.
============
Dr. Michael P. McDonald
Associate Professor
University of Florida
Department of Political Science
234 Anderson Hall
P.O. Box 117325
Gainesville, FL 32611
phone: 352-273-2371 <tel:352-273-2371> (office)
e-mail: dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>>
web: <http://www.electproject.org/> www.ElectProject.org<http://www.ElectProject.org>
twitter: @ElectProject
From: law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>> [mailto:law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>> ] On Behalf Of Michael McDonald
Sent: Friday, October 24, 2014 8:05 PM
To: 'law-election at UCI.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>'
Subject: Re: [EL] Uh oh, Rick...
I'd like to replicate the analyses to have a better criticism, but at first blush I am not as confident that the authors do a convincing job of showing that the people identifying in the survey as non-citizens are actually non-citizens who voted.
Those who favor voter id should welcome my critique, because on p.152 the authors claim that photo id requirements are ineffective to stop non-citizen voting.
The authors' evidence rests on two surveys, the 2008 and 2010 CCES surveys. The sample sizes of these surveys are 32,800 and 55,400. In 2008, 11 respondents identified as non-citizens who said they registered to vote and were matched to a voter list as being registered. In 2010 there were zero respondents in this category (Table 1, p.152). There is no match to verify if these individuals are really non-citizens or had fat thumbs when they pressed buttons on their computer, or even understood the questions (the CCES is an internet survey). To get higher numbers, the authors either use self-reports of non-citizens who self-reported being registered and were not matched to the voter files or reported not being registered and were matched with the voter files. I have many reservations about matching procedures and (from a legal perspective) would want independent confirmation from any of these three matching types that these individuals were in fact non-citizens who were registered to vote, especially since we are talking about small numbers of matches that could be a consequence of statistical flukes or other problems with the matching process.
The authors do not report if any of the 11 noncitizens who were validated as registered in fact voted (again granting that the matching algorithm didn't produce a false positive and that the respondents didn't misreport their citizenship status). They instead use their larger definition to find 48 non-citizens voted and 291 did not. As a percentage of the sample, even these numbers are exceedingly small and survey researchers generally would have less confidence in such small numbers. This is where next the heavy use of weighting that Vladimir references comes in.
Why I want to replicate their findings comes from their Appendix analyses, where the authors attempt to convince the reader that the non-citizen voters are really non-citizen voters. There's a sleight of hand at work here that strikes me as cherry picking of data. In Table A3 (p.156), the authors examine 8 issue areas to convince us that the non-citizens are different than citizens. In Table A4 (p.156) the authors attempt to show through a similar analysis that noncitizen voters are the same as noncitizen non-voters. Here, they present only 3 issue areas because there were zero non-citizen voters who answered affirmatively to 5 issues they choose not to present. They conclude on the 3 issues the non-citizen non-voters are the same as non-citizen voters, but that is obviously not true for the 5 other issue areas that they choose not to report. For this glaring cherry picking of evidence, I'm even more highly skeptical and want to do a full blown replication to see where else there may be issues with their methods.
============
Dr. Michael P. McDonald
Associate Professor
University of Florida
Department of Political Science
234 Anderson Hall
P.O. Box 117325
Gainesville, FL 32611
phone: 352-273-2371 <tel:352-273-2371> (office)
e-mail: dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:dr.michael.p.mcdonald at gmail.com<http://department-lists.uci.edu/mailman/listinfo/law-election>>
web: <http://www.electproject.org/> www.ElectProject.org<http://www.ElectProject.org>
twitter: @ElectProject
From: law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>> [mailto:law-election-bounces at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>] On Behalf Of Kogan, Vladimir
Sent: Friday, October 24, 2014 7:11 PM
To: Rick Hasen; Steve Hoersting
Cc: law-election at UCI.edu<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:law-election at UCI.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>>
Subject: Re: [EL] Uh oh, Rick...
I would be very careful about drawing broader conclusion about the incidence of non-citizen voting based on this study. The authors do a convincing job of showing that (1) the people who identified as being noncitizens in the survey are actually noncitizens and (2) those who say they voted actually voted.
I'm less convinced that we can generalize from this sample. The data is from the Cooperative Congressional Elections Studies survey, which is based on a non-representative opt-in panel from YouGov/Polimetrix. YouGov has a methodology for making their samples look like a random sample, and they have an excellent track record of predicting actual election outcomes. But I would be much more cautious about drawing conclusions about the representativeness of this sub-sub-population. As the authors themselves note, the educational levels among the noncitizens in their sample are much higher than the average among all noncitizens in the U.S. They try to use survey weights to get at this, but this only works as well as the demographics you're using to construct the weights. Unless you think the noncitizens who sign-up to be in the YouGov panel are representative of noncitizens who do not, I'm not sure how much this teaches us about the aggregate rate of non-citizen voting.
_______________________________________________
Law-election mailing list
Law-election at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election> <mailto:Law-election at department-lists.uci.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>>
http://department-lists.uci.edu/mailman/listinfo/law-election
--
Jonathan H. Adler
Johan Verheij Memorial Professor of Law
Director, Center for Business Law & Regulation
Case Western Reserve University School of Law
11075 East Boulevard
Cleveland, OH 44106
ph) 216-368-2535
fax) 216-368-2086
cell) 202-255-3012
<mailto:jha5 at case.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>> jha5 at case.edu<http://department-lists.uci.edu/mailman/listinfo/law-election>
SSRN: http://ssrn.com/author=183995
http://www.washingtonpost.com/news/volokh-conspiracy/wp/author/adlerj/
________________________________
NOTE: This message was trained as non-spam. If this is wrong, please correct the training as soon as possible.
Spam<https://www.spamtrap.odu.edu/canit/b.php?i=03N8uXd9w&m=dde07a3e9e61&t=20141028&c=s>
Not spam<https://www.spamtrap.odu.edu/canit/b.php?i=03N8uXd9w&m=dde07a3e9e61&t=20141028&c=n>
Forget previous vote<https://www.spamtrap.odu.edu/canit/b.php?i=03N8uXd9w&m=dde07a3e9e61&t=20141028&c=f>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://webshare.law.ucla.edu/Listservs/law-election/attachments/20141028/2441dfec/attachment.html>
View list directory