SUPPORTING STATEMENT – PART B
Overseas Citizen Population Survey: 0704-0539
Description of the Activity
The Federal Voting Assistance Program (FVAP) issued Request for Quote for research and analysis services to examine the demographic profile of registered overseas civilian voters, quantify the population of these voters by country, and calculate a voter registration and participation rate for the population using records from state and local election officials. The component of this research covered under this Information Collection submission pertains to a survey of the overseas U.S. civilian registered population, focused on voting-relevant characteristics such as educational attainment, income, and mobility. This data will allow FVAP to compare with greater certainty the voting behaviors of overseas registered U.S. civilians to those of the registered civilian voting age population (CVAP) and active duty military (ADM) population who share similar characteristics. The survey will be conducted with 45,000 registered voters who requested a ballot to an overseas address in the 2024 General Election. It is anticipated that the survey will yield a response rate of approximately 16%, for a total of 7,200 completed surveys, based on the 2018 OCPS (the last survey following a midterm election year) and factoring in response rates due to the election type and contact methodology. The 2022 survey is currently being fielded.
Procedures for the Collection of Information
The following activities will be conducted as part of this survey data collection effort:
Revise a survey of overseas citizens
Administer the survey
Manage received survey data
Analyze the survey data
FVAP will collect data from the overseas citizen population through a web only survey, with paper communication materials, of a sample of registered civilians who requested a ballot to an overseas address in the 2024 General Election. Conducting such a survey will allow FVAP to collect specific, accurate information on voting relevant demographic variables (such as age, gender, race/ethnicity, education, and mobility) to make the comparisons between the overseas, domestic, and ADM populations that are so important to FVAP’s mission. This collection will also provide important information on voting related behaviors that might help address complex and/or unexpected findings within and across countries. Specifically, this data will address overseas voter awareness of FVAP, use of and satisfaction with FVAP services and other resources that these voters use. The survey instrument is designed to parallel FVAP’s Post-Election Survey of ADM and the Census’ Current Population Survey (CPS), facilitating FVAP’s ability to compare the registration and voting behavior of the overseas U.S. civilian population, domestic CVAP, and ADM. The 2024 survey will use a new instrument based in large part on the 2022 survey.
The survey will utilize a mixed-mode approach in which respondents will receive an initial mail contact directing them to a web site where they can complete an online survey. The 2022 survey will be web-only, consistent with the methodology used in 2022. Respondents who do not respond to the online survey will then be sent mail and email reminders. This mixed-mode design1 has significant benefits over soliciting potential respondents by email. Specifically, it ensures that all registered U.S. civilians living overseas have a known probability of being contacted and having the potential to participate, rather than just those with a listed email address; as a result, this increases the likelihood that the final sample of respondents will be more representative of this population with respect to Internet use/access. In addition, costs can be high for printing and mailing paper surveys, especially to an international population; pushing respondents to complete an online survey will lower costs while still capturing a representative sample of the overseas citizen population.2
The communication strategy will be a two-pronged approach involving up to eight international mailings and up to two email messages per participants with email addresses, for a total of eight communications.
The physical materials sent to each voter’s overseas address will include:
One cover letter including the URL for the online survey.
Two reminder cover letters including the survey URL.
Five standard postcards including the survey URL.
If these mailings do not result in a survey response or other contact, follow-up postcards may be sent, including sending a postcard to the individual’s domestic address of record. It is anticipated that this will increase the overall response rate by including responses from some individuals who were residing overseas during the 2024 General Election but have subsequently returned to the United States.
It is expected that around 20% of the sample frame will have a valid email address. Those with a valid email address will receive two emails in place of two international mailings, for a total of eight communications. The email contacts will correspond to the physical mailings, using similar wording and design choices. Emails will include:
An initial email message including the URL for the online survey.
A second, similarly worded email message including the survey URL.
The physical and electronic mailings will be sent concurrently. The reminder contacts will be sent approximately once every two weeks, with the day of the week varying for each contact and/or to reflect international holidays. Once a respondent has completed the survey or unsubscribed, no further communications will be sent.
Because the survey frame will not include email addresses for all overseas registered voters, the post survey analysis will appropriately weight the email and non-email samples to control for potential error introduced by this two-pronged approach. Using all means of contact available will help mitigate the effects of uncertain overseas mail delivery and other obstacles that may arise from trying to contact a population that has been historically difficult to survey.
The survey will be administered to a sample of 45,000 respondents. Each respondent will receive an invitation to participate in a web-based survey.
Estimation Procedures.
Sampling probabilities will be known at each stage of the sampling process and will be incorporated into the analytic weights used for estimation. Sampling design decisions, including the decision of how to allocate the sample across different domains, are key aspects of a study design and have meaningful implications for survey estimation, often involving competing goals and tradeoffs among options. The sampling plan for this study cannot be created yet because the survey frame consisting of absentee voters in the November 2024 general election has not yet been assembled. However, the 2024 sampling plan will use the same criteria and methodology from 2022 as described below.
In this study, one pair of competing goals involves the desire to make estimates of sufficient precision at the country level, while also aiming to produce accurate estimates overall. Thus, although a proportional allocation of sample across countries could lead to more accurate estimates overall, it will be necessary to use disproportionate sampling in order to achieve necessary levels of precision for voters in smaller countries and world regions.
For this study, variables that will be used for stratification and/or in determining selection probabilities are geography (e.g., country of mailing address and State in which the voter is registered) and voter characteristics as measured on the voter file (e.g., demographics and vote history).
After the survey data has been collected, full-sample weights will be developed. At a minimum, these weights will account for differential selection and response rates across strata.
Sample weighting will accomplish the following objectives:
Compensate for differential probabilities of selection among sample members.
Reduce biases occurring because the characteristics of non-respondents may have been different from those of the respondents.
Improve the precision of the survey-based estimates.3
After data collection is complete, experienced project leaders will oversee and direct each analysis (including planning and execution) to ensure proper procedures are applied, utilize reasonable analytical assumptions, and apply the correct approach given the characteristics of the data and objectives of the analysis. All analysis will be repeated by a second analyst to ensure error-free results. Some of the approaches used will include:
Regression analysis (OLS, 2SLS, logit/probit, etc.).
Machine learning model building (CART).
Geographic analysis including thematic and LISA maps.
Degree of accuracy needed for the Purpose discussed in the justification. The goal of using the survey data is to support an analysis of voting rates similar to that applied to the comparison between the ADM and domestic civilian population using the Post-Election Voting Survey of the Active Duty Military. For a comparison between overseas and domestic civilians to have a similar degree of reliability, the number of survey completes for this survey should be similar to the number of completes for the PEVS-ADM. Consequently, the aim will be to obtain approximately 7,200 completed surveys. Assuming an 16% response rate this implies that approximately 45,000 total survey invitations should be sent during the survey fielding.
Unusual problems requiring specialized sampling procedures. Not applicable.
Use of periodic or cyclical data collections to reduce respondent burden. Not applicable.
Maximization of Response Rates, Non-response, and Reliability
Like all surveys, there is concern that non-response, both to the survey as a whole as well as to specific questions, will undermine the accuracy of estimated voting propensity of the overseas registered population, and thus the validity of any comparisons made with the CVAP. Low response rates could undermine the accuracy of estimates of participation propensity by increasing sampling variability and thus uncertainty in how well the sample reflects the greater population. This problem would be reflected in larger confidence intervals around the estimates. More problematic, if non-response is systematically related to unobserved demographic or geographic characteristics that are also relevant to voting, is that the estimated model of participation may be invalid even if the final sample is large. The final report detailing the findings of the 2024 OCPS will include a non-response bias analysis based on the sampling frame. Though methodological changes are not expected to have a significant effect on the overall response rate, the survey instrument was revised to maximize the total number of cases with valid responses on questions such as residency and mobility that are necessary for inclusion in key analyses of interest.
Tests of Procedures
Online survey logistics: The online survey will be thoroughly tested by employees of the contractor who will enter at least 20 test surveys through the web-based instrument. Should the data check reveal errors, necessary changes will be made to the web-based technology and the checks will be conducted again until 100% accuracy is achieved.
Communication strategy: Previous iterations of the OCPS have demonstrated that a lengthy survey period is advisable, which allows respondents with less Internet access or in hard-to-reach mailing geographies enough time to respond to the survey.
Statistical Consultation and Information Analysis
Provide names and telephone number of individual(s) consulted on statistical aspects of the design.
Chris Williams
Fors Marsh Group, LLC
Phone: 501-658-2310
Email: [email protected]
Sidney Turner
Fors Marsh Group, LLC
Phone: 703-517-8980
Email: [email protected]
Provide name and organization of person(s) who will actually collect and analyze the collected information.
Chris Williams
Fors Marsh Group, LLC
Phone: 501-658-2310
Email: [email protected]
Sidney Turner
Fors Marsh Group, LLC
Phone: 703-517-8980
Email: [email protected]
1Lonna Rae Atkeson, Alex N. Adams, and R. Michael Alvarez, “Nonresponse and Mode Effects in Self- and Interviewer-Administered Surveys”, Political Analysis, published online May 28, 2014, doi: 10.1093/pan/mpt049.
2 Approximately 84% of participants in the 2014 OCPS completed their survey online and there were minimal differences in responses between modes.
3Skinner, C. J., Holt, D. & Smith, T. M. (Eds.). (1989). Analysis of complex surveys. New York: Wiley.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Patricia Toppings |
| File Modified | 0000-00-00 |
| File Created | 2023-09-21 |