Supporting_Statement_Part_B_2014_Survey_of_LEOs_08-05-14

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.  Description of the Activity

Sampling frame. The sampling unit, or unit of analysis, of the 2014 Post-Election Voting Survey of Local Election Officials is the local election voting jurisdiction, which for most States is the county but is defined differently from State to State. For example, the States of Alaska and Maine are considered single voting jurisdictions for UOCAVA purposes, whereas Michigan, Wisconsin and the New England States define voting jurisdiction as the individual township. The remaining States define voting jurisdictions as counties with the exception of Illinois and Virginia. These two States define voting jurisdiction mainly by county but also include some cities among their voting jurisdictions. DMDC developed the sampling frame from a file provided to DMDC by FVAP. When compiled, there are 7,303 403 unique voting jurisdictions in the United States and the four territories.

Sample Design, Target Population, and Expected Response Rate. For the 2014 Post-Election Voting Survey of Local Election Officials, 1,500 LEOs will be sampled and contacted.¹ The sample of 1,500 will be designed to represent all 7,303 403 voting jurisdictions. The population will be stratified into six strata:

1. Jurisdictions with more than 25,000 registered voters

2. Jurisdictions with at least 5,000 but fewer than 25,000 registered voters

3. Jurisdictions in Michigan or Wisconsin with at least 1,000 but fewer than 5,000 registered voters

4. Jurisdictions from all other states with at least 1,000 but fewer than 5,000 registered voters

5. Jurisdictions from Michigan or Wisconsin with fewer than 1,000 registered voters

6. Jurisdictions from all other states with fewer than 1,000 registered voters

Michigan and Wisconsin are separated because they have 3,428 jurisdictions and DMDC does not want the sample to contain too many jurisdictions from these states. DMDC will select 250 jurisdictions from each of the largest four strata, 350 will be selected from the stratum of small jurisdictions from Michigan and Wisconsin, and 150 will be selected from the stratum of small jurisdictions from the other states. It is anticipated that 60% of the contacted LEOs will submit a survey with at least 50% of the applicable questions having been answered.

Stratum	Label	Population	Sample Size
1	MI & WI, Fewer than 1000 registered voters	2,393	350
2	all other states, Fewer than 1000 registered voters	302	150
3	MI & WI, 1000-4999 registered voters	772	250
4	all other states, 1000-4999 registered voters	820	250
5	all states, 5000-25000 registered voters	1 943	250
6	all states, more than 25000 registered voters	1,173	250
	Total	7,403	1,500

Response Rate Calculations. Response rates will be calculated using AAPOR RR3, whereby a completed survey is determined by a respondent completing at least 50 percent of the applicable questions and where (Cases of Unknown Eligibility) is an estimate of the non-responding jurisdictions with an unknown eligibility status. However, for the 2014 surveys, all jurisdictions are considered to be eligible.

Completed

[(Completed + Partial Interviews) + (Non-interviews) + e(Cases of Unknown Eligibility)]

To handle unit non-response, DMDC will compute nonresponse adjustment factors within weighting classes defined by State, jurisdiction type (county versus minor civil division) and size classes (number of registered UOCAVA voters). DMDC will collapse cells with too few jurisdictions. This weighting adjustment reduces bias from differential responses rates on these three characteristics.

DMDC will produce statistical estimates of overall percentages, and for States with sufficient numbers of responding jurisdictions, and will conduct basic checks of potential bias due to nonresponse which will be noted in the final Statistics and Methods report produced for this survey.

2.  Procedures for the Collection of Information

Participant recruitment. DMDC will obtain names, postal mailing addresses, e-mail addresses, and telephone numbers of all the current LEOs from FVAP’s administrative staff.

All prospective respondents will be notified of the upcoming surveys by postal mail approximately two weeks prior to the start of data collection. The letters, all of which will include the signature of FVAP’s Director, will describe the purpose of the survey and will include the name of the sponsor, a toll-free phone number to call with questions about how to participate, a unique Ticket Number, and the URL for the website the LEOs can log onto to update their contact information if desired.

An announcement e-mail will be sent to all prospective respondents the day after Election Day (November 5, 2014) informing the LEOs data collection has begun and will include their unique Ticket Number and instructions on how they can log onto the survey website to either complete the web-based survey.

Throughout the course of the field period, the survey operations contractor will maintain a case control system which will be updated daily for web receipts to determine non-respondents. Only those LEOs who have not responded to their respective data collections will be sent any reminder contacts: up to five (5) reminder e-mails and two (2) reminder postal letters. The field period will last approximately four (4) weeks.

The postal letters and emails are included as part of this submission.

The data collection procedures are not expected to involve any risk to participants. Names are used only in communicating with LEOs. These names are kept securely by the survey operations contractor and are not linked to response data. The datasets sent to DMDC contain no names or addresses.

Data collection: Data collection for the survey of LEOs will consist of a web-based instrument that selected LEOs will access by navigating to the survey operations contractor’s web site and entering their unique Ticket Numbers which have been provided to them in the postal letters and emails they have received. Respondents will be directed to a set of web screens, starting with the “Welcome” screen. From there they can view the “Frequently Asked Questions” screen, the “Security Protection Advisory”² screen, and the “OMB Number” screen. The Advisory informs respondents that no information on the individual’s computer or Internet connection is collected in a way that can be associated with the individual or his/her survey responses. Respondents are then directed to the “Agency Disclosure Notice and Privacy/ Informed Consent” screen. This last screen includes the instruction "Click 'Continue' if you agree to do the survey" and informed consent is indicated by clicking the "Continue" button and by answering the survey questions and submitting the survey. This submission includes the following documents to be used for the web-based survey of LEOs Questionnaire; Welcome screen; Frequently Asked Questions screen; OMB Number screen; Security Protection Advisory screen; Agency Disclosure Notice and Privacy Notice.

The survey functions as a web-based instrument whereby respondents are guided through it screen-by-screen via a program that includes any appropriate skip logic. At the close of the field period, the data will be compiled into a single SAS dataset after appropriate cleaning by the data collection contractor. Datasets are then transmitted to DMDC via secure file transfer protocol.

Weighting. The analytic weights for the survey of LEOs will be created to allow for the estimation of population values by eligible survey respondents. DMDC will create survey weights to reflect the initial selection probabilities as well as the adjustments for the differential response rates. LEOs will receive base weights equal to the total number of jurisdictions, 7,303, divided by the number of jurisdictions in the sample, approximately 1,500. After the survey has been conducted and the case dispositions are resolved, the sampling weights for survey will be adjusted for non-response. The eligibility-adjusted weights for eligible respondents will be adjusted to account for eligible jurisdictions that were non-respondents. For this survey we expect all sample jurisdictions to be eligible.

Data security. The 2014 Post-Election Voting Survey of Local Election Officials does not collect or use personally identifiable information and data are not retrieved by personal identifier. Therefore, the information collected is not subject to the Privacy Act of 1974, as amended. Only aggregate data will be reported in the form of statistical summaries.

The network sites for both DMDC and DMDC’s survey contractor, Data Recognition Corporation (DRC), are secure and password protected. Security is strictly enabled by using physical and software access restrictions. All servers are physically located in locked rooms with access permitted only to Technical Services staff through the use of a security card system. Access to the network is allowed only through a login account and password. In addition, employees use password protected screen savers at workstations to protect their systems while they are away from their desks. At DMDC, the network is accessed through the use of Common Access Card (CAC) readers and utilizes Public Key Infrastructure (PKI) security. Logging on to the network requires both physical possession of the CAC and a separately issued Personal Identification Number. All computer systems comply with current Federal Information Security Management Act security standards.

DRC makes daily backup tapes that are stored for five years in fire proof vaults located within a security-card protected area, and all provisions dealing with the protection of human subjects and data security are in force for as long as the contractor retains any protected data. DRC meets the Department of Defense Information Assurance Certification and Accreditation Process (DIACAP) requirements and conducts all necessary updates on an on-going basis to continue to meet DIACAP requirements. DRC’s facilities have an Authority to Operate (ATO) issued by DMDC.

Unusual problems requiring specialized sampling. No unusual problems that would require specialized sampling for any of the component parts of the 2014 Post-Election Voting Survey of Local Election Officials are anticipated.

Use of periodic data collection cycles to reduce burden. This request is for all data collection to take place immediately after the 2014 Federal Election, and therefore it cannot be reduced further.

3.  Maximization of Response Rates, Non-response, and Reliability

Encouraging response. To maximize response rates, DMDC has developed a set of survey-specific postal letters and e-mails that will be sent out to all selected survey participants (See Section B.2. on Participant Recruitment) throughout the field period. The letters will be on FVAP letterhead, signed by FVAP’s Acting Director, and will be personalized; that is, addressed to the specific LEOs identified by FVAP as the contact person for their jurisdiction. The selected individuals will be notified of the upcoming survey by postal notification letter approximately two weeks prior to the start of data collection. The letter explains the purpose of the survey, includes a telephone number the individual can call to confirm the legitimacy of the survey, and informs the individual that participation is voluntary. The notification letter will be followed by an announcement e-mail sent the day after Election Day (November 5, 2014), informing the LEOs data collection has started and includes clear instructions on how to participate. Throughout the field period, the non-responding LEOs will be sent reminder e-mails and reminder postal letters with the same information. DRC now has the capability of sending out emails that are digitally signed on behalf of the Department of Defense, and it is expected that this feature will add to the legitimacy of the surveys.

Analysis of survey non-response. DMDC and OMB are concerned with general declining response rates in the survey industry overall, and in DoD-sponsored surveys in particular, and what, if any, impact such a decline may have on the validity of the reported estimates. To address this concern, OMB issued standards and guidelines for Federal statistical surveys requiring that, for any survey with a response rate below 80 percent, survey agencies conduct a non-response analysis. While a formal non-response study will not be conducted, the application of weights will be used to adjust for possible non-response. And while the 80% threshold will likely not be achieved, this survey has among the highest response rates (60%) of any current survey that DMDC conducts.

4.  Tests of Procedures

DMDC utilizes best practices in its design of web-based surveys (e.g., visual presentation of questions and response options, usability and interactive elements, use of color, font style and size, and screen layout). These features of web-based data collection, in addition to automated skip logic, serve to ease the burden on the respondents, increase data quality, and minimize response error. Similarly, DMDC has incorporated a number of randomized experiments into its surveys over the past several years testing such conditions as number and wording of letters and e-mails, timing of respondent contacts, presence or absence of a brochure, sponsorship, and subject line text.

5.  Statistical Consultation and Information Analysis

FVAP Principal Investigator: Matt Boehmer

Director, Federal Voting Assistance Program

571.371.0727

[email protected]

FVAP Deputy Director for Technology Programs: David Beirne

571-372-0740

[email protected]

FVAP Deputy Director of Election Official Assistance: Paddy McGuire

571-372-0739

[email protected]

FVAP Program Analyst: Samantha Walker

571-372-0741

[email protected]

DMDC Principal Investigator: Kristin Williams

571-372-1033

DMDC Project Manager: Kimberly Hylton

571-372-1113

DMDC Survey Statisticians:

• David McGrath - 571-372-0983

• Eric Falk - 571-372-1098

• Phil Masui - 571-372-1124

• Fawzi Al Nassir - 571-372-1114

DMDC Analysts:

• Elizabeth Davis - 571-372-1105

• Abbie Moore - 571-372-0755

Survey Operations Contractor: Data Recognition Corporation (DRC) (800-826-2368): Contact person: Valerie Waller (763-268-2166)

1 For the 2014 data collection, DMDC will not be conducting the Quantitative survey. All methods for the Qualitative survey are the same as the methods used in 2012.

2 Because the data are collected on a website, the website is required to include Security Protection Advisory information according to the Office of the Secretary of Defense Policy for Establishing and Maintaining a Publicly Accessible Department of Defense Web Information Service (dated July 18, 1997; updated January 9, 1998).

6