CTAS Supporting Statement B FINAL

Supporting Statement B

Internal Revenue Service

Research Applied Analytics & Statistics (RAAS)

Comprehensive Taxpayer Attitude Survey

OMB # 1545-2288

B. STATISTICAL METHODS

1. Universe and Respondent Selection

Taxpayers over the age of 18 that match the demographic characteristics of the U.S. population will be reached via random digit dialing and the vendor’s representative online panel.

2. Procedures for Collecting Information

The 2020 CTAS continues the previous multi-mode data collection methodology, comprised of telephone and online panel random sampling to ensure a representative sample of the general public. IRS’s contracted research firm, PCG, will collect a total of 2,000 completed surveys, split evenly between the telephone and online data collection methodologies, for a margin of error of +/- 2% at the 95% confidence level. The data will be collected using a telephone survey of approximately 500 landlines and 500 cell phone respondents as well as an online survey of 1,000 additional respondents. The survey is the same as in 2019, except for a revised PRA statement that has been moved to the front, per OMB’s preference, and will be an average of 20 minutes (phone)/10 minutes (online) with no open-ended questions.

Potential respondent universe

Adult population (18 years or older) in the United States

Numerical Estimate of the respondent universe

253,881,929, based on population estimates from US Census Bureau for 2018.¹

Number of Entities

Data Collection Method	Potential Respondent Universe (adults 18+ in US)	Requested for participation (based on last year's numbers)	Participants in the survey (survey sample size)
Phone	253,881,929	37,470	1000
Online	253,881,929	1,800	1000
Overall Total	253,881,929	39,270	2000

 

Telephone Methodology

Respondent Selection Method: The telephone sampling approach will include random digit dialing (RDD) of households with landlines in the continental U.S., augmented by an RDD sample of cell phone numbers in order to capture the opinions of U.S. adults in cell phone only (CPO) households. The phone survey participants are selected via RDD of household telephone numbers from working phone area codes and exchanges and by computer generation of the last 4 digits. Since the RDD sampling system is totally computer-based, it provides an equal probability of selection for each and every telephone household. Thus, the sample represents telephone households with both listed and unlisted phones in their proper proportions.

In order to reflect current usage patterns in the U.S., we will collect 500 interviews via RDD from a traditional landline telephone sampling frame, and 500 interviews via RDD from a sampling frame of cell phone numbers, including cell phone only (CPO) and dual users. The total telephone sample size will be 1,000 completed interviews, with a margin of error equal to +/- 4% at the 95% confidence level.

All telephone interviews will be conducted by PCG’s phone subcontractor who will work under the leadership of PCG’s project manager. All interviewers will be professionally trained in interviewing techniques and in IRS surveys prior to the commencement of fielding, and they will be continuously monitored and supervised throughout the fielding period. The interviewing methodology used will be Computer Assisted Telephone Interviewing (CATI). All data are entered and cleaned through the CATI system during the interviewing process, which eliminates editing and keypunch operations. A screening question ensures that respondents are at least 18 years old. For those agreeing to participate in the survey, interviewers administer the survey questionnaire guided by the computer assisted telephone interviewing (CATI) process. All sample phone numbers selected by the sampling procedure are subject to an initial call, followed by at least four follow-up attempts to complete an interview. Last year, the response rate was 2.1% for landline phones and 3.5% for cellphones. Phone response rates are declining every year, so we expect the response rate to be similar to or a little lower than last year’s response rate. Similar to previous years, we expect the 2020 response rate to be around 2.1% for landline phones and 3.5% for cellphones, or a little lower, since phone response rates are declining every year.

Online Methodology

In conjunction with telephone data collection, PCG is planning to collect a total online sample size of 1,000 completed online surveys, with a margin of error equal to +/- 4% at the 95% confidence level.

PCG will program a standard online version of the questionnaire. The survey ‘look and feel’ will have a professional design as used in the 2019 survey and will not identify the IRS as the project sponsor. The online survey will be thoroughly tested internally at PCG to ensure the programming reflects the survey content (questions, response options) as well as any skip logic. These skip patterns are built into the survey program code and are tested along with the survey content. PCG will test each branching of the skip patterns to ensure that the question sequence is programmed correctly. A test survey link will then be shared with the RAAS team for review and comments. PCG will revise the online program as needed based on RAAS feedback. PCG will also send the email invitation text to RAAS for review.

In addition to the standard version of the programmed survey, PCG will also program a second version of the survey that will conform to the technical standards for web-based Intranet and Internet Information and Applications of Section 508 of the Rehabilitation Act of 1998. Having the two versions ensures the best user experience for those reading the survey themselves and for those using JAWS software to read the survey to them. This survey version will also be thoroughly tested internally at PCG, before providing RAAS with a survey link for their own testing process.

Respondent Selection Method: In order to achieve survey data that is representative of the U.S. adult online population, PCG has subcontracted with Ipsos to provide the online sample from their probability based online panel, KnowledgePanel^®. This panel uses an Address-Based Sampling (ABS) methodology which is random-by-mail to recruit members and each quarter a stratified random sample of addresses is selected to replenish the panel. The sampling frame used is the universe of all U.S. residential addresses secured from the latest Delivery Sequence File of the U.S. Postal Service. Panel members are randomly selected to participate in this study.

Ipsos will send out survey invitations to the panel respondents via email containing a link to the survey on the PCG server. Ipsos will send invitations and reminders about a an “opinion survey” with no details regarding IRS sponsorship. We expect a completion rate of at least 50% for this survey (the completion rate in 2019 was 61%). The survey has been confirmed as non-FTI (Federal Taxpayer Information), it is completely anonymous, and no individual respondent will be providing personally identifiable information.

Weighting Strategy

PCG will follow the same weighting strategy as was utilized in the Base Period. The survey data from each data collection mode will be weighted separately to allow for analysis of each sample separately, and comparatively. The two samples will also be combined, and an additional ‘blended’ weight will be used.

The entire telephone sample, consisting of landline and cell phone samples, will be weighted on demographics and telephone usage variables to demographically represent the adult U.S. population according to the most recent national population estimates. This weighting scheme will account for dual cell phone and landline users.

The online sample will be weighted using demographic variables only.

In merging the telephone and online survey datasets, PCG will use a blended weighting to achieve a final dataset that matches the U.S. adult population on CPS-based demographic variables. This weighting will be based on demographic variables as well as behavioral/attitudinal questions about internet and television use as well as attitudes about trying new products.

Analytical Planning

PCG will review an interim survey dataset in the latter stage of fielding. The dataset will be cleaned and basic frequencies for each question will be generated. A review of this initial analysis in addition to our experience analyzing last year’s data will lead to PCG’s analysis and reporting plan for the final dataset(s). PCG will share this plan with RAAS and a joint-team meeting will occur to discuss the proposed plan and whether it sufficiently meets RAAS’s needs.

3. Methods to Maximize Response

For phone surveys, up to five attempts will be made to reach each selected household (an original attempt, plus four more attempts to reach households that did not answer earlier calls). All attempts will be made during evening and weekend hours, since those are the times when working respondents are most likely to be at home. Calling attempts will be scheduled for different days of the week and weekends and will be spaced as far apart as is possible, within the restraints of the survey schedule.

For online surveys, points may be offered by the contractor as a participation incentive, which can be redeemed for merchandise, gift cards, or other items. The contractor will:

• keep surveys open for a minimum of 7-10 days;

• and deliver a reminder to non-completes.

The survey mode has been enhanced by the addition of an online panel to explore the feasibility of realizing cost savings to the government in the future, while retaining the comparability and historical integrity of prior year results and trend lines.

4. Testing of Procedures

The vendor will fully brief field staff responsible for data collection and the fieldwork will be monitored. This is an established survey that has been conducted for many years and which reflects the extensive experience of IRS and the vendor in conducting this and similar surveys. The 2020 questionnaire has been changed only to update the PRA statement and move it toward the front of the questionnaire, per OMB’s preference; therefore, no pretesting is necessary.

5. Contacts for Statistical Aspects and Data Collection

Timothy S. Castle

Chief, Servicewide Support Section

IRS/Statistics of Income Division

Research, Applied Analytics & Statistics

77 K Street, NE

Washington, DC 20002

Donna Baldwin

Senior Program Evaluation & Risk Analyst

IRS/Statistics of Income Division

Research, Applied Analytics & Statistics

77 K Street, NE

Washington, DC 20002

1 https://www.census.gov/quickfacts/fact/table/US/PST045218

4