02-18-15 Part B - PRA Generic Clearence - Retirement Savings Module v5_OMB edits 2-252_KC

02-18-15 Part B - PRA Generic Clearence - Retirement Savings Module v5_OMB edits 2-252_KC.docx

MyRA Retirement Survey

OMB: 1505-0251

Document [docx]
Download: docx | pdf

Request for OMB Approval of a Generic Clearance for Collection of Information

Retirement Savings Module of the Household Financial Survey



SUPPORTING STATEMENT

B. COLLECTIONS OF INFORMATION EMPLOYING STATICAL METHODS

Treasury and the Center for Social Development (CSD) at Washington University in St. Louis will employ a variety of statistical methods in this research effort to ensure the validity and usefulness of findings. The Household Financial Survey (HFS), an already functioning, privately-funded, large-scale survey of low- and moderate-income (LMI) households, will serve as the platform for launching a new Treasury-funded module of survey items (the Retirement Savings Module) aimed at assessing participants’ reaction to the myRA program as well as collecting baseline data on retirement preparedness, attitudes, and expectations among the targeted LMI population. Based on previous years of experience, the team estimates that roughly 8,000 respondents will participate. These participants are drawn from the universe of LMI households who file federal income taxes through the TurboTax Freedom Edition (TTFE). Participation in the survey, and specifically the Retirement Savings Module, is fully voluntary. Statistical techniques will be used to test the representativeness of survey respondents compared to the universe of TTFE tax filers. In the event that the survey sample is not representative of the population universe, statistical techniques, such as sample weighting and propensity score analysis, will be employed to allow for valid inference to be drawn from the sample data and applied to the universe. The research is exploratory and largely descriptive in nature. Findings are not intended to be generalizable to a broader population, but rather specifically focus on LMI online tax filers with refunds. The information collected will inform efforts by the Department of Treasury in designing and administering public education outreach and other aspects of the myRA program.



    1. Describe Potential Respondent Universe

Respondent universe:
The respondent universe consists of households that file federal income taxes using the TurboTax Freedom Edition and that are owed a tax refund. Households qualify to use the program by meeting at least one of the following criteria: (1) household adjusted gross income below $30,000, (2) active military duty with household adjusted gross income below $60,000, or (3) qualification to receive the earned income tax credit. In the past several years, the number of households using TTFE and receiving a refund has been around 800,000. The universe is restricted to those households that have a federal income tax refund because the research will investigate the demand for using part of the tax refund to fund a retirement account.

This research is made possible by an existing collaborative relationship between Intuit, the makers of TurboTax, and a research team consisting of CSD at Washington University in St. Louis and Duke University. This team, known as the Refund to Savings (R2S) initiative, has researched the TTFE population extensively for the past three years, including the collection of HFS data beginning in 2013. The proposed research will consist of a new Treasury-funded Retirement Savings Module to be included in the 2015 version of HFS. The low-income criteria for the TTFE results in a population that aligns closely with the LMI population, one targeted population for the myRA program .



Establishments

State Government Units

Local Government Units

Households

Persons

0

0

0

800,000

0



Total:

Expected response rate:
The research team expects roughly 8,000 households to participate in the overall HFS, including the Retirement Savings Module, which represents a response rate of around 1 percent of the respondent universe. This estimate is based on past applications of the HFS survey in previous years. Response rates in past iterations of the survey have been highly correlated with the amount of monetary incentives offered to participants. An offer of a $20 prepaid debit card resulted in a response rate of approximately 10 percent, while the response rate was close to 1 percent when no incentives were offered.

Concerns about selection bias arise both with samples that respond to offers of incentives and those that respond in the absence of incentives. For the 2015 HFS, the research team proposes a hybrid approach to sampling that will limit the number of households that are invited to the survey but increase the response rate among those that are invited. Among those invited to participate, a monetary incentive will be offered. As mentioned above, the resulting respondent sample will be tested for representativeness with respect to the universe of TTFE tax filers.

The sampling procedure will essentially amount to a random selection of 1 in 2 TTFE tax filers for invitation to participate in the survey, which should equate to roughly 400,000 tax filers. Those that are invited will be offered a $5 incentive, which if historical trends continue, should result in about a 2 percent response rate from the invited sample. In total, the research team thus expects around 8,000 high quality responses that will represent a valid sample of the 800,000 tax filer universe of the study.

Collection conducted previously:
Similar privately-funded HFS collections have been conducted during the 2013 and 2014 tax filing seasons, although covering different material and not covered by the PRA clearance requirements. The federally-funded Retirement Savings Module will be included for the first time in 2015, but the past iterations of the survey provide data that has informed response rate and burden estimates, as well as evidence regarding the representativeness of survey respondents.

    1. Describe Procedures for Collection of Information

Statistical methodology for stratification and sample selection:
All tax filers using TTFE and receiving a federal income tax refund are included in the respondent universe. For this reason, stratified sampling will not be employed. Instead, a random sample will be invited to participate in the survey. Based on estimates discussed above, roughly half of the respondent universe will be invited to participate, and 2 percent of that sample will likely complete the survey when incentives are offered. The first selection (50% invited) will be random, while the second selection (2% complete) is self-directed, and thus not assumed to be random. However, the representativeness of the sample is an empirical question to be tested. In past iterations, the self-selected sample has been shown to be highly representative of the respondent universe based on observable factors, such as income, refund amount, age, and number of dependents. If the sample is found to suffer from sampling bias, sampling weights can be employed to correct for the problem.

Estimation procedure:
Estimates are based on historical data from previous versions of the HFS survey.

Degree of accuracy needed for the purpose described in the justification:
The expected sample size will support exploratory analysis and the production of descriptive statistics to address the research questions provided in Part A.

Unusual problems requiring specialized sampling procedures:
There are no unusual specialized sampling procedures being employed in this work.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden:
Data for this module will be collected at one point in time, coinciding with the date of tax-filing.



    1. Describe Methods to Maximize Response Rates

Methods to maximize response rates:
The main tool used to maximize response rates is the offer of monetary incentives to participants. As discussed above, this method has been used in the past and has been shown to produce response rates at least ten times higher than in instances where incentives were not offered. Invitations to the survey will be presented in their tax filing experience using TTFE and, thus, will hopefully be seen as coming from a trustworthy source. Clear instructions will be provided to potential participants at the outset of the survey, as well as an estimate of the time needed to complete the survey. Survey items will also be designed to be simple to understand and as brief as possible.

Accuracy and reliability of information collected:
The research team expects highly accurate and reliable data to be collected through this research. Data from past surveys has been matched to tax data as a validity check and has proven to be extremely accurate for the variables that can be matched.



Collections based on sampling:
The sampling technique to be employed is described above.



    1. Describe Any Test of Procedures or Methods

The methods employed for this research have been used successfully in past years. The research team developed the survey instrument, drawing on significant experience administering surveys of the targeted population and reflecting significant input from Treasury regarding the desired information to be produced. Many survey items are straightforward assessments of past or current retirement-related behaviors and are often framed as dichotomous response items (“yes/no questions”). Other items ask respondents to react to hypothetical products or focus on more abstract concepts, such as risk tolerance or confidence in retirement preparedness.

The guiding framework for instrument development focused on understandability and positive participate experience in addition to producing valid and reliable data. Questions were designed to be clear and unambiguous. Similarly formatted question have been used in previous iterations of the Household Financial Survey and very seldom have respondents reported issues with clarity of the questions or discomfort with the survey design.

Two additional techniques were employed to further ensure quality construction of the survey module: consultation with experts in the field and small scale pretesting with participants who would meet the qualifications of participants in the R2S study. First, the research team solicited feedback on the general framework of the survey during an in-person meeting with experts in research and policy surrounding household finances and asset building. Many of these experts also have extensive practical experience in instrument development, survey administration, and data analysis of households. These experts include:

  • Clinton Key  - Pew Charitable Trust

  • Rourke O’Brien – Harvard University

  • Amy Brown - The Ford Foundation

  • Frank DeGiovanni – The Ford Foundation

  • Keith Ernst – Federal Deposit Insurance Corporation 

  • Beadsie Woo – Annie E. Casey Foundation

  • Jennifer Tescher – Center for Financial Services Innovation

  • Ben Keys – University of Chicago

  • Ray Boshara – Federal Reserve Bank of St. Louis

After incorporating extensive feedback from these experts and Treasury officials, select individuals (Key, O’Brien, Brown, DeGiovanni, and Ernst) were provided drafts of the survey and asked to make additional suggestions to improve accurate measurement, clarity for participants and quality of the design. Recommendations from the experts were incorporated into the draft survey instrument.

The survey draft was then programmed into Qualtrics survey software for pretesting. A convenience sample of less than 10 participants was recruited to complete the survey and provide extensive feedback to the research team. Participants, who met the income criteria for participation in R2S (i.e. household income less than $30,000) were asked to consider:

  • Question clarity and language comprehension

  • Intent of each question (i.e. what the participant perceives the question is asking)

  • Response options (unnecessary options; missing options)

  • Order of questions and overall user experience

  • Whether any of the questions made them uncomfortable

To stimulate the experience as closely as possible, the pretest instrument was emailed to participants and they were asked to complete it online in a location of their choosing. The research team collected information from pre-testers both in writing and verbally by following up with pretest participants in-person and via phone to collect feedback. In addition, the Qualitrics software allowed the research team to examine the length of time spent on each question in order to further evaluate the usability of the instrument. Feedback from pretest participants was used to improve the survey instrument.

Based on the extensive survey development undertaken, the pretesting conducted, and the relatively straightforward nature of the questions, the research team is confident that the instrument is solid and will produce valid and reliable data.



    1. Provide the name and Number of Individuals Consulted on Statistical Aspects of the Design and Name of Agency Unit

Several individuals have been consulted on the statistical aspects of the research design.

Michal Grinstein-Weiss, PhD

Position: Associate Professor, Washington University in St. Louis

Phone: 314-935-8201


Shenyang Guo, PhD

Position: Professor, Washington University in St. Louis

Phone: 314-935-3092


Blair Russell, PhD

Position: Senior Statistical Data Analyst, Washington University in St. Louis

Phone: 314-935-3710

3


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorBlair Russell
File Modified0000-00-00
File Created2021-01-26

© 2024 OMB.report | Privacy Policy