Making Ends Meet Survey
OMB Control Number: 3170-00XX
OMB Expiration Date: XX/XX/XXXX
Supporting Statement b For
Making Ends meet survey
OMB CONTROL NO.: 3170-00XX
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.
The Consumer Financial Protection Bureau (“CFPB” or “Bureau”) has acquired a nationally representative sample of de-identified consumer credit records (the “Consumer Credit Information Panel” or “CCIP”) from one of the three national credit reporting agencies. The basis for the sample is a 1-in-50 random sample of all credit records drawn from the credit reporting agency’s archive from December 2012. At the end of each month, the CFPB receives updated credit records for these sampled consumers (if available) and a 1-in-50 sample of credit records that were newly created since the previous quarter. This sampling process was designed to provide an ongoing panel of credit records that remains representative of the universe of credit records at each point in time. The contract with the credit reporting agency also allows for CFPB testing to verify that the panel remains representative.
The de-identified credit records that the CFPB receives carefully exclude any direct identifying information to maintain the anonymity and protect the privacy of sampled customers. The records include information about the types of credit accounts, such as mortgages and credit cards, which are included in each consumer’s credit record (though the identity of the creditor is excluded from the information the CFPB receives). The data also include information on non-credit-related debts that have been reported by third-party collection agencies, monetary-related public records (such as tax liens and bankruptcy filings), and details of any credit record inquiries made by lenders in response to an application for credit. The credit information in the CCP is used to monitor conditions in consumer credit markets, to study consumer behavior regarding credit, to evaluate the effects of consumer regulations, or to address other issues in support of the CFPB’s research, monitoring, and supervisory missions.
The sample for this
information collection will consist of two samples from the CCP. One
will consist of the approximately 2,200 consumers who responded to
the Making Ends Meet survey fielded in early 2023. Based on the
response rates to recent follow-up surveys, the CFPB conservatively
expects 1,500 respondents. The second sample will consist of a new
sample of consumers from the Consumer Credit Panel. The sampling
process may include sampling schemes other than simple random
sampling, including stratification or oversampling populations of
interest (such as consumers with low credit scores and consumers of
different age groups). The additional sample is anticipated to be
sent to approximately 15,000 people. Based on historical response
rates we conservatively expect approximately 3,000 total responses.
Survey weights would reflect the sample design and would be adjusted
to account for systematic differences in nonresponse along dimensions
measured in the CCP.
As described above, we plan to survey a sample of individuals with credit records that are included in the CCP. The persistent consumer record numbers for sampled credit records will be sent to the credit reporting agency. The credit reporting agency will identify the consumers associated with each sampled credit record and will mail the survey instrument which was designed by the CFPB staff. The credit reporting agency’s subcontractor will receive the mail or online responses, remove any direct identifying personally identifiable information (PII) that may have been included in the consumer’s response, and send the de-identified data in electronic form to the CFPB. This process allows the CFPB to survey consumers without revealing direct identifying PII to the CFPB and has been successfully used for the National Survey of Mortgage Originations (NSMO), American Survey of Mortgage Borrowers (ASMB), the Survey of Consumer Views on Debt, and the previous rounds of the Making Ends Meet survey.
The field period will be eight weeks and include up to six first-class mailings. The CFPB expect all sampled consumers receive an initial mailing with a cover letter introducing the survey along with a paper questionnaire, postage-paid return envelope, and $5 cash incentive. The cover letter will also include an offer of $20 to be received upon completion of the survey, and $10 to be received if that completion occurs electronically. A reminder letter will be sent one week later. In week five, consumers who have not yet completed the survey or opted out receive a second reminder letter with a replacement questionnaire, postage-paid return envelope, and a reminder of the offer of $20 to be received upon completion of the survey (along with the additional $10 to be received for an online completion). A final reminder letter is sent to remaining non-respondents in week seven. As noted below, we may adjust our incentive structure according to a review of best practices in the field and/or consultation with experts in survey methodology. The final two mailings consist of letters providing the respondents with their post-completion incentive payments.
Obtaining sufficiently high response rates is a challenge for any survey. The CFPB intends to include a cash incentive with each survey to boost response rates. Additionally, the CFPB will incorporate lessons learned by the CFPB from the pilot conducted in 2017. The CFPB has also learned from the National Mortgage Database (NMDB) team, comprised of staff from the CFPB, Freddie Mac, and the Federal Housing Finance Agency (FHFA), which administers the National Survey of Mortgage Originations (NSMO) and the American Survey of Mortgage Borrowers (ASMB) using a similar sampling methodology.
When the CFPB successfully piloted the survey in 2017, we tested whether a $10 incentive would increase response rates to the survey enough to justify the additional cost.1 The CFPB found that the $10 incentive did not significantly increase response rates enough to justify the additional cost. Therefore, we plan to use a $5 incentive for the pre-completion incentive for our survey.
Meta-analyses of mail surveys find that incentives given initially with the questionnaire yield significantly higher response rates than do incentives contingent on return of the survey alone, or no incentives at all. Furthermore, monetary incentives produce a stronger effect than non-monetary incentives. Additional post-completion incentives are also a recommended approach to increasing response rates without increasing costs. Many recurring federally funded surveys use monetary incentives, including the Survey of Consumer Finances, the Survey of Income and Program Participation, and the National Survey of Drug Use and Health, and self-administered surveys such as the Survey of Doctorate Recipients, the National Survey of Recent College Graduates, and the National Survey of Mortgage Borrowers. Incentives have consistently been found to improve response rates across a variety of survey topics and modes. Incentives have been found to be cost-effective in different modes and often reduce the effort required to contact and interview sample persons or the number of follow-up mailings.
More recently, we conducted additional primary research on improving survey response rates with different incentive designs. With the incentive structure outlined in item (2) above proved most cost-effective and showed a 20 percent increase in response rates over pre-completion incentives alone, with a large majority of responses coming electronically. We welcome public comment on additional ways to improve the cost-effectiveness or design of our incentives and may change our incentive design after consultation with other survey experts, review of current best practices in survey administration, and direct experimentation.
In addition, the CFPB believes that response rates can be sufficiently maximized through careful design of survey instrument and clear communication to potential respondents about the survey’s purpose, use, and confidentiality protections. Finally, because we will have the de-identified credit records of both respondents and non-respondents to the survey, we anticipate using this information to model survey nonresponse and to adjust sample weights to reduce the likelihood that the results of any analysis are not biased by correlation between nonresponse and observable credit characteristics.
The extensive information from the de-identified credit records for both respondents and non-respondents will provide a strong basis for investigating potential nonresponse bias relative to the CCP. The data will, for example, permit us to examine differential rates of nonresponse correlated with credit score, dollar amounts of various types of credit lines, and demographics. Based on this analysis, the CFPB will also construct survey weights so that the survey will be representative of American consumers in the target population for a given survey. In the previous rounds of the Making Ends Meet survey, as well as other surveys the CFPB has done using credit-record data as a sampling frame, the CFPB found that consumers who are older and have higher credit scores are slightly more likely to reply. Observable credit characteristics only weakly predict who will respond. The nonresponse bias analyses will be of benefit to other federal agencies using samples based on commercially available administrative data.
For survey respondents, the CFPB will additionally be able to compare the self-reported demographic information in the survey (for example, education, age, income, and marital status) to the auxiliary demographic information included in the credit-record database. This comparison may shed light on the reliability of such auxiliary data and, thus, provide information that may be valuable to government researchers and others that rely on such data when direct measures of these characteristics are unavailable. In the pilot Making Ends Meet survey as well as other surveys the CFPB has done using credit-record data as a sampling frame, the CFPB found that the auxiliary data are predictive of survey respondents’ self-reported demographic information, but there are significant differences for some consumers.
The CFPB plans to use pretesting and cognitive interviewing to test research instruments on a small scale prior to their use in full-scale surveys. This can include a debrief with respondents after the pilot to ensure that they understood the instrument, which in turn ensures that the resulting data collections are effective. These techniques are meant to reduce the total public burden of the information collection by ensuring that the full study information collection is optimized. The CFPB also expects to study potential improvements in mode of contact, including potential outreach to respondents via email. This research would involve creating specific outreach designs (for example, first-contact via mail and follow-up via email) for different groups of potential respondents, then randomly assigning respondents to each group to determine how the different designs affect response rates and survey costs.
Scott Fulford: (202)-570-1200
Eric Wilson: (202)-570-5194
1 This research has been published in: Bucks, Brian, Mick Couper, and Scott Fulford, 2020, “A Mixed-Mode and Incentive Experiment Using Administrative Data,” Journal of Survey Statistics and Methodology, 8(2): 352-369.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Galleher, Michael (Contractor)(CFPB) |
| File Modified | 0000-00-00 |
| File Created | 2024-07-20 |