CPS Public Participation in the Arts pretesting request letter - revised 1-4-21

Cognitive Testing of Public Participation in the Arts Modules for the 2022 Current Population Survey Submitted Under Generic Clearance for Questionnaire Pretesting

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We plan to conduct iterative pretesting of the Public Participation in the Arts (PPA) Supplement to the 2022 Current Population Survey (CPS). This supplement contains questions about arts-related activities and developed by the sponsor, the National Endowment for the Arts. As part of this current submission we are seeking approval for cognitive interviews conducted by survey methodologists from the Center for Behavioral Science Methods at the Census Bureau.

Purpose: The primary purposes of this research are:

1. To evaluate the PPA questions in terms of respondents’ cognitive processes, especially their interpretations of the questions, and their ability to retrieve memories about in-scope activities relative to desired reference periods; and

2. To evaluate the quality of proxy responses by other household members.

We plan to pretest all items in the PPA instrument. The PPA questionnaire is comprised of six modules:

Cores 1 & 2: Participation in Arts and Leisure Activities

Module A: Consuming Art via Electronic Media

Module B: Arts Creation and Performance

Module C: Leisure Activities

Module D: Arts Education

When fielded as part of the CPS, the six modules will be combined in eight form variants, each consisting of three modules, which will be randomly assigned to CPS respondents. Due to the combined length of the modules, it will not be feasible to administer all six to individual participants. Instead we will test them in four combinations:

Version 1: Core 1, module A, module B, module C

Version 2: Core 2, module A, module B, module C

Version 3: Core 1, module A, module B, module D

Version 4: Core 2, module B, module C, module D

Each participant will be administered one version of test form. Module D contains questions about children, so versions 3 and 4 will be randomly assigned to participants sharing households with children. Versions 1 and 2 will be assigned randomly and evenly among the remainder of our pretesting sample. The full text of the questions to be tested can be found in Enclosures 1 and 2.

To evaluate the accuracy of proxy responses for eligible household members for all items in the survey questionnaire, we will recruit pairs of participants from the same household and interview them independently. Each participant will answer part of the questionnaire about themselves and the rest about the other household member. In our analysis, we will compare the responses within each pair, so we can assess the degree of consistency in their answers to questions targeting the other household member. We will not question these participants about any discrepancies between responses. The paired-interview strategy is intended to provide the sponsor with some understanding as to the level of accuracy they might expect in responses provided by proxies.

Testing will be conducted in two rounds. Possible wording changes to address problems identified in the first round will be tested in the second round. The results of the cognitive testing will inform the final revised PPA instrument content that will be used in data collection in 2022.

Population of Interest: The planned cognitive pretesting evaluation will focus on assessing and improving the questionnaire content for the general population.

Timeline: Testing will be conducted from January through May 2021. Recruiting will start in January 2021.

Language: Testing will be conducted in English only.

Method: We plan to evaluate the questions in cognitive interviews over telephone.

Sample: We will recruit a sample with a maximum of 36 participants. Our recruiting efforts will target pairs of participants 18 years old or older who share a household (related or unrelated), in which at least one has any of the following characteristics:

• Participants who have attended a live musical performance, stage production, or any other live artistic performance in the last two years

• Participants who have engaged in creative activities of any kind, whether professionally or for one’s own pleasure, such as making art or crafts, playing music, dancing, acting, etc., in the last two years

• Participants who share a household with one or more children under the age of 18

In addition to these characteristics, we will attempt to recruit participants from diverse demographic groups including race/ethnicity, age, gender, and household size to the maximum extent possible.

Recruitment: Participants will be recruited through internal Census Bureau email announcements, flyers, Craigslist.com, and via personal networks. Participants will be screened and selected based on the survey-relevant characteristics of interest listed in the preceding section. Additional questions to be added to the universal recruiting screener and recruitment advertising text are attached (see Enclosures 3 and 4). Participants will be screened by telephone and may receive an email reminder about the cognitive interview appointment after they agree to participate.

Protocol: CBSM researchers will administer the draft questionnaire and participants will be asked to think aloud as they respond. We will administer retrospective probes at the end of modules or sequences of questions within modules, to examine participants’ interpretations of constructs and terminology used in questions, and their strategies for identifying activities that occurred within the survey reference periods. We will also ask proxy participants how they answered questions about the other member of their household. The combined questionnaire/protocols are included as attachments (Enclosures 1 and 2).

Consent: Participants will be sent a secure link to an online consent form prior to their interviews. We will inform participants that their participation in the study is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. The consent form will also indicate that the participant agrees that the interview can be audio-taped to facilitate analysis of the results (see Enclosure 5). Participants who do not consent to be audio-taped will still be allowed to participate.

Use of Incentive: Each participant will be offered $40 for participation in a seventy-five-minute interview. This incentive will be offered to help ensure that we are able to recruit participants from all desired populations and to offset the costs of participation in this research, such as childcare and phone use. Incentives will be mailed upon completion of the interviews to participants’ preferred address using USPS Priority Mail. Federal employees who participate in this research are not eligible to receive the cash incentive.

Length of interview: We estimate the interviews will last up to 75 minutes per participant on average. Participants will be screened using the Census Bureau’s generic screener and the additional screener questions in Enclosure 3. (The burden hours for the generic screener are covered under a separate request.) We estimate it will be necessary to screen 3 people to obtain a one eligible participant. The total estimated participant burden for this request is 48.6 hours.

Category	No. of Participants	Participation Time	Burden
Screening	108	2 minutes (0.03 hours)	3.6 hours
Cognitive Interviews	36	75 minutes (1.25 hours)	45 hours
Totals			48.6 hours

The materials to be used in this study are enclosed with this letter and include:

Enclosures 1 and 2 – PPA Questionnaire and Cognitive Interview Protocols for Target Respondents and Proxy Respondents

Enclosure 3 – Recruitment Advertisement Text

Enclosure 4 – Recruitment Screener

Enclosure 5 – Consent Form

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Dave Tuttle

Center for Behavioral Science Methods

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-7809

[email protected]

3