Supporting Statement Part B-6.9.21-clean

Supporting Statement for the State Arts Agency CARES Act Funding Survey, Part B
Last Updated: May 27, 2021

1

Table of Contents

B.1 Respondent universe and sampling methods ........................................................................... 1
B.2 Procedures for the collection of information............................................................................ 3
B.3 Methods to maximize the response rates and to deal with nonresponse .................................. 3
B.4 Test of procedures or methods to be undertaken...................................................................... 5
B.5 Individuals consulted on statistical aspects & individuals collecting and/or analyzing data ... 6

Table of Attachments
Attachment A: Communication Materials
Attachment B: Web Survey Instrument
Attachment C: Cognitive Testing Report

2

B.1 Respondent universe and sampling methods
Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of entities
(e.g., establishments, State and local government units, households, or persons) in the
universe covered by the collection and in the corresponding sample are to be provided in
tabular form for the universe as a whole and for each of the strata in the proposed sample.
Indicate expected response rates for the collection as a whole. If the collection had been
conducted previously, include the actual response rate achieved during the last collection.
The potential respondent universe for the CARES Act Funding Survey are 62 staff
members at state arts agencies and regional arts organizations. The survey is one-time census;
thus, the “sample” is the same as the respondent universe. The National Assembly of State Arts
Agencies (NASAA) receives a 100 percent response rate on most surveys to state arts agencies.
It is expected that NASAA will receive a similar response rate, between 90-100 percent, for this
one-time survey.
B.2 Procedures for the collection of information
Describe the procedures for the collection of information, including statistical methodology
for stratification and sample selection, estimation procedure, degree of accuracy needed for
the purpose described in the justification, unusual problems requiring specialized sampling
procedures, and any use of periodic (less frequent than annual) data collection cycles to
reduce burden.
The information collection for CARES Act Funding Survey will consist of one web
survey census of all state arts agencies and regional arts organizations. No special sampling,
stratification, or estimation procedures are anticipated.
B.3 Methods to maximize the response rates and to deal with nonresponse
Describe methods to maximize response rates and to deal with issues of non-response. The
accuracy and reliability of information collected must be shown to be adequate for
intended uses. For collections based on sampling, a special justification must be provided
for any collection that will not yield "reliable" data that can be generalized to the universe
studied.

3

NASAA will administer the web survey to state arts agencies and regional arts agencies
in summer 2021. To implement the survey, NASAA will first send a survey invitation by email
that contains a link to the survey that will allow respondents to save their answers and return to
complete later. NASAA will continue to send email reminders to any respondents with an
incomplete survey. One day before the survey closes, NASAA will send a final email reminder
reminding respondents with an incomplete survey to complete the survey by the close of data
collection. In addition, a few days after the survey deadline has passed, NASAA will call the
outstanding survey respondents to inform them of the survey and the missed due date. NASAA
expects to implement the web survey over a three-week data collection period. Figure 1 provides
an overview of the survey implementation process and expected response rates.
Figure 1: Survey Implementation Flowchart
NASAA sends
survey link

NASAA sends
1st reminder

NASAA sends
2nd reminder

NASAA calls
respondents

n = 62

n = 36

n = 23

n = 14

n = 26 (41%)

n = 13 (21%)

N = 9 (14%)

N = 9 (16%)

Total Response Rate
N = 57 (93%)
Total Nonresponse Rate
N = 5 (7%)

We provide all planned communication materials in Attachment C.
To maximize response rates, the survey invitation email will come directly from NASAA
staff. In addition, all communications with state arts agencies and regional arts organizations will
explain the importance of the survey to NASAA and the Arts Endowment. All communications
will also note that responses to the survey will be kept confidential and reported to the public in
the aggregate. Based on the cognitive testing, NASAA expects respondents to take additional
time to gather required information for the survey. To address this issue, email communications
will contain a notification that respondents may need additional steps to acquire requested data.
Respondents will be able to save and continue in case they need to gather data. This should help
4

prepare respondents for the survey and minimize underreporting due to lack of readily available
information.
NASAA will continue to conduct the data collection using approved methods until an
appropriate response rate is reached. Additional methods NASAA may use to increase the
response rate include:
1. increasing the data collection period from three up to five weeks,
2. incorporating a second round of phone calls at the end of data collection.
B.4 Test of procedures or methods to be undertaken
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an
effective means of refining collections of information to minimize burden and improve
utility. Tests must be approved if they call for answers to identical questions from 10 or
more respondents. A proposed test or set of tests may be submitted for approval separately
or in combination with the main collection of information.
NASAA employed a contractor to conduct cognitive tests with 8 state arts agency staff
members in March 2021. Eight staff members from state arts agencies completed a 30-minute
cognitive interview after testing the survey. NASAA also sent the survey to one regional arts
organization for feedback. The objectives were to detect issues of usability, clarity, and
readability in the survey instrument, and to estimate the average time necessary to complete the
survey. Changes were made to the survey instrument following the completion of cognitive
testing and regional arts organization feedback, per the recommendations in the Cognitive
Testing Report. The Cognitive Testing Report can be found in Attachment C.
The cognitive testing indicated that participants found the survey platform easy to use,
and no technical issues arose when accessing or completing the survey. There was a
recommendation to alert future participants to the information that will be requested in advance
through an option to preview the survey online or by circulating the items in advance.
5

Interviewees also recommended being able to stop and restart the survey and to return to
previous questions. NASAA has made both of these recommended changes.
Aggregated analyses of the Interviewee Recording Forms, survey data, and interviews
identified factors that adversely affected participants’ responses across multiple items and
challenges related to the interpretation of individual items. For instance, the CARES Act funding
may be confused with other types of CARES Act supplemental funding some state arts agencies
are receiving. NASAA addressed this issue by making it clear this survey is assessing the
outcomes of CARES Act funding coming directly from the Arts Endowment, not elsewhere.
Some states and regions may not be collecting the voluntary subgrantee data and NASAA
incorporated response options to allow for states and regions to respond accurately that they do
not have the data. States and regions may have the data in formats that would be too onerous to
aggregate for the purposes of the survey. NASAA included an option to upload different types of
data in case their data does not match with the structure of the questions.

B.5 Individuals consulted on statistical aspects & individuals collecting and/or analyzing
data
Provide the name and telephone number of individuals consulted on statistical aspects of
the design and the name of the agency unit, contractor(s), grantee(s), or other person(s)
who will actually collect and/or analyze the information for the agency.
Table 1. Individuals Consulted
Name

Title (Project Role) Organizational Affiliation and
Address
Persons conducting the survey
Ryan Stubbs
Senior Director of
National Assembly of State Arts
Research
Agencies
1200 18th Street NW, Suite 1100
Washington, D.C. 20036
Patricia
Research Manager
National Assembly of State Arts
Mullaney-Loss
Agencies
1200 18th Street NW, Suite 1100

Phone
Number
202-347-6352
x115
202-347-6352
x118
6

Washington, D.C. 20036

Arts Endowment personnel consulted
Patricia Moore Deputy Director |
National Endowment for the Arts
Shaffer
Office of Research
400 7th Street SW, Washington, DC
& Analysis
20506
Melissa
Program Analyst |
National Endowment for the Arts
Menzer
Office of Research
400 7th Street SW, Washington, DC
& Analysis
20506
Persons conducting the cognitive testing
Shawn
Consultant, founder ProgramWorks LLC
Bachtler
of ProgramWorks
Seattle, WA

202-682-5535
202-682-5548

206-323-9329

7