Supporting Statement Part B_Regional and State Arts Agency ARP Funding Survey_v2FINAL

Supporting Statement for the Regional and State Arts Agency ARP Funding Survey, Part B
Last Updated: August 22, 2022

1

Table of Contents

B.1 Respondent universe and sampling methods .............................................................. 3
B.2 Procedures for the collection of information ................................................................. 3
B.3 Methods to maximize the response rates and to deal with nonresponse ..................... 4
B.4 Test of procedures or methods to be undertaken ........................................................ 5
B.5 Individuals consulted on statistical aspects & individuals collecting and/or analyzing
data ............................................................................................................................. 6

Table of Attachments
Attachment A: Communication Materials
Attachment B: Web Survey Instrument
Attachment C: Cognitive Testing Report

2

B.1 Respondent universe and sampling methods
Describe (including a numerical estimate) the potential respondent universe and
any sampling or other respondent selection method to be used. Data on the number
of entities (e.g., establishments, State and local government units, households, or
persons) in the universe covered by the collection and in the corresponding
sample are to be provided in tabular form for the universe as a whole and for each
of the strata in the proposed sample. Indicate expected response rates for the
collection as a whole. If the collection had been conducted previously, include the
actual
response
rate
achieved
during
the
last
collection.
The potential respondent universe for the Regional and State Arts Agency
American Rescue Plan (ARP) Funding Survey is 62 staff members at state arts
agencies and regional arts organizations. The survey is one-time census; thus, the
“sample” is the same as the respondent universe. The National Assembly of State Arts
Agencies (NASAA) receives a 100% response rate on most surveys to state arts
agencies. It is expected that NASAA will receive a similar response rate, between 90
and 100%, for this one-time survey.
B.2 Procedures for the collection of information
Describe the procedures for the collection of information, including statistical
methodology for stratification and sample selection, estimation procedure, degree
of accuracy needed for the purpose described in the justification, unusual
problems requiring specialized sampling procedures, and any use of periodic (less
frequent than annual) data collection cycles to reduce burden.
The information collection for the Regional and State Arts Agency American
Rescue Plan (ARP) Funding Survey will consist of one online survey census of all state
arts agencies and regional arts organizations. No special sampling, stratification, or
estimation procedures are anticipated.

3

B.3 Methods to maximize the response rates and to deal with nonresponse
Describe methods to maximize response rates and to deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to
be adequate for intended uses. For collections based on sampling, a special
justification must be provided for any collection that will not yield "reliable" data
that can be generalized to the universe studied.
NASAA will administer the online survey to state arts agencies and regional arts
agencies in winter 2023. To implement the survey, NASAA will first send a survey
invitation by email that contains a link to the survey that will allow respondents to save
their answers and complete it at their convenience. NASAA will send up to three email
reminders to any respondents with an incomplete survey. A few days after the survey
deadline has passed, NASAA will call the outstanding survey respondents to inform
them of the survey and the missed due date. NASAA expects to implement the online
survey over a six-week data collection period. Figure 1 provides an overview of the
survey implementation process and expected response rates. All planned
communication materials are provided in Attachment A.
To maximize response rates, the survey invitation email will come directly from
NASAA staff. In addition, all communications with state arts agencies and regional arts
organizations will explain the importance of the survey to NASAA and the NEA. All
communications will also note that responses to the survey will be kept confidential and
reported to the public in the aggregate. Based on the cognitive testing, NASAA expects
respondents to take additional time to gather required information for the survey. To
address this issue, email communications will contain a notification that respondents
may need additional steps to acquire requested data. Respondents will be able to save
and continue in case they need to gather data. This should help prepare respondents
4

for the survey and minimize underreporting due to lack of readily available information.
NASAA will continue to conduct the data collection using approved methods until an
appropriate response rate is reached. NASAA may also email or call agencies and
organizations to help increase the response rate.
B.4 Test of procedures or methods to be undertaken
Describe any tests of procedures or methods to be undertaken. Testing is
encouraged as an effective means of refining collections of information to minimize
burden and improve utility. Tests must be approved if they call for answers to
identical questions from 10 or more respondents. A proposed test or set of tests
may be submitted for approval separately or in combination with the main
collection of information.
NASAA employed a contractor to conduct cognitive tests with 8 state arts agency
staff members in March 2021 for the CARES Act Funding Survey, which the Regional
and State ARP Funding Survey is based on. The two surveys are identical with the
exception that programmatic information, like grant program name and description, is
updated in the Regional and State ARP Funding Survey. Eight staff members from state
arts agencies completed a 30-minute cognitive interview after testing the CARES Act
survey. NASAA also sent the survey to one regional arts organization for feedback. The
objectives were to detect issues of usability, clarity, and readability in the survey
instrument, and to estimate the average time necessary to complete the survey.
Following the completion of cognitive testing and regional arts organization feedback,
changes were made to the survey instrument for CARES, and then later adopted for the
ARP survey. The Cognitive Testing Report for the CARES Act survey can be found in
Attachment C.
The cognitive testing indicated that participants found the survey platform easy to
use, and no technical issues arose when accessing or completing the survey. There
5

was a recommendation to alert future participants to the information that will be
requested in advance through an option to preview the survey online or by circulating
the items in advance.
Interviewees also recommended being able to stop and restart the survey and to
return to previous questions. NASAA has made both of these recommended changes.
Aggregated analyses of the Interviewee Recording Forms, survey data, and
interviews identified factors that adversely affected participants’ responses across
multiple items and challenges related to the interpretation of individual items. For
instance, the ARP funding may be confused with other types of ARP supplemental
funding some state arts agencies are receiving. NASAA addressed this issue by making
it clear this survey is assessing the outcomes of ARP funding coming directly from the
NEA, not elsewhere.
Some states and regions may not be collecting the voluntary subgrantee data and
NASAA incorporated response options to allow for states and regions to respond
accurately that they do not have the data. States and regions may have the data in
formats that would be too onerous to aggregate for the purposes of the survey. NASAA
included an option to upload different types of data in case their data does not match
with the structure of the questions.
B.5
Individuals consulted on statistical aspects & individuals collecting
and/or analyzing data
Provide the name and telephone number of individuals consulted on statistical
aspects of the design and the name of the agency unit, contractor(s), grantee(s), or
other person(s) who will actually collect and/or analyze the information for the
agency.

6

Table 1. Individuals Consulted
Name

Title (Project Role)

Persons conducting the survey
Senior Director of
Ryan Stubbs
Research

Mohja Rhoads

Research Manager

Organizational Affiliation and
Address

Phone
Number

National Assembly of State Arts
Agencies
1200 18th Street NW, Suite 1100
Washington, D.C. 20036

202-552-0843

National Assembly of State Arts
Agencies
1200 18th Street NW, Suite 1100
Washington, D.C. 20036

202-552-0846

NEA personnel consulted
Patricia Moore Deputy Director |
202-682-5535
National Endowment for the Arts
Shaffer
Office of Research
400 7th Street SW, Washington, DC
& Analysis
20506
Melissa
202-682-5548
Program Analyst |
National Endowment for the Arts
Menzer
Office of Research
400 7th Street SW, Washington, DC
& Analysis
20506
Liz
202-682-5743
ARP Grants
National Endowment for the Arts
Deichm
Management
400 7th Street SW, Washington, DC
ann
Analyst
20506
Persons conducting the original cognitive testing for the CARES Act survey
Shawn
Consultant, founder ProgramWorks, LLC
206-323-9329
Bachtler
of ProgramWorks
Seattle, WA

7