0985-New APS Tarc Practice Survey Supporting Statement Part B

PART B: COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
0985-NEW APS TARC PRACTICE SURVEY
B.1 RESPONDENT UNIVERSE AND SAMPLING METHODS
There’s no sampling methods applied to this data collection program. Variation among states
and the need to provide state-level data make sampling an inappropriate approach. The plan is
to use a census approach for complete enumeration.
B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION
Not applicable.
B.3 METHODS TO MAXIMIZE THE RESPONSE RATES AND TO DEAL WITH NONRESPONSE
Data collection will occur in several stages: (1) an advance email notification sent by ACL or the
APS TARC; (2) a second email to provide directions for completing the survey and a link to
access the online survey; and (3) follow-up email reminders to non-responders to request that
they complete the survey. One week after the advance email notification, the APS TARC team
will send the second email to State APS program managers requesting that they complete the
survey. This second email will contain directions for logging on to the website, as well as the
website URL, instructions for completing the survey; and a reminder to contact the APS TARC
team member if they need assistance completing the survey. APS TARC team members will be
available to answer respondents’ questions about survey questions and other data
requirements or to help resolve technical problems with the online survey throughout the data
collection period. The APS TARC team will monitor survey completion throughout the data
collection period using the survey software capabilities. Team members will maintain contact
with the State contact person to ensure the submission of all information. The APS TARC team
will review the survey response rates on a weekly basis.

1

OMB Number 0985-New APS TARC Practice Survey
Typically, multiple follow-up email reminders to respondents to encourage them to complete
surveys are needed to achieve an adequate response rate.1 To promote a high response rate,
the APS TARC team will send multiple follow-up email reminders to non-respondents. The first
email reminder will be sent one week after the email request to complete the survey.

Additional email reminders will be sent to all non-respondents a week later. A decision to send
additional email reminders or to make follow-up phone calls will be made after weekly reviews
of response rates and feedback from team members about their interactions with State APS
staff and their perceptions about the potential effectiveness of additional email contacts. Team
members will make follow-up phone calls to State and local staff to clarify survey responses, as
needed.

The APS TARC team will consult the ACL Contracting Officer Representative (COR) about
requesting a follow-up letter from ACL that encourages participation in the survey among those
who have not completed it after 30 days. To monitor survey completion throughout the data
collection, the APS TARC team will develop a Survey Tracker spreadsheet for recording survey
completion, dates of reminders sent, and follow-up telephone calls. The APS TARC team will
send regular updates with response rates to the ACL COR identifying progress.
B.4 TEST OF PROCEDURES OR METHODS TO BE UNDERTAKEN
The APS TARC Evaluation team conducted a pilot of the survey instrument between July 13,
2017 and July 26, 2017. The Pilot Test Group members completed the survey on a pdf copy
and/or online, and then provided feedback about the survey instrument by completing a
feedback form (see Appendix B). The APS TARC Evaluation team sent an email to respondents
with instructions for completing the pilot survey and returning the feedback form.

1

Dillman, D., Smyth, JD and LM Christian (2014). Internet, Mail and Mixed-Method Surveys: The Tailored Design
Method (4th Edition). New York: John Wiley and Sons, Inc.

2

OMB Number 0985-New APS TARC Practice Survey
The Pilot Test Group members received this email on July 13, 2017, followed by emails to
remind them to complete the survey and to respond to requests for additional time to
complete the survey and feedback form. Eight respondents completed online surveys by July
26, 2017. One additional respondent completed the survey in September 2017. In total, nine
respondents completed the pilot survey.
The APS TARC Evaluation team used the feedback provided by the pilot respondents to modify
the survey instrument and to correct technical problems with the survey software.
The APS TARC Evaluation team deleted or modified some questions on the survey to reduce the
time burden on respondents. The average response time for the survey was 3 hours for the
nine State officials testing the online version. No questions were added or deleted as a result of
the 60-day Federal Register Notice, so no changes were made the burden estimate based on
the 60-day notice.
B.5 INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS & INDIVIDUALS COLLECTION AND/OR
ANALYZING DATA
No sampling methods will be applied to this data collection program, so there will be no
consultation on statistical aspects. The following subject-matter experts provided consultation
on the survey design and content:
Ying-Ying Yuan, Ph.D.

[email protected]

Zach Gassoumis, Ph.D.

[email protected]

The following APS TARC Evaluation team members from WRMA, Inc. will collect and analyze
information for ACL.
Karl Urban, MPaff

[email protected]

Gila Shusterman, Ph.D.

[email protected]

Andrew Capehart

[email protected]

The person responsible for receiving and approving the contract deliverables for ACL is
Contracting Officer Representative, Aiesha Gurley.

3