SSA - Generic for ACF Prog Support - SPARK

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Supporting Partnerships to Advance Research and Knowledge

Formative Data Collections for Program Support

0970 - 0531

Supporting Statement

Part A

December 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer: Victoria Kabak

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for Program Support (0970-0531).

• Description of Request: The purpose of Supporting Partnerships to Advance Research and Knowledge (Project SPARK) is to equip state and local TANF programs with tools and skills to be better users and producers of evidence. Mathematica is the contractor for Project SPARK. Through this work, state and local programs should be better positioned to implement, replicate, and scale interventions that promote employment and self-sufficiency for low-income families.

In March 2020, state and local TANF programs experienced an unprecedented disruption in service delivery as a result of the COVID-19 pandemic. The crisis spurred three primary changes to their work—a shift from in-person to remote service delivery, an increase in demand for public services, and the need for staff to manage work and family responsibilities while working from home. This information collection request is to administer short surveys with participating TANF program staff to understand the current response of state and local TANF programs to COVID-19 and shed light on the successes and challenges they are experiencing. Data collected through these surveys will be used to support programs as they identify, develop, and then implement innovative strategies in response to the COVID-19 pandemic; data collected will not be generalized to a broader population. We do not intend for this information to be used as the principal basis for public policy decisions.

• Time Sensitivity: It is important to conduct the surveys as soon as possible so that (1) they capture in real time issues TANF programs are facing during the pandemic, and (2) the resultant data are accessible while programs are still wrestling with the effects of the pandemic. The information gathered from these surveys will support programs with designing strategies that others have recommended or implemented successfully, and will ensure that ACF can provide timely and improved support to programs.

A1. Necessity for Collection

In March 2020, state and local TANF programs experienced an unprecedented disruption in service delivery as a result of the COVID-19 pandemic. The crisis spurred three primary changes to their work—a shift from in-person to remote service delivery, an increase in demand for public services, and the need for staff to manage work and family responsibilities while working from home. Through Project SPARK, (under contract with Mathematica) ACF has been providing technical assistance (TA) to programs as they wrestle with these and related challenges. Short surveys of participating programs are necessary to assess the value of the assistance provided to date and tailor future assistance to better meet program needs. They are also important to assess the ongoing challenges programs face and to generate and disseminate ideas on promising practices and potential solutions.

There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):

• Delivery of targeted assistance and workflows related to program implementation or the development or refinement of program and grantee processes, and the development and refinement of recordkeeping and communication systems.

• Planning for provision of programmatic or evaluation-related training or technical assistance (T/TA).

The purposes of the information collection are twofold: (1) to inform ACF about ways that state and local TANF programs are responding to COVID-19 and the successes and challenges they are experiencing as programs innovate, and (2) to collect feedback about how programs perceive the value of TA provided under Project SPARK. ACF will use the resultant information to inform its TA to TANF programs around continuing to provide safe and quality services to participants during the COVID-19 pandemic, and TANF programs will use the information to learn about potential strategies they can employ to maintain program operations and support participants and staff.

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Guiding Questions

Three questions guide the information collection:

1. What are the circumstances of programs and the families they serve during the COVID-19 crisis, and how have they evolved over the course of the crisis?

2. How have programs responded to COVID-19? What strategies have programs developed, and what strategies show the most promise?

3. How has the TA helped programs respond to COVID-19? What methods and topics have been most and least helpful? What are the lessons for providing remote TA that might carry over into a post-COVID-19 service environment?

Study Design

This information collection will use the Learn, Innovate, Improve (LI²) framework, developed with OPRE under an earlier ACF contract (Advancing Welfare and Family Self-Sufficiency Research Project), which is currently being used to provide TA under Project SPARK. The LI² framework is a series of replicable, evidence-informed program improvement activities supported by collaboration between practitioners and applied researchers. The purpose of the Learn phase is to gather information to understand a program’s motivation for change, such as whether there is a specific problem that needs to be solved, or whether a program is seeking to try something new. In the Improve phase, researchers work collaboratively with programs to develop solutions based on available evidence. During guided, systematic planning sessions, researchers work with program staff to think intentionally about what outcomes they seek and what activities will lead to these outcomes. After a program change is developed, the goal of the Improve phase is to improve its implementation through an iterative test-and-refine process. Together, these phases are designed to create innovations that are practical, effective, scalable, and sustainable (Derr et al. 2017).

This project uses the LI² framework to guide delivery of evaluation TA because it is designed to help practitioners use high-quality research methods in accessible ways for the purpose of improving their programs. Practitioners are rapidly adapting and innovating in response to the changes brought on by the COVID-19 pandemic; LI² offers a systematic and nimble approach to innovation, informed by the best available research and with an eye toward generating evidence in the process.

The data collection procedures for study activities are included in Supporting Statement B, section B4. This is a one-time data collection from a nonrepresentative, nongeneralizable population. The surveys are designed to be short and low-burden, and provide basic, real-time information that can be used to inform ACF about the ways in which state and local TANF programs are responding and adapting to COVID-19. Detailed qualitative information pertaining to the research questions will not be gathered through this data collection.

Data Collection Activity	Instrument(s)	Respondent, Content, Purpose of Collection	Mode and Duration
Staff survey	Instrument 1	Respondents: TANF program staff Content: Steps that TANF programs have taken to ensure a safe working environment and develop a safety plan for operations during the COVID-19 crisis. Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes
Staff survey	Instrument 2	Respondents: TANF program staff Content: How TANF programs are approaching serving clients during COVID-19 (in person or remote) and flexibility of TANF programs in allowing staff to work remotely. Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes
Staff survey	Instrument 3	Respondents: TANF program staff Content: General well-being of program staff during COVID-19 crisis Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes
Staff survey	Instrument 4	Respondents: TANF program staff Content: Remote activities that TANF program staff have participated in together Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes
Staff survey	Instrument 5	Respondents: TANF program staff Content: Types of participant activities that TANF programs have offered remotely and in-person during the COVID-19 crisis; challenges and successes with service delivery during the COVID-19 crisis. Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes
Staff survey	Instrument 6	Respondents: TANF program staff Content: TANF program staff and participants’ access to and use of technology Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes
Staff survey	Instrument 7	Respondents: TANF program staff Content: Adjustments that TANF programs have made to program requirements and enforcement of program requirements during the COVID-19 crisis Purpose: To increase the government’s knowledge of how state and local TANF programs are responding to the COVID-19 crisis and help improve the government’s support of and TA to TANF programs.	Mode: Web survey Duration: 10 minutes

Other Data Sources and Uses of Information

During the study period, we will support opportunities for TANF program staff from different localities to talk informally and share lessons with each other related to how they and their programs are navigating the COVID-19 crisis. They will not be facilitated, but a project team member will attend to provide logistical support and take notes to capture a record of their experiences. No burden is associated with this activity because it is a direct observation.

The study team will document information learned about the experiences and stories of programs and families through TA calls with TANF programs requesting additional support related to COVID-19. Notes from these calls will support TA and will supplement survey data.

A3. Use of Information Technology to Reduce Burden

The web-based staff surveys (Instruments 1-7) will be administered using SurveyMonkey. SurveyMonkey is an intuitive platform that is easy for respondents to navigate. It is also optimized for mobile use, so respondents will have the option to use a smartphone or tablet to complete the survey.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The COVID-19 pandemic is unprecedented and ongoing, which means that information about state and local TANF program operations and needs is constantly changing and not readily available elsewhere. The study team is collecting information that is tailored to the individual sites receiving TA under Project SPARK. None of the instruments ask for information that can be reliably obtained through other sources.

A5. Impact on Small Businesses

The information collection request involves state and local TANF program offices and the offices of their contracted service providers. Participation in the data collection is voluntary. The surveys will be provided on-demand and will be open for windows of time, so that respondents can complete it when it is most convenient for them.

A6. Consequences of Less Frequent Collection

This is a one-time data collection. Each survey will be administered once.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. A subsequent notice, updated with more specific information, was published on June 18, 2019, Volume 84, Number 117, page 28307, and provided a thirty-day period for public comment. During the notice and comment periods, no substantive comments were received.

Consultation with Experts Outside of the Study

The project team did not consult with experts outside of the study for this information collection.

A9. Tokens of Appreciation

There are no tokens of appreciation associated with this information collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

This information collection request includes the collection of minimal personally identifiable information (PII). The project team has access to staff names and email addresses through ongoing TA. The collection of email addresses is necessary to administer the web-based staff surveys (Instruments 1-7).

Each survey asks for general background information (state, organization type, job role, and urbanicity of the region that the TANF program serves) to facilitate analysis. It does not ask respondents to identify themselves by name. Staff names will not be connected with survey responses.

Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ name and email address. This study does not use an information system that uses personal identifiers to retrieve data.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed that their participation is voluntary, and that their information will be kept private to the extent permitted by law prior to the start of any data collection, and the purpose of the data collection. As specified in the contract, Mathematica will comply with all Federal and Departmental regulations for private information. Assurances will be made on the first screen of the surveys before any questions are given.

Data Security and Monitoring

This project will comply with Mathematica’s data security policies. Only staff from Mathematica will handle data collected under this clearance. All Mathematica staff involved in the project receive annual training on (1) limitations of disclosure; (2) safeguarding the physical work environment; and (3) storing, transmitting, and destroying data securely. All Mathematica staff sign the Mathematica Confidentiality Agreement, complete online security awareness training when they are hired, and receive annual refresher training thereafter. Training addresses security policies and procedures found in the Mathematica Corporate Security Manual.

Survey responses will be downloaded to an encrypted project folder on Mathematica’s server each week. Mathematica uses access control lists to restrict access to the encrypted project folders where sensitive and confidential project data are stored. Access to the project folder is explicitly authorized by the Project Director on need-to-know and least privilege bases. Mathematica staff are required to change their password for computer and network access every thirty days, and passwords must adhere to strict composition standards. Staff access rights to the project folder are revoked when they leave the project. If a staff member leaves Mathematica, his or her access to computing assets, including network access, is terminated.

A11. Sensitive Information ¹

The information collection will not ask about sensitive information. Institutional Review Board (IRB) approval is not needed for this collection; the relevant IRB determination is attached (Appendix H).

A12. Burden

Explanation of Burden Estimates

The current request includes burden estimates to cover the following activities:

Instruments 1-7: Web surveys

• 310 staff members answer a 10-minute web survey once a week, over seven weeks, for a total annualized burden of about 364 hours.

Estimated Annualized Cost to Respondents

We expect the total annual cost for respondents to be $14,371 for the information collection in the current request. For each instrument included in the burden table, we calculated the total annual cost by multiplying the total burden hours by the average hourly wage.

For the planned web surveys (composed of Instruments 1-7), we estimate the annualized cost to respondents based on the average hourly wage estimates for deriving total annual costs based on Current Population Survey data for the fourth quarter of 2019 (Bureau of Labor Statistics 2019). For respondents, we used the median usual weekly earnings for full-time wage and salary workers ages 25 and older with a bachelor’s degree ($39.49 per hour). We divided weekly earnings by 35 hours (how the Current Population Survey defines full-time) to calculate hourly wages.

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total/annual Burden (in hours)	Average Hourly Wage Rate		Total Annual Respondent Cost
Instrument 1 – Safety	310	1	.168 hours (10 minutes)	52	$39.49		$2,053
Instrument 2 – Approaches to serving customers	310	1	.168 hours (10 minutes)	52	$39.49		$2,053
Instrument 3 – General wellbeing	310	1	.168 hours (10 minutes)	52	$39.49		$2,053
Instrument 4 – Engaging with customers remotely	310	1	.168 hours (10 minutes)	52	$39.49		$2,053
Instrument 5 – Remote services for customers	310	1	.168 hours (10 minutes)	52	$39.49		$2,053
Instrument 6 – Access to and use of technology	310	1	.168 hours (10 minutes)	52	$39.49		$2,053
Instrument 7 – Performance and accountability	310	1	.168 hours (10 minutes)	52	$39.49	$2,053
Totals:	310	7	1.18 hours (70 minutes)	364		$14,371

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The estimated costs in the table below were calculated using the hourly rate of Mathematica staff involved in instrument development, field work, and publications for Project SPARK and a level of effort estimated from similar tasks on other Mathematica research contracts.

Cost Category	Estimated Costs
Instrument Development and OMB Clearance	$5,236
Field Work	$6,481
Publications/Dissemination	$2,838
Total/annual costs over the request period	$14,555

A15. Reasons for changes in burden

This is for an individual information collection under the umbrella formative generic clearance for program support (0970-0531).

A16. Timeline

Activity	Anticipated timeline
Data collection	Over the course of the two months following OMB approval
Internal dissemination to OPRE (regular updates)	Over the course of the three months following OMB approval

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Appendix A: Instrument 1 – Safety

Appendix B: Instrument 2 – Approaches to serving customers

Appendix C: Instrument 3 – General wellbeing

Appendix D: Instrument 4 – Engaging with coworkers remotely

Appendix E: Instrument 5 – Remote services for customers

Appendix F: Instrument 6 – Access to and use of technology

Appendix G: Instrument 7 – Performance and accountability

Appendix H: IRB Determination

Appendix I: Proposed cover email to respondents

References

Bureau of Labor Statistics. “Labor Force Statistics from the Current Population Survey.” Washington, DC: Bureau of Labor Statistics. Available at https://www.bls.gov/cps/earnings.htm. Accessed February 20, 2020.

Derr, Michelle, Ann Person, and Jonathan McCay. “Learn, Innovate, Improve (LI²): Enhancing Programs and Improving Lives.” OPRE report 2017-108. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation, December 2017. Available at https://www.acf.hhs.gov/opre/resource/learn-innovate-improve-li2-enhancing-programs-and-improving-lives.

1 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

10