Attachment 1 - TANF Data Innovation (TDI) Needs Assessment Survey 2018-08-27

Temporary Assistance for Needy Families (TANF) Data Innovation (TDI) Project

Attachment 1 - TANF Data Innovation (TDI) Needs Assessment Survey 2019-02-06

Attachment 1 - TANF Data Innovation (TDI) Needs Assessment Survey 2018-08-27

OMB: 0970-0522

Document [docx]

Download: docx | pdf

TDI: Assignment Module

Start of Block: Introduction

Q1 Hello [TANF Director name],

Thank you for your willingness to participate in the TANF Data Innovation [hyperlink to website] survey about how TANF agencies use data. The goal of this survey is to collect firsthand information about your capacities, capabilities, and current use of data. Your answers will help HHS to: (1) better understand the challenges and barriers states and territories face in using data and research to inform program decision making and (2) design future technical assistance activities for TANF agencies to address these challenges. We will report a summary of aggregate findings from the project survey back to you.

This survey contains seven modules focusing on different topical areas related to TANF data usage. Each module is intended to be answered by agency staff with expertise in that topical area and will take approximately 10-15 minutes to complete. Your participation is voluntary.

Please assign the survey modules to the appropriate staff member based on the descriptions provided below.

The TANF Data Innovation project is sponsored by the Office of Planning, Research, and Evaluation in collaboration with the Office of Family Assistance in the Administration for Children and Families, U.S. Department of Health and Human Services. The project is led by MDRC, in partnership with Chapin Hall at the University of Chicago, Actionable Intelligence for Social Policy (AISP) at the University of Pennsylvania, and New York University.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0XXX and the expiration date is XX/XX/XXXX.

End of Block: Introduction

Start of Block: Module List

Q2 The next questions walk you through the staff assignment process.

Please use the Staff Assignment descriptions to identify the best person or person(s) from your agency to complete each module. The modules have been targeted so that assigned staff are positioned to effectively and efficiently complete specific modules, reducing the time burden while capturing the most accurate information.

Only one contact is requested but you may assign up to two people to complete each module. If appropriate, one staff member may be assigned multiple modules. Each assigned staff member will receive an email with a private link to complete his or her assigned survey modules.

All survey responses are private. This survey is being administered by Chapin Hall at the University of Chicago and MDRC; your individual responses will not be shared with anyone outside of the research team. Only aggregate state-level analyses will be shared with HHS. In the case of the Federal Reporting module, no state-level data will be shared with HHS. HHS will use the findings from this study for research purposes only and not for corrective action. These data will be used to develop and target technical assistance for TANF agencies. We will also report a summary of aggregate findings across all states back to you. We do not anticipate any risks or direct benefits from participating in this survey.

Page Break

Q3 TANF Data Use and Opportunities

This module seeks information about the overall context of data usage throughout your TANF agency, barriers to data use, and opportunities for technical assistance to increase data use capabilities in TANF agencies. Preview the TANF Data Use and Opportunities survey here [link to survey pdf].

Staff Assignment: This module is intended to be completed by someone in agency executive leadership (i.e. TANF Director). Enter your own information if appropriate.

Name ________________________________________________
Email ________________________________________________

Q4 Having trouble deciding who to assign to the TANF Data Use and Opportunities survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the TANF Data Use and Opportunities survey? = Check here to assign a second staff member to complete this module

Q5
Additional staff assignment for the TANF Data Use and Opportunities survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the TANF Data Use and Opportunities survey? = Check here if you are not sure who should complete this module

Q6 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the TANF Data Use and Opportunity module.

Q7 Data Collection and Documentation

This module seeks information about the TANF agency's data--what is collected, how, and how it is documented and validated. Preview the Data Collection and Documentation survey here [link to survey pdf].

Staff Assignment: This module is intended to be completed by the agency's "go-to" data person. This person could be in research, IT, or operations. Ideally, the respondent would be someone with good institutional knowledge and perhaps someone who has been in several different roles. The individual would also ideally have direct experience in data collection or in training frontline staff to do data collection.

Name ________________________________________________
Email ________________________________________________

Q8 Having trouble deciding who to assign to the Data Collection and Documentation survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the Data Collection and Documentation survey? = Check here to assign a second staff member to complete this module

Q9
Additional staff assignment for the Data Collection and Documentation survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the Data Collection and Documentation survey? = Check here if you are not sure who should complete this module

Q10 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the Data Collection and Documentation module.

Q11 Data Infrastructure

This module seeks information about the TANF agency's data infrastructure. Preview the Data Infrastructure survey here [link to survey pdf].

Staff Assignment: This module is intended to be completed by the agency's IT director or someone in a similar role. This person should be familiar with the hardware and software that underlie the agency's data access and use.

Name ________________________________________________
Email ________________________________________________

Q12 Having trouble deciding who to assign to the Data Infrastructure survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the Data Infrastructure survey? = Check here to assign a second staff member to complete this module

Q13
Additional staff assignment for the Data Infrastructure survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the Data Infrastructure survey? = Check here if you are not sure who should complete this module

Q14 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the Data Infrastructure module.

Q15 Data Sharing

This module seeks information about the status of data sharing practices in your TANF agency. Preview the Data Sharing survey here [link to survey pdf].

Staff Assignment: This module should be completed by the staff member with responsibility for managing data sharing agreements and memoranda of understanding with other state entities and with external partners. If your agency does not have anyone who handles these agreements, this module should be answered by someone in agency leadership who is familiar with data sharing requests or agreements.

Name ________________________________________________
Email ________________________________________________

Q16 Having trouble deciding who to assign to the Data Sharing survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the Data Sharing survey? = Check here to assign a second staff member to complete this module

Q17
Additional staff assignment for the Data Sharing survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the Data Sharing survey? = Check here if you are not sure who should complete this module

Q18 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the Data Sharing module.

Q19 Research and Analytic Capacity

This module seeks information about capacity to conduct research and analysis using TANF data. Preview the Research and Analytic Capacity survey here [link to survey pdf].

Staff Assignment: If your agency has a research or data analysis team, the leader of that team should complete this module. If not, this module should be completed by someone in your organization who has responsibility for reporting data or policy analysis.

Name ________________________________________________
Email ________________________________________________

Q20 Having trouble deciding who to assign to the Research and Analytic Capacity survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the Research and Analytic Capacity survey? = Check here to assign a second staff member to complete this module

Q21
Additional staff assignment for the Research and Analytic Capacity survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the Research and Analytic Capacity survey? = Check here if you are not sure who should complete this module

Q22 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the Research and Analytic Capacity module.

Q23 Federal Reporting

This module seeks information about the data you prepare to submit to HHS as part of the TANF and SSP-MOE Data Reports (ACF-199 and ACF-209). Preview the Federal Reporting survey here [link to survey pdf].

Staff Assignment: This module should be completed by the staff member with primary responsibility for those reports.

Name ________________________________________________
Email ________________________________________________

Q24 Having trouble deciding who to assign to the Federal Reporting survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the Federal Reporting survey? = Check here to assign a second staff member to complete this module

Q25
Additional staff assignment for the Federal Reporting survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the Federal Reporting survey? = Check here if you are not sure who should complete this module

Q26 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the Federal Reporting module.

Q27 Payment Integrity

This module seeks information about how your state uses data to assess payment integrity. Preview the Payment Integrity survey here [link to survey pdf].

Staff Assignment: This module should be completed by the staff member who has primary responsibility for identifying or addressing improper payments.

Name ________________________________________________
Email ________________________________________________

Q28 Having trouble deciding who to assign to the Payment Integrity survey?

Check here to assign a second staff member to complete this module
Check here if you are not sure who should complete this module

Display This Question:

If Having trouble deciding who to assign to the Payment Integrity survey? = Check here to assign a second staff member to complete this module

Q29
Additional staff assignment for the Payment Integrity survey:

Name ________________________________________________
Email ________________________________________________

Display This Question:

If Having trouble deciding who to assign to the Payment Integrity survey? = Check here if you are not sure who should complete this module

Q30 No problem! Leave this one blank for now and continue with the rest of the module assignments.

After you submit your responses, someone from the TDI team will follow up within 2 business days to help you assign the Payment Integrity module.

End of Block: Module List

Start of Block: Closing

Q31 If you are finished with this survey, click "SUBMIT" below.

Once you submit responses you will not be able to make any changes in the survey platform. Use the survey back button below to return to the assignments page to review and make any desired edits.

If you would like to update your module assignments in the future please reach out to us at [email@address] or [phone number].

End of Block: Closing

TDI: TANF Data Use and Opportunities

Start of Block: Introduction

This survey contains seven modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the TANF Data Use and Opportunities module. This module seeks information about the overall context for data usage throughout your TANF agency, barriers to data use, and seeks to identify opportunities for technical assistance to increase data use capabilities in TANF agencies. It will take approximately 10 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

Page Break

Q2 Answer to the best of your knowledge and make an educated guess if you are unsure.

We encourage you to be as candid as possible. Your responses are private. This survey is being administered by Chapin Hall at the University of Chicago and MDRC; your individual responses will not be shared with anyone outside of the research team. Only aggregate state-level analyses will be shared with HHS. We do not anticipate any risks or direct benefits from participating in this survey.

Q3 Please enter your contact information below. This information may be used to follow up with questions, clarifications, or further discussion. We will also report a summary of findings back to you. Your contact information will not be shared outside the research team.

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Q6
The TDI project aims to improve agencies' ability to use data to inform TANF program decision making. To get a complete picture of agency data use across domains and purposes, we ask this question in every module.

From your perspective, how well does your TANF agency use data to inform program decision making? Rate your response on a scale from 0 (not well at all) to 10 (extremely well).

Not well at all

Extremely well

Agency use of data

Q7 Thinking about all the data activities in your agency, please indicate how important you believe each activity is for TANF agencies.

	Very important	Moderately important	Slightly important	Not at all important
Federal reporting
Other regular reports (e.g., state mandatory reports)
Program integrity
Performance management
Quality improvement
Data visualization
Record linkage/data integration
Program evaluation
Predictive analytics

Q8 Among these same activities, please indicate how effective your agency is at performing each activity.

	Very effective	Moderately effective	Slightly effective	Not at all effective	N/A, we don't do this
Federal reporting
Other regular reports (e.g., state mandatory reports)
Program integrity
Performance management
Quality improvement
Data visualization
Record linkage/data integration
Program evaluation
Predictive analytics

Page Break

Q9
About how often does the leadership of your agency receive reports (regular or ad hoc) of aggregated administrative data about your TANF program?

Weekly
Monthly
Quarterly
Yearly
Less than once a year

Q10
About how often does the leadership of your agency currently receive specific data analyses related to TANF that combine TANF data and data from another program? For example, SNAP data or UI wage data.

Weekly
Monthly
Quarterly
Yearly
Less than once a year

Page Break

Q11 Which data points are contained in regular reports of aggregated administrative data submitted to agency leadership? (For example, reports for fiscal tracking purposes.) Check all that apply.

Applications
Case entries
Case closings
Case or family characteristics
Caseloads by family type (e.g. 2-parent, 1-parent, child only)
Time limit clocks
Work participation
Sanctions
⊗Not applicable, no regular reporting
Other data point. Please describe: ________________________________________________

Page Break

Q12
About how often does the leadership of your agency request specific data analyses related to TANF that are outside of regular reporting (i.e. ad hoc)? For example, examining caseload dynamics and trends in employment.

More than once a month
Monthly
More than once a quarter
Quarterly
More than once a year
Yearly
Less than once a year

Q13
Please briefly describe a recent leadership request for TANF data analysis outside regular reporting activities.

________________________________________________________________

Q14
In some states, TANF administrative data is available to government analysts or researchers outside the TANF agency. How useful has any research conducted outside the TANF agency with your administrative data been to informing your agency’s operations and planning?

	Extremely useful	Very useful	Moderately useful	Slightly useful	Not at all useful	N/A (no research conducted)
Studies by other governmental agencies
Studies by external partners (e.g., academic researchers or research firms)

Page Break

Q15
Within your TANF agency, how would you characterize the level of activity around...

Far too little activity

Far too much activity

...use of data overall?
...data linkage and integration?
...using data to build evidence about the TANF program?
...data sharing?

Q16

Within your TANF agency, how would you characterize the availability of financial resources, including personnel availability, for data usage specifically for...

Extremely inadequate

Extremely adequate

... data systems and management?
... regular reporting?
... ad hoc analyses and research

Page Break

Q17 How regularly does anyone from your TANF agency communicate with people from the following entities about using your data? (For example, through working groups, conferences, webinars, phone calls, etc.)

	A great deal	A lot	A moderate amount	A little	None at all
Other entities in your state
Other states
Federal government
External research partnerships (e.g., university researchers or research firms)
Advocates or media

Page Break

Q18 The next questions ask about the TANF policy and programmatic issues that are most important in your agency. Select the top THREE most important topics for your agency by dragging and dropping them into the box.

Let us know about any important topics that are not listed here!

Top Three Topics

______ Work requirements

______ Employment outcomes

______ Income disregard

______ Sanctions

______ Redetermination

______ Child-only cases

______ Time limits

______ Subsidized child care (CCDF)

______ Early education (pre-K)

______ Child welfare

______ Improper payments

______ Caseload dynamics

______ Effectiveness of programs

______ Disability

______ Special populations such as homeless families, immigrants or refugees, substance abuse, etc. Please specify:

______ Other policy or program issue(s). Please list:

Q19 For each of these common TANF policy and programmatic issues, indicate if it is an area of current analytic work for your agency.

	We are currently working on this issue	No current work but interest in pursuing	No current work, no interest
Work requirements
Employment outcomes
Income disregard
Sanctions
Redetermination
Child-only cases
Time limits
Subsidized child care (CCDF)
Early education (pre-K)
Child welfare
Improper payments
Caseload dynamics
Effectiveness of programs
Disability
Special populations (e.g., homeless families, immigrants, substance abuse, etc.). Please specify:
Other policy or program issue. Please list:

Q20 What are the primary barriers to data and analysis that limit the ability of your TANF agency to use data to inform key policy and programmatic issues? Check all that apply.

Information gathered here will inform technical assistance (TA) activities to help state TANF programs overcome barriers to data use and analysis.

Accessing TANF data that are formatted for analysis
Legal issues
Political issues
Staff time
Staff skills
Availability of technology and data tools
Communication challenges between IT and policy staff
Availability of financial resources
Other barriers. Please describe: ________________________________________________

Page Break

Q21
The last few questions ask more specifically about potential technical assistance (TA) offerings to support TANF agency data use.

What topics is your state most interested in receiving TA to address? Rank each topic as being of high, medium, or low priority.

Let us know about any important topics that are not listed here!

	High priority	Medium priority	Low priority
Data documentation/metadata
Data sharing and legal agreements
Structuring data for analysis
Data integration and record linkage
Data visualization
Use of software tools (e.g., SAS, R, Python)
Evaluation design (e.g., RCT, rapid cycle tests, quasi experimental)
Predictive analytics
Other areas for TA. Please list:

Q22
Thinking about the top priority areas for technical assistance (TA) you listed in the previous question, indicate the modes of technical assistance that would be most useful for your TANF agency.

	Extremely useful	Very useful	Moderately useful	Slightly useful	Not at all useful
Webinars
Workshops
Topical briefs
In-person TA
Sample analyses
Peer learning with other TANF agencies
One-on-one consultation on a specific project
On demand distance learning
Other modes of TA. Please list:

Page Break

Q23 This concludes the TANF Data Use and Opportunities module. Is there anything else you would like us to know about data usage in your TANF agency or state?

________________________________________________________________

Q24
If you are finished with this survey, click "SUBMIT" below.

Once you submit responses you will not be able to make any changes in the survey platform. Use the survey back button below to return to the survey to review and make any desired edits.

If you would like to update your answers in the future please reach out to us at [email@address] or [phone number].

End of Block: Introduction

TDI: Data Collection and Documentation

Start of Block: Introduction

Q1 Thank you for your willingness to participate in the TANF Data Innovation [hyperlink to website] survey about how TANF agencies use data. The goal of this survey is to collect first-hand information about your capacities, capabilities, and current use of data. Your answers will help HHS to: (1) better understand the challenges and barriers states and territories face in using data and research to inform program decision making and (2) design future technical assistance activities for TANF agencies to address these challenges. We will report a summary of aggregate findings from the project survey back to you.

This survey contains seven modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the Data Collection and Documentation module. This module seeks information about the TANF agency's data -- what is collected, how, and how it is documented and validated. It will take approximately 10 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0XXX and the expiration date is XX/XX/XXXX.

End of Block: Introduction

Start of Block: Module Specific Intro

Q2 Answer to the best of your knowledge and make an educated guess if you are unsure.

We encourage you to be as candid as possible. Your responses are private. This survey is being administered by Chapin Hall at the University of Chicago and MDRC; your individual responses will not be shared with anyone outside of the research team. Only aggregate state-level analyses will be shared with HHS. We do not anticipate any risks or direct benefits from participating in this survey.

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Q6
The TDI project aims to improve TANF agencies' ability to use data to inform program decision making. To get a complete picture of agency data use across domains and purposes, we ask this question in every module.

From your perspective, how well does your TANF agency use data to inform program decision making? Rate your response on a scale from 0 (not well at all) to 10 (extremely well).

Not well at all

Extremely well

Agency use of data

End of Block: Module Specific Intro

Start of Block: Data Collected

Q7 The next three questions ask about what data are collected or otherwise accessible to the TANF agency, and the source of those data.

For each type of information, please indicate the source for that information or if that information is not accessible for TANF program management. If the information is added from multiple sources, check all that apply.

Q8 Information about eligible individuals in the household:

	Reported by recipient	Collected or verified by agency staff or contractors	Integrated with TANF data from another data system (periodic match)	Integrated with TANF data from another data system (real time)	Not accessible for TANF program
Name
SSN
Birthdate
Educational attainment (for adults)
School information (for children)
Employment status (for adults)
Employer (for adults)
Income (for adults)
Assets
Disability status
Child welfare involvement

Q9 Information about the other benefits received by some or all of the individuals in the household:

	Reported by recipient	Collected or verified by agency staff or contractors	Integrated with TANF data from another data system (periodic match)	Integrated with TANF data from another data system (real time)	Not accessible for TANF program
Food assistance (SNAP, WIC)
Workforce or economic development services (non-TANF)
Housing assistance
Medical assistance (Medicaid, CHIP)
Disability assistance
Subsidized child care (CCDF)
Early education (pre-K, Head Start)
Child support

Q10 Information about work activities:

	Reported by recipient	Collected or verified by agency staff or contractors	Integrated with TANF data from another data system (periodic match)	Integrated with TANF data from another data system (real time)	Not accessible for TANF program
Activity type
Level of participation completed (hours, etc.)
Pay, if applicable
Start and end dates

Q11 If you would like to provide additional detail to clarify any of your answers about what data is collected and accessible to the TANF agency, please use this space to do so:

________________________________________________________________

Q12 For information self-reported by recipients, how are those data collected and entered into the system? Check all that apply.

Recipients complete eligibility forms electronically, and these data populate the system.
Recipients meet with frontline staff to complete eligibility processing, and frontline staff enter the data into the system.
Recipients complete paper eligibility forms; staff enter these into the system but do not directly interact with the recipients.
Other (please explain) ________________________________________________

Page Break

Q13 For which of these data points is historical information retained when the data are updated? Check all that apply.

Household size
Personal identifiers for head of household
Personal identifiers for other eligible individuals
Personal identifiers for individuals who are not eligible
Employment information
Household income
Address
Participation in other programs
Previous membership in other cases (old case numbers, etc)
Payment history
Sanction history

Q14 Are TANF application data, particularly data about applicants who did not receive benefits, preserved in an electronic form?

Page Break

Q15 When are payment data considered "final" and appropriate for analytical use?

Data are considered final from the time they are first entered into the system.
Data are stable and ready for analytical use within a month.
Data are stable and ready for analytical use within 3 months.
Data are stable and ready for analytical use after more than 3 months.

Q16 When are most eligibility data elements considered "final" and appropriate for analytical use?

Data are considered final from the time they are first entered into the system.
Data are stable and ready for analytical use within a month.
Data are stable and ready for analytical use within 3 months.
Data are stable and ready for analytical use after more than 3 months.

End of Block: Data Collected

Start of Block: Documentation and Validation

Q17 How frequently do frontline staff collecting data communicate with other data users about the quality of the data?

Never or rarely
Sometimes
Frequently

Q18 What data documentation is available to users?

	Not consistently documented	Somewhat documented (documentation may be scattered or unhelpful)	Well documented
List of fields
Data types (character, integer, etc.)
Field definitions
Code values
Limitations of certain fields
Detail about how data are collected

Q19 What strategies are used to maintain or improve data quality? Check all that apply.

Restrictions on valid values during data entry
Restrictions on data entry to block null values in certain fields
Routine data audits (automated)
Routine data audits (manual)
Validation of data elements against other databases
Training for staff doing data entry in appropriate values
Written data entry manuals
Aggregate data are shared with staff doing data entry
Other (please describe) ________________________________________________

End of Block: Documentation and Validation

Start of Block: Wrap Up

Q20 This concludes the Data Collection and Documentation module. Is there anything else you would like us to know about the data in your TANF agency?

________________________________________________________________

Q21
If you are finished with this survey, click "SUBMIT" below.

Once you submit responses you will not be able to make any changes in the survey platform. Use the survey back button below to return to the survey to review and make any desired edits.

If you would like to update your answers in the future please reach out to us at [email@address] or [phone number].

End of Block: Wrap Up

TDI: Data Infrastructure

Start of Block: Introduction

Q1 Thank you for your willingness to participate in the TANF Data Innovation [hyperlink to website] survey about how TANF agencies use data. The goal of this survey is to collect first-hand information about your capacities, capabilities, and current use of data. Your answers will help HHS to: (1) better understand the challenges and barriers states and territories face in using data and research to inform program decision making and (2) design future technical assistance activities for TANF agencies to address these challenges. We will report a summary of aggregate findings from the TDI survey back to you.

This survey contains seven modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the Data Infrastructure module. This module seeks information about the TANF agency's data infrastructure. It will take approximately 10-15 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

End of Block: Introduction

Start of Block: Module Specific Intro

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Q6
The TDI project aims to improve TANF agencies' ability to use data to inform program decision making. To get a complete picture of agency data use across domains and purposes we ask this question in every module.

From your perspective, how well does your TANF agency use data to inform program decision making? Rate your response on a scale from 0 (not well at all) to 10 (extremely well).

Not well at all

Extremely well

Agency use of data

End of Block: Module Specific Intro

Start of Block: Data Infrastructure and Security

Q7
First, we will ask some questions about your TANF agency's primary data system. This is the data system that is most used for information about cases, participants, and payments.

Do all counties in your state use the same primary data system to maintain information about cases, participants, and payments?

Yes
No
Not sure. Please describe: ________________________________________________

Skip To: Q10 If First, we will ask some questions about your TANF agency's primary data system. This is the data... = Yes

Q8 If different counties use different systems, how is that data integrated at the state level?

There is a central data system that fully integrates the data.
There is a central data system that integrates some but not all of the data.
There is no central data system, although individual-level data may be integrated across systems for analytic or reporting purposes.

Q9 For the next series of questions in this module, please answer relative to the primary data system that is the most comprehensive source of TANF data on cases, participants, and payments within the state. This might be a central data system that integrates most or all of the data across systems or it might be the data system used by a subset of counties that contains the largest share of the caseload.

Which of the following best characterizes the primary data system you will be thinking of to answer these questions?

A central, state data system
A data system used by a subset of counties containing at least 50% of the state's TANF caseload
A data system used by a subset of counties containing less than 50% of the state's TANF caseload

Page Break

Q10 Which of the following types of data are found in the TANF agency's primary data system?

	Contained entirely in primary data system	Partially contained in primary data system	Entirely contained in another data system	Not tracked
Current eligibility
Historical eligibility
Payment history
Work activities
Sanctions
Information about other benefits, such as SNAP
Referrals to other service providers

Q11 Is your primary data system off-the-shelf/previously developed software, previously developed software with some customization, or a completely custom product?

Off-the-shelf or other previously developed software
Off-the-shelf or other previously developed software that has been adapted for internal purposes
Custom (built in-house or by a vendor as a custom product)
Other (please explain) ________________________________________________

Q12 Please list the name of the software vendor or developer of your primary data system.

________________________________________________________________

Q13 How long has your department been using the current primary data system?

Less than 5 years
5-10 years
10-20 years
20+ years

Q14 Is your agency planning to move the primary data system to a new platform, and/or have you recently completed a database conversion?

We recently converted our primary data system to a new platform (within the last three years).
Money has been set aside for a database conversion.
Executive leadership is formally exploring a change in platform (i.e., through a committee or working group).
We have not recently changed database platforms and there is no active planning to do so.

Display This Question:

If Is your agency planning to move the primary data system to a new platform, and/or have you recent... = Money has been set aside for a database conversion.

Or Is your agency planning to move the primary data system to a new platform, and/or have you recent... = Executive leadership is formally exploring a change in platform (i.e., through a committee or working group).

Q15 Why are you planning to change platforms for your primary data system? Select all that apply.

Current system is hard to use.
Current system is hard to maintain or adapt.
Current system lacks desired quality or validation controls.
Current system does not integrate well with desired reporting or visualization tools.
It is difficult to export usable data from current system (limited output options).
The planned data system will contain a different universe of data than the current system (for example, changing to a system that integrates TANF and SNAP data from one that stores TANF data alone).
It is difficult to maintain desired level of security with current system.
There is federal funding available to update data systems.
There is non-federal funding available to update data systems.
Department leadership wants a change in data systems.
Other ________________________________________________

Display This Question:

If Is your agency planning to move the primary data system to a new platform, and/or have you recent... = We recently converted our primary data system to a new platform (within the last three years).

Q16 Why did you change platforms for your primary data system? Select all that apply.

Previous system was hard to use.
Previous system was hard to maintain or adapt.
Previous system lacked desired quality or validation controls.
Previous system did not integrate well with desired reporting or visualization tools.
It was difficult to export usable data from previous system (limited output options).
The current data system contains a different universe of data than the previous system (for example, changing to a system that integrates TANF and SNAP data from one that stores TANF data alone).
It was difficult to maintain desired level of security with previous system.
There was federal funding available to update data systems.
There was non-federal funding available to update data systems.
Department leadership wanted a change in data systems.
Other ________________________________________________

Page Break

Q17 What query tools are available to access, analyze, or extract data from the primary data system?

	Accessible to IT	Accessible to power users outside IT	Accessible to external developers (consultants)
Direct access with SQL
Direct access with other programming languages (Python, JavaScript, etc.)
Direct access through a querying, visualization, or business intelligence tool (Business Objects, Tableau, etc.)
Direct access through statistical software (SAS, SPSS, R, etc.)

Q18 Please list the primary tools and languages the audiences above are using to view, query, manipulate, and analyze these data.

________________________________________________________________

Page Break

Q19 Are any of your agency's TANF data stored in the cloud?

Yes, including data from the primary data system
Yes, but not the data in the primary data system
No

Display This Question:

If Are any of your agency's TANF data stored in the cloud? = Yes, including data from the primary data system

Or Are any of your agency's TANF data stored in the cloud? = Yes, but not the data in the primary data system

Q20 Which service(s) do you use for cloud storage (e.g., GCP, AWS, Azure, etc.)?

________________________________________________________________

Page Break

Q21 In addition to the primary data system discussed above, are there other systems used to store TANF data? Check all that apply.

One or more other database systems
Sharepoint
Excel spreadsheets
Other flat files
Other (please explain) ________________________________________________

Q22 Are there any tools used to access or analyze data from these other systems that you have not listed previously? If so, please list.

________________________________________________________________

Page Break

Q23 What processes exist that integrate data between the primary data system and one or more additional data systems? Check all that apply.

There is automated, real-time or on-demand integration between at least two systems for front-end viewers.
There is automated, real-time integration between at least two systems for reporting and analysis purposes (i.e., it is possible to pull reports of integrated data).
There is an automated process that routinely integrates the data between two systems and stores a crosswalk file or integrated dataset. Please indicate frequency of process (daily, monthly, etc.): ________________________________________________
There have been one or more completed manual or ad-hoc projects to integrate data across systems.
There is discussion or planning around additional automated integration across systems.
There is discussion or planning around additional integration (not automated) across systems.
There are data integration or interoperability efforts that are not captured by the above categories (please describe:) ________________________________________________
⊗I am not aware of any current or planned data integration or interoperability efforts.

Q24 Thinking about the above processes of data integration, please characterize the types of data contained in the data systems that are integrated with the primary data system:

	Automatically integrated	Integrated manually or on an ad-hoc basis	Planned for integration	No integration
Additional TANF program data (such as work activities or payments)
Child welfare
Food assistance (SNAP, WIC)
Workforce or economic development programs (non-TANF)
Employment security (unemployment insurance)
Other source of wage data
Housing assistance
Homelessness (HMIS)
Medical assistance (Medicaid, CHIP)
Juvenile justice
Corrections
Disability assistance
Early care and education (CCDF, pre-K, Head Start)
K-12 education
Postsecondary education
Child support
Other type of data (please list)

Q25 To the best of your knowledge, has your agency discussed, reviewed, or considered implementing the National Human Services Interoperability Architecture (NHSIA)?

Have not heard of it
Have heard of it, but have not discussed or considered implementing
Have discussed but chose not to implement
Have discussed but no decision was made
Have discussed and are exploring implementation in at least one data system
Have implemented in at least one data system that is used by the TANF office for program management purposes
Have implemented in the primary TANF data system

Q26 To the best of your knowledge, has your agency discussed, reviewed, or considered implementing the National Information Exchange Model (NIEM)?

Have not heard of it
Have heard of it, but have not discussed or considered implementing
Have discussed but chose not to implement
Have discussed but no decision was made
Have discussed and are exploring implementation in at least one data system
Have implemented in at least one data system that is used by the TANF office for program management purposes
Have implemented in the primary TANF data system

End of Block: Data Infrastructure and Security

Start of Block: Wrap Up

Q27 This concludes the Data Infrastructure module. Is there anything else you would like us to know about the data infrastructure in your TANF agency?

________________________________________________________________

Q28
If you are finished with this survey, click "SUBMIT" below.

Once you submit responses you will not be able to make any changes in the survey platform. Use the survey back button below to return to the survey to review and make any desired edits.

If you would like to update your answers in the future please reach out to us at [email@address] or [phone number].

End of Block: Wrap Up

TDI: Data Sharing

Q1 Thank you for your willingness to participate in the TANF Data Innovation [hyperlink to website] survey about how TANF agencies use data. The goal of this survey is to collect firsthand information about your capacities, capabilities, and current use of data. Your answers will help HHS to: (1) better understand the challenges and barriers states and territories face in using data and research to inform program decision making and (2) design future technical assistance activities for TANF agencies to address these challenges. We will report a summary of aggregate findings from the project survey back to you.

This survey contains seven modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the Data Sharing module. This module seeks information about the status of data sharing practices in your TANF agency and state. It will take approximately 10 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

End of Block: Introduction

Start of Block: Block 2

Q3 Please enter your contact information below. This information may be used to follow up with questions, clarifications, or further discussion. We will also report a summary of survey findings back to you. Your contact information will not be shared outside the research team.

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one role.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Not well at all

Extremely well

Agency use of data

Q7 This survey is focused on data sharing in your agency. Does your TANF agency have any agreements (e.g., contracts, DSAs, DULs, MOUs, or DUAs) in place regarding sharing data outside your agency? Check all that apply.

	Agreements to send TANF data to another entity	Agreements to bring data in to the TANF agency
Other state entities (e.g., agencies or programs)
City or county government agencies
Federal agencies (outside of federal reporting)
External researchers (e.g., academic institutions or research firms)
Contractors
Other organizations (e.g., non-profits, foundations, advocacy groups or media)
⊗No data sharing agreements

Q8 About how often does your agency receive requests from external entities to access or obtain TANF data?

	Weekly	Monthly	Quarterly	Annually	None in the last three years
Other state entities (e.g., agencies or programs)
City or county government agencies
Federal agencies (outside of federal reporting)
External researchers (e.g., academic institutions or research firms)
Contractors
Other organizations (e.g., non-profits, foundations, advocacy groups or media)

Page Break

Q9 Does your agency have a formal process or processes for handling external requests to access or obtain TANF data?

Display This Question:

If Does your agency have a formal process or processes for handling external requests to access or o... = No

Q10 In the absence of a formal process, please briefly describe what would happen if the agency received an external request for TANF data.

________________________________________________________________

Display This Question:

If Does your agency have a formal process or processes for handling external requests to access or o... = Yes

Q11 Select the responses that best describe how external requests to access or obtain TANF data are handled by the agency. Check all that apply.

Reviewed by a formal committee
Reviewed by agency leadership
Reviewed by designated decision maker outside of agency leadership
Other. Please describe: ________________________________________________

Display This Question:

If Does your agency have a formal process or processes for handling external requests to access or o... = Yes

Q12 On average, about how long does it take to work through the review process for external data requests?

Less than one month
1 - 3 months
4 - 6 months
7 - 12 months
More than 12 months

Display This Question:

If Does your agency have a formal process or processes for handling external requests to access or o... = Yes

Q13 About how long has the current practice for reviewing external data requests been in place?

Less than a year
About 1 - 5 years
About 6 - 10 years
At least 10 years

Page Break

Q14 How many of the following other government agencies or programs has your agency shared data with in the last five years? Please check all that apply.

In some cases, sharing data between the TANF agency and the agencies listed may be a part of regular operations (e.g., within the same department as TANF, sharing a common data system, colocated in an integrated database). Please still select those agencies, even if data sharing is routine and does not require additional agreements or contracts.

Note that generic names are listed. Agencies or programs may have different names in your state.

⊗No data sharing has occurred
Employment security (UI, wage data)
Child welfare
Food assistance (SNAP, WIC)
Workforce or economic development (non-TANF)
Medical assistance (Medicaid, CHIP)
Juvenile justice
Corrections
Housing assistance
Homelessness (HMIS)
Disability assistance
Subsidized child care (CCDF)
Early education (pre-K, Head Start)
K-12 education
Post-secondary education
Child support
Other agencies or departments. Please list: ________________________________________________

Page Break

Q15 Has your agency shared data with external researchers (e.g., academic institutions or research firms) in the last five years?

Yes
No
Don't know

Display This Question:

If Has your agency shared data with external researchers (e.g., academic institutions or research fi... = Yes

Q16 Are external researchers ever given access to data with PII (personally identifiable information, e.g., name or SSN)?

Yes
No
Don't know

Display This Question:

If Has your agency shared data with external researchers (e.g., academic institutions or research fi... = Yes

Q17 How is data shared with external researchers? Check all that apply.

Data file is sent to researchers.
Researchers connect to the agency database directly.
Data is shared through another agency or organization that centralizes data sharing (such as an integrated database).
Other method of data sharing. Please describe: ________________________________________________

Q18 Overall, how would you characterize your TANF agency's comfort with sharing data outside the agency?

Not at all comfortable

Extremely comfortable

With other state agencies
With external researchers (e.g., university or research firm)

Page Break

Q19 What are the primary barriers to data sharing in your TANF agency and state? Check all that apply.

No interest in data sharing
Provides limited value to the TANF program
Technology is not sufficiently advanced
Concerns about how data are used
Concerns about maintaining data privacy and security
Staff availability to oversee and manage data sharing
Legal issues/statutory limitations
Political issues (e.g., legislature or governor)
Availability of financial resources
Other. Please list: ________________________________________________

Q20 Please briefly describe a recent example or specific barrier to data sharing in your TANF agency or state.

________________________________________________________________

Page Break

Q21 This concludes the Data Sharing module. Is there anything else you would like us to know about data sharing in your TANF agency or state?

________________________________________________________________

Q22
If you are finished with this survey, click "SUBMIT" below.

Once you submit responses you will not be able to make any changes in the survey platform. Use the survey back button below to return to the survey to review and make any desired edits.

If you would like to update your answers in the future please reach out to us at [email@address] or [phone number].

End of Block: Block 2

TDI: Research and Analytic Capacity

Start of Block: Introduction

This survey contains 7 modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the Research and Analytic Capacity module. This module seeks information about the TANF agency's current capacity to conduct data analysis and research to inform agency decisions. It will take approximately 10-15 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

The TANF Data Innovation project is sponsored by the Office of Planning, Research, and Evaluation in collaboration with the Office of Family Assistance in the Administration for Children and Families, U.S. Department of Health and Human Services. The project is led by MDRC, in partnership with Chapin Hall at the University of Chicago, Actionable Intelligence for Social Policy (AISP) at the University of Pennsylvania, and New York University.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0XXX and the expiration date is XX/XX/XXXX.

End of Block: Introduction

Start of Block: Module Specific Intro

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Not well at all

Extremely well

Agency use of data

Q7 This module begins with questions about staff capacity. Among analytic staff in your TANF agency, please indicate the highest level of knowledge in the following areas:

	Extremely knowledgeable	Moderately knowledgeable	Slightly knowledgeable	Not knowledgeable at all	N/A, we don't do this in-house
Descriptive statistics
Performance indicators
Data manipulation
Record linkage/data integration
Data visualization
Program evaluation
Statistical inference
Mapping/geospatial
Predictive analytics

Q8 Among analytic staff in your agency, indicate the highest level of proficiency with the following programming languages and tools:

	Expert	Advanced	Intermediate	Novice	None, no users
SQL
SAS
Stata
SPSS
R
Python
Microsoft Excel
Microsoft Access
ArcGIS/QGIS, etc.
Tableau/Power BI, etc.
Spark/Hive/Hadoop, etc.
Record linkage package or software. Please list:
Other language or tool. Please list:

Page Break

Q9
The next questions ask about staff access to and usage of administrative data, specifically access to case- and individual-level datasets extracted for report generation or data analysis.

About how frequently do analytical staff access extracted datasets of case- or individual-level data for report generation or data analysis?

Weekly
Monthly
Quarterly
Annually
Every few years
Never

Q10 How up to date are the case- and individual-level TANF datasets that can be most easily accessed by analytic staff?

Prior month
Prior quarter
Prior year/fiscal year
More than 1 year old

Q11 How do analytic staff access extracted case- and individual-level TANF data? Check all that apply.

Prebuilt reports
Business intelligence software
Direct queries
⊗No access to extracted case- or individual-level data
Other access method. Please describe: ________________________________________________

Q12 Besides TANF data, about how frequently do analytic staff access these other sources of administrative data to aid in ad hoc reporting and other analyses (i.e., activities outside of regular reporting procedures)?

	Weekly	Monthly	Quarterly	Annually	Every few years	Never
Wage or earnings data (UI)
Child welfare
Food assistance (SNAP, WIC)
Workforce or economic development (non-TANF)
Medical assistance (Medicaid, CHIP)
Juvenile justice
Corrections
Housing assistance
Homelessness (HMIS)
Disability assistance
Subsidized child care (CCDF)
Early education (pre-K, Head Start)
K-12 education
Post-secondary education
Child support
Other source(s). Please list data source:

Display This Question:

If Besides TANF data, about how frequently do analytic staff access these other sources of administr... = Weekly

Q13 Indicate all sources of wage data that your agency uses for analyses. Check all that apply.

Unemployment Insurance (UI)
National Directory of New Hires (NDNH)
State directory of new hires
Social Security
U.S. Census data
Labor market data (e.g., Department of Labor or Burning Glass)
Other source(s) of wage data. Please list: ________________________________________________

Q14 Which of the other sources of administrative data are integrated with your primary sources of TANF data (i.e., analysts can report on individuals across multiple data sources)?

	Completely integrated (same data system or real-time interoperability)	Integrated for analytic purposes on a regular basis	Integrated for analytic purposes on an occasional basis	Not integrated with TANF data for analytic purposes
Wage or earnings data (UI)
Child welfare
Food assistance (e.g. SNAP, WIC)
Workforce or economic development (non-TANF)
Medical assistance (Medicaid, CHIP)
Juvenile justice
Corrections
Homelessness (HMIS)
Housing assistance
Disability assistance
Subsidized child care (CCDF)
Early education (pre-K, Head Start)
K-12 education
Post-secondary education
Child support
Other source(s). Please list data source:

Q15 Who manages access to any integrated data described above? If there are multiple sources of integrated data, select all that apply and describe in your answer to the next question.

No sources of integrated data
The agency or department that houses the TANF program
The governor's office
A central data or technology department within the state
Another agency (e.g., education, labor, etc.). Please specify: ________________________________________________
A university or other external research partner
A private contractor external to the state
Other. Please specify: ________________________________________________

Q16 If there are multiple sources of integrated data managed by different organizations or agencies, please explain.

________________________________________________________________

Q17
The last questions in this module ask about production and reporting of research and analyses.

Does your agency make any reports about the TANF caseload publicly available?

Display This Question:

If The last questions in this module ask about production and reporting of research and analyses. D... = Yes

Q18 If possible, please provide citations or hyperlinks to any recent public caseload reports using your TANF administrative data.

________________________________________________________________

Q19 Indicate whether your agency, other organizations, or external researchers have conducted at least one evaluation of TANF or a program component in the last five years.

	Yes	Maybe	No
TANF agency
Another governmental entity
External researchers (e.g. university researcher)

Display This Question:

If Indicate whether your agency, other organizations, or external researchers have conducted at leas... = TANF agency [ Yes ]

Q20 Were any evaluation reports by the TANF agency made public?

Display This Question:

If Indicate whether your agency, other organizations, or external researchers have conducted at leas... = Another governmental entity [ Yes ]

Q21 Were any evaluation reports by other governmental entities made public?

Display This Question:

If Indicate whether your agency, other organizations, or external researchers have conducted at leas... = External researchers (e.g. university researcher) [ Yes ]

Q22 Were any evaluation reports by external researchers made public?

Display This Question:

If Indicate whether your agency, other organizations, or external researchers have conducted at leas... = TANF agency [ Maybe ]

Q23 You indicated there might have been a recent TANF evaluation. Please briefly describe what you do know about any such evaluation(s).

________________________________________________________________

Display This Question:

If Indicate whether your agency, other organizations, or external researchers have conducted at leas... = TANF agency [ Yes ]

Q24 If possible, please provide citations or hyperlinks to any recent public evaluation reports using your TANF administrative data.

________________________________________________________________

Q25 Please briefly describe the primary barriers your agency experiences in conducting analytic activities and research.

________________________________________________________________

Q26 This concludes the Research and Analytic Capacity module. Is there anything else you would like us to know about the research practices of your TANF agency?

________________________________________________________________

Q27
If you are finished with this survey, click "SUBMIT" below.

Once you submit responses you will not be able to make any changes in the survey platform. Use the survey back button below to return to the survey to review and make any desired edits.

If you would like to update your answers in the future please reach out to us at [email@address] or [phone number].

Page Break

TDI: Payment Integrity

Start of Block: Introduction

This survey contains seven modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the Payment Integrity module. This module seeks information about how your agency uses data to support TANF quality assurance efforts focused on payment integrity. It will take approximately 15 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

Page Break

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Q6
The TDI project aims to improve TANF agencies' ability to use data to inform program decision making. To get a complete picture of agency data use across domains and purposes, we ask this question in every module.
From your perspective, how well does your TANF agency use data to inform program decision making? Rate your response on a scale from 0 (not well at all) to 10 (extremely well).

Not well at all

Extremely well

Agency use of data

Q7 This module contains questions concerning payment integrity, based on applicable state and federal requirements.

Payment integrity is the process for preventing, identifying, and addressing improper payments. Improper payments are payments calculated incorrectly, denied incorrectly, or issued without sufficient proof of eligibility.

Q8 How satisfied are you with the timeliness and efficiency of your state’s TANF payment integrity process?

Very satisfied
Moderately satisfied
Slightly satisfied
Not satisfied at all

Q9 How satisfied are you with the accuracy of your state’s TANF payment integrity process?

Very satisfied
Moderately satisfied
Slightly satisfied
Not satisfied at all

Q10 Over the years, HHS has suggested a number of strategies to improve the accuracy of TANF eligibility determinations and benefit calculations. Based on your experience:

How effective are each of the following methods for minimizing initial errors in eligibility/benefit determinations?

	Decrease incidence of problems	Have little or no effect	Increase the incidence of problems	No experience with that method
Hiring more staff responsible for determining eligibility
Training staff to solicit accurate and complete information
Including measures of payment integrity in staff performance reviews
Designing new ways to remind TANF recipients to provide complete and timely information
Creating or accessing integrated databases of multiple assistance programs
Changing the frequency of re-determinations to better detect changes in case composition or income
Using databases (PARIS, SDNH or NDNH, IEVS, etc.) to collect information on household eligibility status and income

Q11 How effective are each of the following methods for identifying existing errors in eligibility/benefit determinations?

	Decrease incidence of problems	Have little or no effect	Increase the incidence of problems	No experience with that method
Creating quality control units
Assigning more staff to quality control units
Conducting quality control reviews
Performing quality control checks, using available case file data
Using state databases (PARIS, SDNH, etc.) to collect information on household eligibility and income
Using predictive analytics to flag cases most likely to have payment integrity issues

Page Break

Q12 What are the barriers to correcting improper payments once they are discovered? Choose up to three.

Staffing levels
Staff skills and experience
Funding for payment integrity efforts
Procedures incorporating new information that affects eligibility or benefit calculations
Difficulties recouping overpayments from beneficiaries
Statutory limitations or limitations stemming from state laws
Restrictions on data matching or use in current data sharing agreements
Technology challenges
Other. Please list: ________________________________________________

Page Break

Q13 On average, about how long does it take for a case manager to learn about an increase in a TANF recipient’s:

	Less than a month	About 1-4 months	About 5-8 months	About 9-12 months	More than 12 months
Earned income?
Child support?
Other income?

Q14 Which federal or multistate data systems does your state access for TANF payment integrity purposes? Check all that apply.

Public Assistance Reporting Information System (PARIS)
Income Eligibility Verification System (IEVS)
National Directory of New Hires (NDNH)
Social Security
Don't use any of these data systems
Don't know/uncertain

Display This Question:

If Which federal or multistate data systems does your state access for TANF payment integrity purpos... = Public Assistance Reporting Information System (PARIS)

Q15 Indicate the specific uses of the PARIS system for payment integrity in your agency. Check all that apply.

Interstate public assistance match
Department of Defense match
Veterans match
Don't know/uncertain

Display This Question:

If Which federal or multistate data systems does your state access for TANF payment integrity purpos... = Income Eligibility Verification System (IEVS)

Q16 Indicate the specific uses of IEVS data for payment integrity in your agency. Check all that apply.

State Wage Information Collection Agency
State Unemployment Compensation Agency
Homeland Security
Internal Revenue Service
Don't know/uncertain

Display This Question:

If Which federal or multistate data systems does your state access for TANF payment integrity purpos... = National Directory of New Hires (NDNH)

Q17 Indicate the specific uses of NDNH data for payment integrity in your agency. Check all that apply.

W-4 New Hire data
Quarterly Earnings data
Unemployment Insurance Benefits data
Don't know/uncertain

Display This Question:

If Which federal or multistate data systems does your state access for TANF payment integrity purpos... = Social Security

Q18 Indicate the specific uses of Social Security data for payment integrity in your agency. Check all that apply.

Numident
Master Beneficiary Record Database
Don't know/uncertain

Page Break

Q19 Which state or local data systems does your state access for TANF payment integrity purposes? Check all that apply.

Unemployment Insurance (UI) wage and benefits
Supplemental Nutrition Assistance Program (SNAP)
Medicaid
Child support
Public or subsidized housing
Vital statistics
State or local multiagency integrated database
⊗Don't use any state or local data systems
Other. Please list: ________________________________________________

Q20 Does your state calculate a TANF payment error rate?

Yes
No
Don't know/uncertain

Display This Question:

If Does your state calculate a TANF payment error rate? = Yes

Q21 About how many cases per year does your state include in the Quality Control sample?

________________________________________________________________

Display This Question:

If Does your state calculate a TANF payment error rate? = Yes

Q22 Where do you report the TANF payment error rate information? Check all that apply.

In a report to the governor or state legislature
In a document or tables posted on the agency website
In an internal or confidential document
Other. Please describe: ________________________________________________

Display This Question:

If Does your state calculate a TANF payment error rate? = Yes

Q23 Do you report the TANF payment error rate as...

	Yes	No
A statewide average?
An average for each county or region?
An average by program type (federally funded, state funded, mixed federally and state funded)?

Page Break

Q24 One of the goals of the TANF Data Innovation project is to identify and disseminate best practices for improving the accuracy and timeliness of TANF eligibility and benefits data. We would like to learn more about current efforts by state and local human services administrators and other “stakeholders” to share information. In your opinion, which jurisdictions (state, county, or territory), other than your own, use innovative or effective approaches to improving payment integrity that other states should learn about? Name up to three jurisdictions.

State/Jurisdiction 1 ________________________________________________
State/Jurisdiction 2 ________________________________________________
State/Jurisdiction 3 ________________________________________________

Q25 In your opinion, which organizations, if any, are particularly helpful in disseminating best practices for improving payment integrity? Select up to three.

National Governors Association
National Association of Counties
National Association of State TANF Administrators
American Public Human Services Association
National Association for Program Information and Performance Management
American Association of Health and Human Services Attorneys
Other. Please list: ________________________________________________

Page Break

Q26 We may have a few follow-up questions or clarifications. In case you aren't available, please provide contact information for a colleague in your TANF agency or state that would be another good source of information about payment integrity and quality assurance. Contact information will not be shared outside the research team.

If you have no one to recommend, leave these fields blank and skip to the next question. Thanks!

Name ________________________________________________
Email ________________________________________________

Q27 This concludes the Payment Integrity module. Is there anything else you would like us to know about the payment integrity efforts in your TANF agency?

________________________________________________________________

End of Block: Introduction

TDI: Federal Reporting

Start of Block: Introduction

Q1 Thank you for your willingness to participate in the TANF Data Innovation [hyperlink to website] survey about how TANF agencies use data. The goal of this survey is to collect first-hand information about your capacities, capabilities, and current use of data. Your answers will help HHS to: (1) better understand the challenges and barriers states and territories face in using data and research to inform program decision making and (2) design future technical assistance activities for TANF agencies to address these challenges. We will report a summary of aggregate findings from the project survey back to you.

This survey contains seven modules focusing on different components of TANF data usage. Each module is intended to be answered by agency staff with expertise in a given topic. You have been selected to complete the Federal Reporting module. This module seeks information about the data you prepare to submit to HHS as part of the TANF and SSP-MOE Data Reports. It will take approximately 10 minutes to complete. You can preview the survey here [link to survey pdf]. Your participation is voluntary.

Contact [email@address] or [phone number] with any questions or concerns.

Thank you so much for your thoughts and time! Your contributions are important.

End of Block: Introduction

Start of Block: Survey specific intro

Q2 Answer to the best of your knowledge and make an educated guess if you are unsure.

We encourage you to be as candid as possible. This survey is being administered by Chapin Hall at the University of Chicago and MDRC; your responses are private and will only be viewed by the research team. Although this project is federally sponsored, results will only be shared with HHS in aggregate form that does not identify individual respondents or states. We do not anticipate any risks or direct benefits from participating in this survey.

Name ________________________________________________
Job title ________________________________________________
Email ________________________________________________
Phone ________________________________________________

Q4 Which of the following best characterizes your role relative to the TANF program? Select your primary role if you have more than one.

Executive leadership
Reporting or data analysis
Research or evaluation
Information technology
Program staff
Legal
External consultant
Other. Please list: ________________________________________________

Q5 How would you characterize your role relative to the TANF program?

Working solely or mostly on TANF
Working across multiple programs, including TANF

Not well at all

Extremely well

Agency use of data

End of Block: Survey specific intro

Start of Block: DIsaggregated Data

Q7 The first set of questions in this module are about the disaggregated data (Sections One and Two of the TANF Data Report/ACF-199 and SSP-MOE Data Report/ACF-209) that your state reports to HHS each quarter. This set of questions comprises the majority of the module.

Q8 About how many hours do staff from your state spend preparing the disaggregated data to report federally each quarter? Include time spent extracting, formatting, validating, aggregating, and transmitting the data, as well as any time needed to answer HHS questions or complete requested revisions, but do not include initial set up time (i.e., time spent on report design) that only happens once.

Hours required to prepare the report each quarter: ________________________________________________

Q9 Are there any additional costs to preparing these disaggregated data to report federally each quarter? Check all that apply.

Paying a consultant to prepare data or run report
Maintaining a software license primarily for use running federal reporting
Paying another state agency to prepare or provide data or run report
Other (please specify) ________________________________________________
⊗No additional costs to preparing report

Q10 For each of the types of data included in disaggregated reporting each quarter, please indicate the level of difficulty (in terms of resources) in preparing those data for federal use.

	Extremely easy	Somewhat easy	Somewhat difficult	Extremely difficult
Information about TANF benefits (months, grant amount, case status)
Personal identifiers for individual recipients
Case geography (county, zip code)
Information about other public benefits received
Work participation status for the case
Work activity detail for adult recipients

Page Break

Q11 Please indicate whether your agency conducts a review, quality control, or validation on any of the following types of data as part of the process of preparing for submission of disaggregated data.

	Data is reviewed prior to submission	Data is submitted "as is"
Information about TANF benefit (months, grant amount, case status)
Personal identifiers for individual recipients
Case geography (county, zip code)
Information about other public benefits received
Work participation status for the case
Work activity detail for adult recipients

Q12 For each of the types of data included in disaggregated reporting each quarter, please indicate your sense of the accuracy of the data that is ultimately submitted.

	Extremely accurate	Very accurate	Moderately accurate	Slightly accurate	Not accurate at all
Information about TANF benefit (months, grant amount, case status)
Personal identifiers for individual recipients
Case geography (county, zip code)
Work participation status for the case
Work activity detail for adult recipients
Receipt of subsidized housing
Receipt of medical assistance
Receipt of food stamps
Receipt of subsidized child care
Child support amount

Q13 For your state's submission to HHS of disaggregated TANF data, do you submit universe or sample data?

Universe data
Sample data

Skip To: End of Block If For your state's submission to HHS of disaggregated TANF data, do you submit universe or sample d... = Universe data

Q14 Why does your agency submit sample data to HHS? Check all that apply.

Preparing universe data would take more time
Preparing universe data would cost more money
Concerns about the quality of universe data (cannot be quality controlled in the same way as sample data)
Lack of staff capacity to access universe data (build reports, etc.)
Lack of technical capacity to access universe data (reporting infrastructure)
Other difficulties extracting universe data
Other ________________________________________________

Q15 Please estimate the staff time, in hours, required specifically to sample the data and prepare the stratum file (Section Four of the TANF Data Report and SSP-MOE Data Report) each quarter. Do not include initial set-up time (i.e., time spent on report design) that only happens once.

Hours required to sample data and prepare stratum file: ________________________________________________

Page Break

End of Block: DIsaggregated Data

Start of Block: Aggregated Data

Q16 The next set of questions are about the aggregated data your state reports to HHS each quarter (Section Three of the TANF Data Report and the SSP-MOE Data Report).

Q17 About how many hours do staff from your state spend preparing the aggregated data to report federally each quarter? Include time spent extracting, formatting, validating, aggregating, and transmitting the data, as well as any time needed to answer HHS questions or complete requested revisions, but do not include initial set-up time (i.e., time spent on report design) that only happens once.

Hours required to prepare the report each quarter: ________________________________________________

Q18 Please indicate the overall level of difficulty in preparing the aggregated data for federal reporting.

Extremely easy
Somewhat easy
Somewhat difficult
Extremely difficult

Q19 Please indicate your sense of the accuracy of the aggregated data that is ultimately submitted each quarter.

Extremely accurate
Very accurate
Moderately accurate
Slightly accurate
Not accurate at all

End of Block: Aggregated Data

Start of Block: General Questions about Federal Reporting

Q20 This final set of questions asks about the federal reporting process overall.

Q21 Please indicate whether your TANF agency uses the data extracts prepared for federal reporting for any other purposes. Check all that apply.

For routine internal reports and analyses
For ad hoc internal reports and analyses
For another purpose (please list) ________________________________________________
⊗None of the above

Q22 Please describe any aspects of the federal guidance for reporting TANF data that are confusing or difficult to follow.

________________________________________________________________

Q23 How would you characterize the process of transmitting data to OFA each quarter?

Extremely easy
Somewhat easy
Somewhat difficult
Extremely difficult

Q24 What are the most significant barriers or challenges your state faces with regard to federal reporting? Check all that apply.

Required data elements are not tracked in a single state data system
Difficult to extract reports from data system in desired format
Lack of internal capacity to run or monitor federal reporting
Data quality issues in required elements
Written guidance is confusing
Transmission process is difficult
Rejection notes and HHS questions are difficult to address
Other - please describe ________________________________________________
⊗State does not experience any challenges with federal reporting

Q25 How could HHS improve support for states like yours in preparing these reports for federal submission? Check all that apply.

Improved written guidance
Supplemental guidance documents (FAQ, etc.)
Technical assistance hotline or email address to answer questions
Connect states with peers with similar data systems
Connect states with peers with similar TANF policies or caseload characteristics
Improve the process for transmitting data files
Hands-on consultation
Other ideas or additional detail on any of the above ________________________________________________
⊗Improved support is not needed

End of Block: General Questions about Federal Reporting

Start of Block: Wrap Up

Q26 This concludes the Federal Reporting module. Is there anything else you would like us to know about the process of federal reporting for your state?

________________________________________________________________

End of Block: Wrap Up

Page 88 of 88

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	TDI: Assignment Module
Author	Qualtrics
File Modified	0000-00-00
File Created	2021-01-15