SSB - Program Support GenIC: ACF Privacy and Confidentiality Toolkit Case Studies

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

ACF Privacy and Confidentiality Toolkit Case Studies

Formative Data Collections for ACF Program Support

0970 – 0531

Supporting Statement

Part B

December 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Brett Brown

Aaron Goldstein

B1. Objectives

Study Objectives

The U.S. Department of Health and Human Services’ Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation (OPRE) seeks approval for Westat to ask human service organizations participating in responsible data sharing initiatives, and their partners, to complete questionnaires and participate in semi-structured interviews. These collections will be used to develop case study reports that are likely to (a) inspire other human service organizations to share data and provide ideas on how to accomplish those goals, and (b) show ACF how to better assist organizations who want to share data.

Generalizability of Results

The questionnaires will help Westat determine (a) if a case study about an organization is likely to inspire other organizations to share data and (b) the organization’s interest in participating. The interviews will help Westat collect internally-valid information about potential organizations. However, since each organization has a different structure, uses different technology, must abide by different state laws, and faces different external influences, the case studies will not include specific guidance that is generalizable to, or otherwise represents, all other organizations.

Appropriateness of Study Design and Methods for Planned Uses

The questionnaires will help Westat determine which organizations will create the best case studies. The results will be considered alongside (a) publicly available information about the organization and (b) any knowledge from a panel of experts working with Westat. Since the questionnaire is only one tool that Westat may use, making it brief and high-level, and asking the organization’s leader or delegate to complete it, should efficiently acquire necessary information.

The semi-structured interviews of about 8 individuals connected to each data sharing initiative should, when combined with publicly available data, provide Westat the information needed to develop a valuable case study report. Using semi-structured interviews gives (a) interviewees opportunity to fully discuss their experiences and (b) Westat opportunity to adjust their questions to assure they gather the full story. Interviewing multiple individuals connected to each data sharing initiative balances the fact that individuals in different roles often have different experiences and perspectives. Limiting involvement to 8 individuals per organization will minimize the time staff are asked to spend on this project.

B2. Methods and Design

Target Population

OPRE identified at least 20 different human services organizations participating in some form of data sharing initiative via a preliminary scan of public documents. Westat’s panel of experts may identify several additional organizations.

Westat plans to complete three case study reports per year and therefore anticipates sending up to 9 questionnaires to organizational leaders and interviewing up to 24 individuals connected to the data sharing initiative. As noted in Supporting Statement A, this initial request is for 36 months (i.e., 9 case studies, 27 questionnaires, and 72 interviews). We plan to request an extension to continue data collection through August 2025.

Sampling and Site Selection

Westat’s goal is to identify and develop case study reports about different types of human service organizations that developed data sharing initiatives and have a story that is likely to inspire, and provide helpful and relevant information to, other organizations to develop their own initiative. For example, we know the challenges associated with sharing data for research purposes are different than the challenges associated with sharing data that impacts an individual’s services. Likewise, the challenges associated with sharing employment data are different than the challenges associated with sharing health care data. Westat may give more weight to an organization if ACF or Westat’s panel of experts think there is heightened interest in how that organization shares its data.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

Westat’s goal is to identify and develop case study reports about different types of human service organizations that developed data sharing initiatives and have a story that is likely to inspire other organizations to develop their own initiative.

Westat designed the questionnaire to identify organizations that share data, how they share data, and if they would like to participate. They designed the interview questions to collect the information needed to understand their data sharing activities, how they developed, what went well, and what led to challenges. The interviews are semi-structured to allow Westat to individualize the questions based on the interviewee’s role and their responses to questions as they move through the interview.

Since these collections are not designed to be statistically significant, we do not have a process in place to measure error. However, the leader of the organization selected for each case study will have opportunity to review the draft case study report and request changes to anything inaccurate before it is published.

B4. Collection of Data and Quality Control

All information will be collected by Westat, Inc., under a contract with ACF.

Westat plans to identify organizations by reviewing publicly available information and talking to a small panel of experts. They may look online to find the contact information for leaders of the more promising organizations and email them both a (a) letter about the project and (b) questionnaire about their data sharing initiative and interest in becoming a case study. Westat will ask that the questionnaire be completed by either the leader or their delegate.

Once Westat selects an organization, they will send the leader an email further explaining the project and a letter of support from the ACF Deputy Assistant Secretary for Planning Research & Evaluation. They will try to gather and review additional harder-to-find public information about the organization. Human service organizations usually publicize a privacy statement that summarizes current privacy and confidentiality practices. However, they generally do not publicize how a data sharing initiative evolved, such as the history, underlying concerns that shaped the end results, parties that participated in the decisions, and why certain plans were cancelled. For this reason, Westat will also interview individuals who work for the organization and potentially their partners. Since each organization has a different structure, uses different technology, has different sharing partners, and faces different external influences, the roles and number of individuals chosen to discuss each data sharing initiative will vary.

Interviews may occur in person, via phone, or via virtual conference software depending on the interviewee’s preferences. Each interview will either be recorded, with respondent’s permissions, or a second person will take notes. That will help assure that the individual asking the questions is able to focus on the questions and the transcript can be accurately captured. Moreover, the leader of each organization will have an opportunity to review the draft case study report and request changes to anything inaccurate before it is published.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The questionnaires and interviews are not designed to create specific guidance that is generalizable to, or otherwise represents, all other organizations. For that reason, response rates will not be reported.

NonResponse

Participants will not be randomly sampled and findings are not intended to be representative so non-response bias will not be calculated or reported. The respondent’s employer and their role may be documented and reported in written materials associated with the data collection. This will be explained to respondents prior to participation.

B6. Production of Estimates and Projections

The data will not be used to make policy decisions or generate population estimates for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

Questionnaires. Westat will ask the leader of the organization to complete a questionnaire or delegate that task to someone with who understands the data sharing initiative. The respondent may either (a) complete it electronically and email it back to Westat or (b) have Westat ask the questions over the phone and respond over the phone. If Westat has any follow-up clarifying questions they will cover those via email or phone. Westat will offer these methods because they are generally the most convenient and the information is not sensitive enough to warrant another method.

Interviews. Once Westat selects an organization for a case study, they will ask its leader to identify key individuals who can best discuss the data sharing initiative from different perspectives and determine when they can meet for a semi-structure interview with Westat. Those interviewees will have the option to participate in-person, via a web conference, or via telephone because those are generally the most convenient forums. They will also have the option to e-mail documents to Westat if that is more efficient than orally responding to a question. The leader of each organization will have opportunity to review the draft case study report and request changes to anything inaccurate before it is published.

The information that Westat collects will be stored in various Microsoft Office or Adobe formats (e.g. Word, Excel, PDF) because the volume of data will not be large, complicated, or sensitive enough to require other software.

Data Analysis

The data collected for this project will not be coded because there is not enough information to need a system. To select the best organizations, Westat will simply review their notes about each potential organization, to include any questionnaires, and make a decision. Likewise, when developing the case studies, Westat will simply review their notes from the interviews and documents, identify where it best fits within the base outline, and then write the paper.

Data Use

The questionnaires for each organization will be evaluated, along with all publicly available documents and the comments from the panel of experts, to determine which organization should become a case study.

Westat will compile notes from the semi-structured interviews and public documents into a case study report that will be published on ACF’s website. Information is intended to be used by similar organizations and other stakeholders as they develop or evolve their own data sharing initiatives. ACF will also use the information to inform internal planning around support for organizations development or improvement of data sharing initiatives.

B8. Contact Persons

The Federal employee leads for this project are:

• Brett Brown, [email protected],

• Aaron Goldstein, [email protected]

The Westat employees who will collect and process the information include:

• Mary Gabay, [email protected]

• Frank Bennici, [email protected]

Attachments

Instrument 1 - Questionnaire to help select organizations

Instrument 2 - Semi-Structured Interviews

Attachment 1 - Example Letter of Engagement

Attachment 2 - Draft Case Study Outline

6