Ssa

Prevention of Child Maltreatment through Policy Change

SUPPORTING STATEMENT

INFORMATION COLLECTION REQUEST

Part A

Supported by:

Department of Health and Human Services (DHHS)

Centers for Disease Control and Prevention (CDC)

National Center for Injury Prevention and Control (NCIPC)

Division of Violence Prevention (DVP)

Government Project Officer:

Natasha E. Latzman, Ph.D., Science Officer

Phone# 770-488-3935

Email: [email protected]

Fax# 770-488-1662

March 10, 2014

Table of Contents

Abstract

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

2. Purpose and Use of Information Collection

3. Use of Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Business or other Small Entities.

6. Consequences of Collecting Information Less Frequently.

7. Special Circumstances Relating to the Guidelines of 5CFR 13205

8. Comments in Response to Federal Register and Efforts to Consult Outside the Agency.

9. Explanation of Any Payments or Gifts to Respondents.

10. Assurance of Confidentiality Provided to Respondents.

11. Justification for Sensitive Questions.

12. Estimates of Annualized Burden Hours, and Costs.

13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers.

14. Annualized Costs to the Federal Government

15. Explanation for Program Changes

16. Plans for Tabulation, Publication, and Project Time Schedule.

17. Reason(s) Display of OMB Expiration Date Inappropriate.

18. Exemptions to Certification for Paperwork Reduction Act Submissions

LIST OF ATTACHMENTS

Attachment A Authorizing Legislation

Attachment B Published 60-Day Federal Register Notice

Attachment C IRB Approval Memo

Attachment D Survey of County TANF and Child Welfare Respondents

Attachment E Survey of State-Level Administrators

Attachment F Interview Guide for County Directors of Human Services

Attachment G Interview Guide for Child Welfare/Colorado Works Leadership/Manager

Attachment H Interview Guide for Child Welfare/Colorado Works Case Manager, Technician, and other Client Serving Staff

Attachment I Interview Guide for Allied Staff

Attachment J Interview Guide for Partners

Attachment K Screenshots of Web-based Survey of County TANF and Child Welfare Respondents

Attachment L Screenshots of Web-based Survey of State Administrators

Attachment M Nondisclosure Agreement

Attachment N Transcriber Data Security Agreement

Attachment O Project Information and Informed Consent Statement for Interviews

Attachment P Interview Guide for State-Level Administrator

Attachment Q Interview Guide for Data Managers

Attachment R Public Comment

Attachment S Interview Respondent Verbal Consent Form

A. Justification

A.1. Circumstances Making the Collection of Information Necessary

Background

The Centers for Disease Control and Prevention (CDC) is seeking OMB approval to conduct a new information collection for a study entitled, “Child Maltreatment Prevention through Policy Change,” over a period of 2 years (2014-2016). Child maltreatment (CM) is a serious public health problem in the United States, with more than 3 million reports of child maltreatment received by state and local agencies each year (United States Department of Health and Human Services, 2011). The estimated lifetime cost of CM is a staggering $124 billion per year, as CM can affect long-term brain development, and leaves children vulnerable to mental/emotional and physical health problems later in life, such as substance abuse, obesity, and heart disease (Fang et al., 2012).

A key strategy in preventing child maltreatment is promoting safe, stable, and nurturing relationships (SSNR) and environments, particularly through policy interventions. To support this strategy, CDC is contracting with ICF International to conduct case studies and an outcome evaluation to better understand if and how county-administered policy strategies of the Temporary Assistance for Needy Families (TANF) program result in lower rates of child maltreatment and associated child welfare outcomes. TANF is the hallmark Federal policy providing resources to States so that they may assist low-income families with financial support and employment services. Understanding how service integration between TANF and child welfare affects child maltreatment may be very important to improving CDC’s ability to devise and implement effective population-based prevention policies.

In Colorado, child welfare services are county-administered and State-supervised by the Division of Child Welfare (DCW) within the Colorado Department of Human Services (DHS). While the Colorado Welfare Settlement Agreement of 1994 mandated more statewide standards in the services that each county was to make available, counties still have the flexibility to provide services directly or through a contract with other private or public providers. Colorado county departments of human services have the flexibility to use a portion of the available TANF funds for services that support participants and may be provided directly by a county or indirectly through contracted community partners.

It is in this policy context that counties can create county innovation in service delivery and program model. El Paso County in Colorado is an example of how this type of innovation can be created and can take hold. El Paso County is a community that had a vision for change in how they delivered their TANF and child welfare programs starting in 1997 and then acted on that vision (Berns & Drake, 1999; Hutson, 2003; Gardiner & Turner, 2006; Tungate, 2008). Following the passage of welfare reform (P.L. 104-193), jurisdictions such as El Paso County used the legislative mandate and funding as an opportunity to declare a formal initiative to “eliminate poverty and family violence.” This involved policy changes that resulted in a family-centered approach that co-located income and child welfare workers, mandated a single case plan for families involved with the child welfare and income support systems, connected younger/teen parents to prevention and employment services, created support groups for kinship caregivers, and creatively leveraged policies and funding to tear down system barriers in support of families and children. While this effort focused to some degree on families that had already reached the door of the public assistance and/or the child welfare systems, after initiating their new policies and procedures, El Paso County reportedly reduced the number of children and families needing formal involvement with the child welfare system. Although these claims were not in the context of a study design that would allow for assumptions about causation, El Paso suggested their new service model resulted in one third fewer foster care placements, a drop of 40% in institutional placements, a 50% reduction in abuse and neglect court filings, and a quadrupled number of adoptions (Hutson, 2003).

The proposed data collection aims to understand if, in Colorado, county-administered policy strategies of the TANF program result in lower rates of child maltreatment (CM) and associated child welfare outcomes. This evaluation and proposed data collections will occur in two parts. First, the case study component of the project will describe how and to what extent nine selected counties in Colorado explore, adopt, and implement an integrated welfare and child welfare service model between the years 1995-2014. The second component of the project will explore whether an integrated TANF-CW service model is related to reductions in CM and related outcomes over time. We are seeking approval for the first component of the study, the case studies. The CM data for the outcome evaluation is publically available and will be accessed and managed by the contractor. Information related to the outcome evaluation is provided throughout this justification for context and to provide a more complete picture of the ways in which the data collected during the case studies will be used.

The details of the case study data collection design and samples are in Section 2. In summary, 190 Colorado state and county employees and partners form the sample population. Specifically, state- and county-level employees working in welfare and/or child welfare agencies will be invited to complete a brief survey and an hour-long semi-structured interview. This study population includes individuals employed in the following positions: County-Level Child Welfare Workers, State-Level Administrators, County Directors of Human Services, Child Welfare Services and Colorado Works Leadership/Manager, Child Welfare Services and Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff. An additional 72 individuals employed as Data Managers, employed by Allied Staff (e.g., Housing, Supplemental Nutrition Assistance Program, Medicaid, Child Care) and Partners of Child Welfare and Colorado Works will also be invited to complete an hour-long semi-structured interview. The sample recruitment and data collection procedures described in this proposal describes data collected by a contractor (ICF International).

Previous systematic literature reviews on this topic have established there are no significant existing studies in this area, and that there is a clear need for a new instrument and data collection on this topic.

The proposed data collection fits into the National Center for Injury Prevention and Control Research Agenda Priorities in Preventing Child Maltreatment (http://www.cdc.gov/injury/ResearchAgenda/index.html) with regard to Tier 1 Part D to “Evaluate the effectiveness of public and organizational policies for preventing child maltreatment and promoting SSNRs” and Tier 2 Part G to “Evaluate the impact of extreme community and environmental stressors on child maltreatment.”

Authority for CDC’s National Center for Injury Prevention and Control to collect this data is granted by Section 301 of the Public Health Service Act (42 U.S.C. 241) (Attachment A). This act gives federal health agencies, such as CDC, broad authority to collect data and do other public health activities, including this type of study.

Privacy Impact Assessment

1. Overview of the Data Collection System

Data collection will be conducted by qualified individuals employed by the contractor, ICF International. Data collectors will have extensive training in data collection procedures, including survey and semi-structured interview administration. The following steps will be implemented by CDC to safeguard the objectivity of the evaluation: 1) all data collectors (also referred to as the case study team) will receive human subjects training; 2) documents will be developed to support data collection which contain standardized responses to common questions (i.e., a Frequently Asked Questions document with standardized responses to be utilized by data collectors); and 3) the contractor, with CDC oversight, will conduct site visits, will hold weekly or bi-weekly conference calls with the data collectors to provide oversight and discuss data collection procedures.

As the ICF International principal investigator on the ICF IRB protocol, Dr. Catherine Lesesne is responsible for overseeing the scientific and human subjects integrity of the study.

Sample and Recruitment. The nine Colorado counties in the proposed data collection are: Adams, Arapahoe, Boulder, Denver, El Paso, Jefferson, Larimer, Pueblo, and Weld. This sample includes the county which first adopted and fully implemented an integrated welfare and child welfare service model (El Paso), as well as all Colorado counties of comparable population size and similar demographic and geographic profiles. Colorado has 64 counties in total; 11 of these counties had Census population estimates of over 100,000 for the year 2000. (Exhibit 1; 2000 census estimates are shown here to gauge population sizes proximal to the study baseline year). Two of these eleven counties, Douglas and Mesa, will be omitted from the study. Douglas was not included here because its median household income was much higher and poverty rate was much lower than El Paso and other comparison counties. Mesa was not included because its population was substantially smaller, and it is located in the Western Slope of Colorado, which separates it from the other select counties (see map in Exhibit 2). The remaining eight comparison counties included in this study are the only counties in the State of Colorado with population size over 100,000 people with reasonably comparable demographic characteristics and represent a convenience sample of counties similar to El Paso.

Exhibit 1. 2000 Census Estimates of Select Colorado Counties with Population Greater than 100,000

Colorado Counties with Population Estimates Greater Than 100,000	2000 Population Estimate	2010 Population Estimate	2010 Median Household Income	4/2012 Percent Unemployment Rate	2006–2010 Percent Below Poverty
Counties included in this study
Adams County	363,857	441,603	$52,785	9.4%	13.9%
Arapahoe County	487,967	572,003	$58,152	7.7%	11.6%
Boulder County	291,288	294,567	$62,215	5.9%	12.8%
Denver County	554,636	600,158	$45,415	8.6%	19.2%
El Paso County	516,929	622,263	$51,553	9.2%	11.1%
Jefferson County	527,056	534,543	$64,181	7.5%	8.0%
Larimer County	251,494	299,630	$54,739	6.4%	13.3%
Pueblo County	137,337	159,063	$40,699	10.4%	17.3%
Weld County	180,936	252,825	$52,334	8.8%	13.9%
Counties not included in this study
Douglas	175,766	285,465	$97,806	6.1%	2.9%
Mesa	116,255	146,723	$52,067	9.10%	12.40%
State & National
Colorado	4,301,261	5,029,196	$54,411	8.0%	12.2%
United States	281,421,906	308,745,538	$50,046	8.1%	13.8%
Source: U.S. Census Bureau (2000, 2010) and Bureau of Labor Statistics (2010, 2012)

Exhibit 2. Map of Colorado Counties (counties included in this study shown in yellow)

A systematic process will be used to conduct outreach, invite counties to participate in the research and engage points of contact in conducting the case study protocol. The contractor has identified initial county point-of-contacts through environmental scans from reports, organizational charts, and county Web sites. The initial points-of-contact will be asked to, in conjunction with their leadership, identify staff who will best serve as county liaisons for this project (9 total, one for each county). Each liaison will serve as a liaison between his or her agency and the contractor.

Once county liaisons have agreed to serve in this role, the contractor will ask each county liaison for assistance in identifying key staff with whom they should speak in order to gather historical and current information on the level of implementation of integrated TANF and child welfare service delivery occurring in each county. The liaisons from each county will help the contractor identify the appropriate respondents for both the brief survey and the interview. The contractor will work closely with county liaisons to schedule the interviews and make other logistical arrangements for the site visit. Using this process, the contractor will identify a sample of key informants to invite to participate in the surveys and interviews. The key informants will be selected to ensure diversity in role and agency/operating division (i.e., informants from child welfare and TANF programs, variety in role and position from leadership to direct service as well as allied and external partners). Once the list of key informants is received, the site visitors will select up to 18 persons to invite to participate in brief surveys and key informant interviews.

The primary inclusion criterion for site informants is the recommendation and/or approval for inclusion by the county liaison and fulfilling a job description that fits the titles/descriptions outlined below:

State-level/Field Administrator (8 informants from the state-level): This person provides administrative consultation and a direct connection between the State and other county departments. This includes Colorado State staff, such as the Colorado Works Director, Field Administrators, and administrators from the Colorado Department of Human Services.

County Directors of Human Services (18 site informants, 2 per county): This person has general oversight of the TANF or Child Welfare program. This person usually is involved in higher level conceptual planning and developing the vision for the program, sometimes in securing and managing funding as well. This person usually is in charge of other staff working in TANF or Child Welfare.

Child Welfare/Colorado Works Leadership/Manager (36 site informants, 4 per county): This category of persons has direct oversight of case managers, caseworkers, technicians, and other client-serving staff. These persons serve in a leadership role and are responsible for the day-to-day management and work closely with the program and to oversee the activities of case workers, social workers, technicians, and other staff.

Child Welfare/Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff (54 site informants, 6 per county): This category includes people who carry out the daily activities of the program, such as a case manager, social worker, and other staff.

Allied Staff (36 site informants, 4 per county): This category of staff typically provides auxiliary services to TANF-CW recipients. This includes representatives from the State or County Housing department, Supplemental Nutrition Assistance Program (SNAP), Medicaid, and Child Care services.

Partners (28 site informants, at least 3 per county): Programs may have external/community-based public or private partners who assist with the program activities or its infrastructure in important ways. These could be organizations or specific people who provide various services, resources, and/or funding to the program.

Data Managers (10 site informants, at least 1 per county): This category of staff typically include staff who are in charge of overseeing the data management efforts (for TANF/child welfare services) for the county.

Potential informants who do not speak English will be excluded from participation; however, we expect that this will be rarely if at all applicable in this study. Further, in order to participate, individuals must have worked in the agency for at least one year – this is to ensure that site informants have had a period of time during which to become familiar with the agency roles and responsibilities. No other formal exclusion criteria will be employed.

2. Items of Information to be Collected

The case studies include a brief web-based survey and semi-structured interviews. The purpose of the case studies and interviews are to describe the extent and nature of current and past service integration between TANF and child welfare within nine Colorado county departments of human and social services and to inform the development and refinement of an Implementation Index, which is a tool which can be used to systematically determine the level of integration between welfare and child welfare service delivery. Each of these components will be described in turn.

Web-based survey. Approximately 13 individuals from each county (individuals employed in the following positions: County-Level Child Welfare Workers, State-Level Administrators, County Directors of Human Services, Child Welfare Services and Colorado Works Leadership/Manager, Child Welfare Services, Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff) will be asked to complete a brief web-based survey (Attachment D). This survey is an adapted measure previously fielded in these counties (Tungate, 2008); specifically, the survey developed by Tungate (2008) has been expanded to include additional items relevant to the current study particularly related to addressing the domains that we believe will be important in service of developing the Implementation Index. A brief web-based survey (Attachment E) will also be administered to approximately 8 State-level staff to collect information about integration. The survey will be administered before the site visit by sending a secure link via email to those individuals identified by the site liaisons. An informed consent statement will be presented on the initial screen of the survey.

Semi-structured Interviews. In addition to the brief survey, the two case study team members for each county and State will conduct 190 in-person (or over the phone, if needed) semi-structured interviews during a site visit with the key informants: County Directors of Human Services (Appendix F), State-Level Administrators (Appendix P), Child Welfare/Colorado Works Leadership/Manager (Appendix G), Child Welfare/Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff (Appendix H), Allied Staff (Appendix I), Partners (Appendix J), and Data Managers (Appendix Q).

2. Purpose and Use of the Information Collection

The information collected under the proposed data collection will be used to:

1. Understand how a state policy allowing counties to administer TANF programs with flexibility contributes to county-level adoption of integrated welfare and child welfare service models. Following completion of the project, information will be disseminated to stakeholders on how to develop, resource, implement and sustain an integrated welfare-child welfare service delivery system. No data source currently exists to provide this guidance to stakeholders. (Primary Aim 1)

2. Develop and refine an Implementation Index, which will quantify the degree of integration between welfare and child welfare services. Following completion of the project, the Implementation Index will be made publically available for use by state and county governments to assess the degree to which their welfare and child welfare systems are integrated (i.e., collaborating and working together toward a common goal). No index or tool of this nature currently exists. (Primary Aim 2)

3. Inform the larger outcome evaluation; that is, data collected during the case studies will form the independent variable in the subsequent outcome evaluation, which examines whether TANF policies and program supports reduce rates of child maltreatment when they are delivered in an integrated welfare and child welfare service model. Results regarding the impact of integrated welfare-child welfare service delivery on county-level rates of child maltreatment will be disseminated to a broad audience (e.g., researchers, federal, state and county-level government stakeholders). (Primary Aim 3).

The prevention of child maltreatment is perhaps the most important and most difficult tasks facing public health. The legal, interpersonal, social, physical health and psychological consequences of child maltreatment are far-reaching and are a constant strain on individuals and society. Expanding the understanding of effective policies and programs to prevent child maltreatment and related outcomes builds the evidence base, a goal which will benefit children and families at risk of child maltreatment in addition to those who have already experienced this form of violence. The data collected from participants during the case studies will allow us to develop and populate the Implementation Index, which will in turn be used in the outcome evaluate to determine how effective an integrated welfare-child welfare service model is to prevent child maltreatment. There are numerous negative consequences of not obtaining the proposed data; not obtaining the proposed data would preclude: the provision of guidance to stakeholders on how to develop, resource, implement and sustain an integrated welfare-child welfare system, the development the Implementation Index, and building the evidence base regarding effective policies that lead to a reduction in child maltreatment.

Privacy Impact Assessment Information

1. How information will be shared and for what purpose

The purpose of this data collection request is to understand how a state policy allowing counties to administer TANF programs with flexibility contributes to county-level adoption of integrated welfare and child welfare service models, to develop and refine an Implementation Index, and to inform the outcome evaluation which examines the impact of integrated welfare and child welfare service models on rates of CM. Personally identifiable information to be collected during the semi-structured interview is the participant’s name, job title, and length of service/time at current employer, which will be used to frame the semi-structured interview. The contractor will make arrangements before the interview to conduct the interview in a quiet place that provides privacy (or over the phone, if necessary). Interviews will be audio recorded with permission and transcripts will be developed for each interview. The transcripts will be cleared of identifiers and housed in an ATLAS.ti database on encrypted, password protected electronic storage files. ATLAS.ti is a suite of software that supports analysis of qualitative and/or unstructured data. Personally identifiable information to be collected on the survey is the participants’ job title and length of service/time at current employer. E-mail addresses will also be obtained in order to send out the link to the survey. The survey data will also be housed in a database on encrypted, password protected electronic storage files.

2. Impact of the proposed collection on respondents’ privacy

The impact of this data collection on participants’ privacy is very low since no information that could link a participants’ name to his/her interview will be released to anyone outside the project team (e.g., site visitors) and the risk of breach of privacy is minimal.

3. Use of Improved Information Technology and Burden Reduction

We will utilize advanced technology to collect and process data to reduce respondent burden and make data processing reporting more timely and efficient. In all data collections, the number of questions will be held to the absolute minimum required for the intended use of the data.

Surveys will take place online using electronic survey forms. The semi-structured interviews will be administered using paper and pencil format; however, they will be audio recorded to reduce the amount of time the site visitor needs to spend recording the respondents’ responses. Screen shots of all questions to be administered electronically are included in Attachment K (Survey of County Level TANF and Child Welfare Respondents) and Attachment L (Survey of State Level Administrators).

4. Efforts to Identify Duplication and Use of Similar Information

Consultation with experts in the field in addition to a literature search (conducted in July 2012 of the MEDLINE and PsycINFO databases) found no evidence of case studies, evaluation, or any other type of research documenting how Colorado’s policy allowing counties to administer TANF programs with flexibility contributes to county-level adoption of integrated welfare and child welfare service models. Further, this search found no documentation of the existence of an Implementation Index (or index/tool of any other name) that helps stakeholders and researchers quantify the degree of integration between welfare and child welfare services. No publically available data on this topic exists and as such no other existing data may be used to assess the variables of interest in the current proposal. Thus, a new information collection is requested.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

6. Consequences of Collecting the Information Less Frequently

The present study only requires one wave of data collection; that is, respondents will be asked to complete one survey and one semi-structured interview. Thus, repeated data collection is not necessary for this study. If this single wave of data collection is not conducted, we will not be able to provide stakeholders information on how to develop, resource, implement and sustain an integrated welfare-child welfare delivery system, including a tool (Implementation Index) that allows stakeholders to assess integration.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. Federal Register

A 60-day Federal Register Notice was published in the Federal Register on September 3, 2013, Vol. 78, No. 170, pp. 54253-54 (see Attachment B). There was one public comment received. The standard CDC response was sent.

B. Efforts to consult with persons outside the agency

Formative, consultation work was conducted to identify promising policies that may play a role in preventing child maltreatment. The following list identifies those individuals consulted by CDC:

Theresa Rafael, Executive Director, National Alliance of Children’s Trust and Prevention Funds, [email protected]

Charlie Bruner, Executive Director, Iowa Child and Family Policy Center, [email protected], phone: (515) 280-9027

Lisbeth Schorr, Director, Harvard Project on Effective Interventions, Senior Fellow, Center for the Study of Social Policy, [email protected] 

Martha Reeder, Project Director, Early Childhood Coordinating System and Strengthening Families, Arizona Department of Health and Human Services, Early Childhood Coordinating System and Strengthening Families,[email protected]

Judy Langford, Director, Strengthening Families Initiative, Center for the Study of Social Policy, [email protected]

James Krieger, Public Health Department in Seattle & King County, phone: 206-263-8227
[email protected]
Fred Wulczyn, Chapin Hall Center for Children, Phone Number: 

773.256.5212

Ajay Chaudry, Urban Institute, Phone: (202) 261-5709

The following list identifies those individuals consulted by ICF International in the development of the specific case study aims and methods:

Lloyd Malone, Former Director, Colorado Department of Human Services, Division of Child Welfare Services, Phone: 303-866-5932

Jonathan Sushinsky, Manager, Research, Evaluation, and Data, Colorado Department of Human Services, Division of Child Welfare Services, [email protected]

Susan Tungate, Director of Field Education, School of Social Work, Colorado State University, Phone: (970) 491-4695, [email protected]

Kimber Johnson, MOS Master Instructor, AFCARS/NCANDS Federal Liaison, Colorado Department of Human Services, Division of Child Welfare Services, [email protected]

9. Explanation of Any Payment or Gift to Respondents

Small gifts will be provided to individual interview respondents in recognition of their time to respond to this data collection request. A $25 gift card will be provided to each respondent who participates in the interviews, including those who choose to stop or discontinue the interview.

All participant gifts have been approved by the ICF International IRB. The IRB Approval Form is provided in Attachment C.

10. Assurance of Confidentiality Provided to Respondents

This submission has been reviewed by CIO who determined that the Privacy Act does not apply.

We will obtain informed consent for both the county- and state-level surveys (Attachments D and E) and the semi-structured interviews (Attachments F, G, H, I, J, P and Q). The consent statement that will be read aloud to the interview respondents can be found in Attachment S. Respondents will also be given a paper copy that details project information and also includes consent information (Attachment O). All respondents will be informed that their participation in the survey and semi-structured interviews are completely voluntary. All surveys and interviews will be conducted in a secure manner. The researchers will not identify participants by name in any reports, and participant information will remain secure. No quotes of survey responses will be tied to any individual in our reporting, and efforts will be made protect the anonymity of the individuals and county in subsequent uses of the data. As stated in the informed consent, participation—or decline of participation—in this project will not affect their job or relationships with colleagues. The interviewee's name will not be associated with specific quotes or comments. In addition, the case study report will be written in a way in which no comments will be attributed to any one person. Minimal information of a personal or sensitive nature will be collected (e.g., duration of employment at agency, role), but this will be reported in aggregate. During the interviews, the site visitor will take detailed notes and audio record the discussion. Data from the interviews will be kept with the site visitor at all times while on site or in a secured storage, and following transcription, interviews will be entered into a secure study-developed database and imported into ATLAS.ti for systematic analysis.

This project will implement the following standard practices and procedures to protect the security and nondisclosure of information:

• All site visitors and transcribers will be asked to sign a nondisclosure agreement (Attachment M) and confidentiality agreement (Attachment N).

• All project team members will be trained on the project's specific nondisclosure and security requirements.

• During data collection in the field, site visitors will maintain data collection materials (notes, audio recordings, or program/innovation materials) in their possession or in secured storage at all times until their reports are written. Site visitors will be instructed to destroy notes upon completion of the reports. Destruction of all hard copy documents will be accomplished by shredding.

• Three years after the project expires or is terminated, the contractor’s project director, in consultation with the client, will supervise the authorized destruction (shredding) of all hard copy documents (e.g., notes, document review materials) and deletion of all electronic files and/or audio recordings.

• The contractor will deliver to CDC data resulting from the case studies in de-identified data files and transcripts.

IRB Approval

The project contractor, ICF International, has obtained local IRB approval to collect data from study participants. IRB approval expires on August 15, 2015, with the next annual review date of April 17, 2014

Privacy Impact Assessment Information

1. This project is not subject to the Privacy Act. Respondents are acting in their official roles.

2. Data that are collected will be stored physically and electronically by the contractors collecting the respective data at their offices. Electronic databases will be transferred to CDC on an annual basis. Hard copies of data will be destroyed after the data has been successfully entered, cleaned and backed up.

3. Respondent consent will be obtained prior to data collection. Verbal informed consent will be obtained for all semi-structured interviews (Attachment S) and electronic informed consent will be obtained for the web-based surveys. All interviewers/site visitors will use a script to obtain verbal consent for the semi-structured interviews (Attachment S). A screenshot of the respondent consent form for the web-based survey is attached (Attachments K and L). In the beginning of the semi-structured interview, the interviewer will read an informed consent statement (see Attachment S). This statement describes the purpose of the study, how the information will be used, and the steps that will be taken to protect participant information. Interviewers will provide participants with specific contact information for the contractor’s project director should participants have any questions once the interview is complete. The study information, including content of the consent statement and the contractor’s project director contact information will be left with the participant to take with them (Attachment O).

4. Participants will be informed that the interview and survey are voluntary and that they may choose to discontinue participation in either the interview or the survey at any time for any reason. If a participant chooses to stop the interview, the site visitor will ask the respondent whether he or she wishes to withdraw all of the data (his/her responses) that has already been contributed. If he/she chooses to stop the interview, but allows the data already collected to be used, the site visitor will thank the respondent for their participation and follow data security and handling procedures as for a completed interview. If the respondent chooses to withdraw all data, the site visitor will thank the respondent for their time and end the interview. Site visitors will then erase the audio recording, shred any handwritten notes, and not type or share these responses.

11. Justification for Sensitive Questions

No sensitive questions will be asked as part of this data collection request.

12. Estimates of Annualized Burden Hours and Costs

Burden estimates were derived based on the number and nature of the questions, and the administration methods (e.g., open-ended questions).

A.12.A. Burden

Table A-12 details the annualized number of respondents, the average response burden per survey, the average response burden per interview, and the total response burden. Estimates of burden for the survey are based on simulated runs with staff answering each survey and interview, in addition to a test of the protocol and surveys with less than 9 county-level staff. We anticipate that the survey will take approximately 15 minutes to complete and that each semi-structured interview will take approximately 1 hour to complete. For the survey, 126 project participants will respond to the survey once, where each response requires 15 minutes: 126 (responses total) x 1 (responses per total project period) x 15/60 (hour per response) = 32 total survey burden hours (16 hours per year). For the semi-structured interview, 190 project participants will respond to the interview once, where this response requires 1 hour = 190 (responses total) x 1 (responses total per project period) * 1 (hour per response) = 190 total semi-structured interview burden hours. The total burden for this proposed data collection: 32 burden hours for the survey + 190 burden hours for the semi-structured interview = 222 total burden hours. The estimated annual burden hours per year for 2 years is 111 hours.

Table A.12- Estimate of Annual Burden Hours.

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Avg. Burden per Response (in hrs)	Total Burden (in hrs)
County Directors of Human Services	Survey of County TANF and Child Welfare Respondents, (Attachment D)	9	1	15/60	2
	Interview Guide of County Director of Human Services (Attachment F)	9	1	1.0	9
State Level Administrators	Survey of State Level Administrators (Attachment E)	4	1	15/60	1
	Interview of State Level Administrator (Attachment P)	4	1	1.0	4
Child Welfare/Colorado Works Leadership/Manager	Survey of County TANF and Child Welfare Respondents (Attachment D)	18	1	15/60	5
	Interview of Child Welfare/Colorado Works Leadership/Manager (Attachment G)	18	1	1.0	18
Child Welfare Services and Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff	Survey of County TANF and Child Welfare Respondents (Attachment D)	27	1	15/60	7
	Interview Guide of Child Welfare and Colorado Works Case Manager, Caseworker, Technician and Other Client-Serving Staff (Attachment H)	27	1	1.0	27
Allied Staff (e.g., Housing, Supplemental Nutrition Assistance Program, Medicaid, Child Care)	Interview of Allied Staff (e.g., Housing, Supplemental Nutrition Assistance Program, Medicaid, Child Care) (Attachment I)	18	1	1.0	18
Partners of Child Welfare and Colorado Works	Interview Guide for Partners (Attachment J)	14	1	1.0	14
County Data Manager	Survey of County TANF and Child Welfare Respondents (Attachment D)	5	1	15/60	1
	Interview of Data Managers (Attachment Q)	5	1	1.0	5
	Total				111

A.12.B. Estimated Annualized Burden Cost

The hourly wage used to calculate the respondent costs are based on professions of comparable experience using the 2012 Department of Labor wage tables (www.dol.gov). Estimated Annualized Burden Cost for this evaluation is $2,185.82 (Total Respondent Cost for this evaluation for 18 months is $4,371.64).

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Total Burden (in hrs)	Hourly Wage Cost	Respondent Cost
County Directors of Human Services	Survey of County TANF and Child Welfare Respondents (Attachment D)	9	1	2	$30.36	$60.72
	Interview Guide of County Director of Human Services (Attachment F)	9	1	9	$30.36	$273.24
State-Level Administrators	Survey of State-Level Administrators (Attachment E)	4	1	1	$21.88	$21.88
	Interview of State-Level Administrator (Attachment P)	4	1	4	$21.88	$87.52
Child Welfare/ Colorado Works Leadership/ Manager	Survey of County TANF and Child Welfare Respondents (Attachment D)	18	1	5	$20.14	$100.70
	Interview of Child Welfare/Colorado Works Leadership/Manager (Attachment G)	18	1	18	$20.14	$362.52
Child Welfare Services and Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff	Survey of County TANF and Child Welfare Respondents (Attachment D)	27	1	7	$17.21	$120.47
	Interview Guide of Child Welfare and Colorado Works Case Manager, Caseworker, Technician and Other Client-Serving Staff (Attachment H)	27	1	27	$17.21	$464.67
Allied Staff (e.g., Housing, Supplemental Nutrition Assistance Program, Medicaid, Child Care)	Interview of Allied Staff (e.g., Housing, Supplemental Nutrition Assistance Program, Medicaid, Child Care) (Attachment I)	18	1	18	$15.51	$279.18
Partners of Child Welfare and Colorado Works	Interview Guide for Partners (Attachment J)	14	1	14	$16.21	$226.94
County Data Manager	Survey of County TANF and Child Welfare Respondents (Attachment D)	5	1	1	$31.33	$31.33
	Interview of Data Managers (Attachment Q)	5	1	5	$31.33	$156.65
Total	$2,185.82

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Respondents will incur no capital or maintenance costs.

14. Annualized Cost to the Federal Government

Contractual costs:

This is a contracted data collection, led by ICF International under contract for CDC. The total cost of the contract over the 18 months of data collection is $447,867, with an annualized cost to the government of $223,933.50

Year	Budget
Personnel Costs	$193,385
Travel	$27,107
Participant $25 Gifts (n = 95)	$2,375
Year 1 Total	$222,867
Personnel Costs	$195,625
Travel	$27,000
Participant $25 Gifts (n = 95)	$2,375
Year 2 Total	$225,000
TOTAL	$447,867
Annualized Contractual Cost	$223,933.50

Federal employee costs:

NCIPC has assigned a Project Officer and Science Officer to assist with and oversee this data collection. A CDC project officer (GS-11) and science officer (GS-13) devote 20% of their FTE for an estimated cost of $32,000 per year.

Year	Budget
Year 1	$32,000
Year 2	$32,000
TOTAL	$64,000

Year 1 Contract Cost + CDC Labor = $254,867

Year 2 Contract Cost + CDC Labor = $257,000

Total project cost for Years 1 & 2 is $511,867. The average annualized cost is $255,933.50.

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

A.16.A. Tabulation and Analysis Plan:

The purpose of the case studies is to describe the extent and nature of current and past service coordination and collaboration between TANF and child welfare within nine Colorado county departments of human and social services and at the State level. The case studies will closely examine the current and historical experience of TANF and child welfare integration using the 1999 El Paso County model as described by Hutson (2003) as the gold standard of TANF-CW service delivery. As noted earlier, the case studies will employ a range of qualitative methods including a brief survey and in-person interviews with key leaders and staff in each county. The results of the case studies will be used to develop an Implementation Index and then assign each county an intervention status (i.e., a level of integration) to serve as independent variables for the quantitative outcome assessment. Specific details regarding methods and plans for analysis are outlined below.

The qualitative data obtained from the semistructured interviews will be summarized and analyzed using a thematic approach and systematic coding using qualitative analysis software (ATLAS.ti). Results from respondents within each county will be examined individually and in aggregate for the presence or absence of evidence of integrated TANF-child welfare service models. The results of the analysis of interviews will be combined with data from the survey.

On the basis of the recommendations offered by Miles and Huberman (1994), the researchers will use a process that entails (1) data capture, (2) codebook development, (3) intercoder reliability assessment, (4) code application using ATLAS.ti and coding guidelines developed specifically for this study, and (5) detailed documentation of themes , by counties, and by respondent categories of interest. Each of these activities is described in more detail below.

Data capture. The audiotapes from semi-structured interviews with key informants will be transcribed and imported into ATLAS.ti, which facilitates the marking and subsequent search, retrieval, classification, and cross-classification of the text.

Codebook development. A study codebook will be developed that defines the codes to be used to code and retrieve data in the analysis phase. An initial list of coding categories will developed and based on the study questions and components in the implementation index. The codes and definitions will be guided by our research questions and will relate to statements about TANF and child welfare integration, evidence of integration, perceptions about fidelity to integration, program changes, and policies that help or hinder integration For example, codes could include vision, leadership, communications, data systems, funding, etc.

Intercoder Reliability. To establish intercoder reliability, the principal investigator will select segments of text from interview transcripts for the research team to review and code. The team will compare their codings for each transcript, and a Fleiss’ kappa statistic will be calculated to assess intercoder reliability. The team will meet to discuss the process, resolve differences, redefine existing codes as necessary, and ascertain the need for new codes. The team will aim for a Fleiss’ kappa of at least 0.80.

Code application. Once reliability is established, codes will be applied to the remaining transcripts before analysis. The coded data will be extracted and analyzed for themes, patterns, and interrelationships relevant to the study objectives. We will examine the commonalities across cases in how they approach innovation within their TANF and child welfare service systems. We will also look within the sample for cases sharing common approaches or intervention status later in the time series (in more recent years) in order to examine barriers and facilitators of their program model implementation.

Documentation of themes. Upon completion of onscreen coding, ATLAS.ti will used to search, retrieve, classify, and cross-classify the coded data. This will allow for the conduct of within-case and cross-case analyses to determine thematic and content consistency and variability. Both thematic and content analyses will be used, which allows for identification themes, patterns, relationships, differences, and level of child welfare-TANF integration in the data. These themes will be incorporated into the development of the Implementation Index. Within counties and across counties, patterns by respondent type (e.g., case workers, partners) and years will be examined.

Surveys. A brief web-based survey will be used as the first data collection effort with county respondents. The survey is an adapted measure previously fielded in these counties (Tungate, 2008); the survey developed by Tungate 2008 has been expanded to include additional items relevant to the current study particularly related to addressing the domains presented in the implementation index. The quantitative data gathered from the county and state surveys will be imported and analyzed in SPSS to generate frequencies, means, standard deviations, as appropriate for the response set. For the item using a Likert-type rating scale to assess perceived barriers to integration, individual items and summative scale means will be examined. Frequencies on all other items will be assessed across respondents within each county. Responses to the specific survey question will be used to complement the interviews and other data to help categorize counties in level of implementation (using the implementation index) across the years of the study. In addition, for the cross-case analysis, the research team will examine descriptive statistics (frequencies) for specific questions across counties.

Triangulation of qualitative (interview) and quantitative (survey) data. Because of the mixed methods approach to this study, many of the variables of interest related to each research question will be informed both by qualitative data from interviews and quantitative data from the survey. A significant aspect of data analysis will involve the triangulation of data from these mixed methods. The exploratory nature of this study allows for the data to be viewed as complementary—creating a richer picture of child welfare and TANF integration from the perspective of various respondents at the counties and State. After analyzing each set of qualitative and quantitative data, the team will examine data across methods, noting similarities and differences in the identified themes as found in the interviews and surveys. Together this information will create a fuller description of how child welfare and TANF collaborate (or don’t collaborate) at the county and State levels. Both the interview and survey data will be used to develop, refine, and complete the Implementation Index.

A.16.B. Publications

The results of the case studies and subsequent outcome evaluation will be reported in peer-reviewed journal articles, conference presentations, research briefs, and Web-based papers for dissemination to researchers, states, and the public. Materials related to the Implementation Index will be made publically available on the CDC website.

Table A.16-1. Time Schedule

Activity	Time schedule
Recruitment of study participants	1 month after OMB approval
Participants complete online survey Conduct site visits in all nine counties Conduct semi-structured interviews with participants	1-24 months after OMB approval

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is not inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification

References

Berns, D.A. & Drake, B.J. (1999). Combining child welfare and welfare reform at a local level. Policy & Practice, March, 26-34.

Fang, X., Brown, D.S., Florence, C.S., & Mercy, J.A. (2012). The economic burden of child maltreatment in the United States and implications for prevention. Child Abuse & Neglect, 36(2), 156-165.

Gardiner, K. & Turner, L., The Lewin Group (2006, June 21). Family preventative services in Colorado (Report prepared for the State of Colorado Department of Human Services). Falls Church, VA: The Lewin Group.

Hutson, R. (2003). A vision for eliminating poverty and family violence: Transforming child welfare and TANF in El Paso County, Colorado. Center for Law and Social Policy: Policy Brief, Child Welfare Series, 1, 1-7.

Tungate, S. L. (2009). Welfare and child welfare collaboration. Dissertation Abstracts International Section A: Humanities and Social Sciences, 70(2-A), p. 706. Ann Arbor, MI: ProQuest Information & Learning.

United States Department of Health & Human Services (2011). Child maltreatment 2011. Washington, DC: U.S. Government Printing Office.

19