Ssb

Prevention of Child Maltreatment through Policy Change

SUPPORTING STATEMENT

INFORMATION COLLECTION REQUEST

Part B

Supported by:

Department of Health and Human Services (DHHS)

Centers for Disease Control and Prevention (CDC)

National Center for Injury Prevention and Control (NCIPC)

Division of Violence Prevention (DVP)

Government Project Officer:

Natasha E. Latzman, Ph.D., Science Officer

Phone# 770-488-3935

Email: [email protected]

Fax# 770-488-4222

3 February, 2014

Table of Contents

B. Collections of Information Employing statistical procedures

1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with Non-Response

4. Test of Procedures or Methods to be Undertaken

5. Individuals consulted on Statistical Aspects and Individuals collecting and/or analyzing information

B. Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

In summary, 188 Colorado state and county employees and partners form the sample population. Specifically, state- and county-level employees working in welfare, child welfare, or partner agencies will be invited to complete a brief survey and/or an hour-long semi-structured interview. This study population includes individuals employed in the following positions: County-Level Child Welfare Workers, State-Level Administrators, County Directors of Human Services, Child Welfare Services and Colorado Works Leadership/Manager, Child Welfare Services and Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff. An additional 72 individuals employed as Data Managers, employed by Allied Staff (e.g., Housing, Supplemental Nutrition Assistance Program, Medicaid, Child Care) and Partners of Child Welfare and Colorado Works will also be invited to complete an hour-long semi-structured interview. The sample recruitment and data collection procedures described in this proposal describes data collected by a contractor (ICF International).

There are nine Colorado counties in the proposed data collection, which include: Adams, Arapahoe, Boulder, Denver, El Paso, Jefferson, Larimer, Pueblo, and Weld. This sample includes the county which first adopted and fully implemented an integrated welfare and child welfare service model (El Paso), as well as all Colorado counties of comparable population size and similar demographic and geographic profiles. Colorado has 64 counties in total; 11 of these counties had total 2000 Census population estimates of over 100,000. (Exhibit 1; 2000 census estimates are shown here to gauge population sizes proximal to the study baseline year). Two of these eleven counties, Douglas and Mesa, will be omitted from the study. Douglas was not included here because its median household income was higher and poverty rate much lower than El Paso and other comparison counties. Mesa was not included because its population was substantially smaller, and it is located in the Western Slope of Colorado, which separates it from the other select counties (see map in Exhibit 2). The remaining eight comparison counties included in this study are the only counties in the State of Colorado with population size over 100,000 people with reasonably comparable demographic characteristics and represent a convenience sample of counties similar to El Paso.

Exhibit 1. 2000 Census Estimates of Select Colorado Counties with Population Greater than 100,000

Colorado Counties with Population Estimates Greater Than 100,000	2000 Population Estimate	2010 Population Estimate	2010 Median Household Income	4/2012 Percent Unemployment Rate	2006–2010 Percent Below Poverty
Counties included in this study
Adams County	363,857	441,603	$52,785	9.4%	13.9%
Arapahoe County	487,967	572,003	$58,152	7.7%	11.6%
Boulder County	291,288	294,567	$62,215	5.9%	12.8%
Denver County	554,636	600,158	$45,415	8.6%	19.2%
El Paso County	516,929	622,263	$51,553	9.2%	11.1%
Jefferson County	527,056	534,543	$64,181	7.5%	8.0%
Larimer County	251,494	299,630	$54,739	6.4%	13.3%
Pueblo County	137,337	159,063	$40,699	10.4%	17.3%
Weld County	180,936	252,825	$52,334	8.8%	13.9%
Counties not included in this study
Douglas	175,766	285,465	$97,806	6.1%	2.9%
Mesa	116,255	146,723	$52,067	9.10%	12.40%
State & National
Colorado	4,301,261	5,029,196	$54,411	8.0%	12.2%
United States	281,421,906	308,745,538	$50,046	8.1%	13.8%
Source: U.S. Census Bureau (2000, 2010) and Bureau of Labor Statistics (2010, 2012)

Exhibit 2. Map of Colorado Counties (counties included in this study shown in yellow)

A systematic process will be used to conduct outreach, invite counties to participate in the research and engage points of contact in conducting the case study protocol. The contractor has identified initial county point-of-contacts through environmental scans from reports, organizational charts, and county Web sites. The initial points-of-contact will be asked to, in conjunction with their leadership, identify staff who will best serve as county liaisons for this project (9 total, one for each county). Each liaison will serve as a liaison between his or her agency and the contractor.

Once county liaisons have agreed to serve in this role, the contractor will ask each county liaison for assistance in identifying key staff with whom they should speak in order to gather historical and current information on the level of implementation of integrated TANF and child welfare service delivery occurring in each county. The liaisons from each county will help the contractor identify the appropriate respondents for both the brief survey and the interview. The contractor will work closely with county liaisons to schedule the interviews and make other logistical arrangements for the site visit. Using this process, the contractor will identify a sample of key informants to invite to participate in the surveys and interviews. The key informants will be selected to ensure diversity in role and agency/operating division (i.e., informants from child welfare and TANF programs, variety in role and position from leadership to direct service as well as allied and external partners). Once the list of key informants is received, the site visitors will select up to 18 persons to invite to participate in brief surveys and/or key informant interviews.

The primary inclusion criterion for site informants is the recommendation and/or approval for inclusion by the county liaison and fulfilling a job description that fits the titles/descriptions outlined below:

State-level/Field Administrator (8 informants from the state-level): This person provides administrative consultation and a direct connection between the State and other county departments. This includes Colorado State staff, such as the Colorado Works Director, Field Administrators, and administrators from the Colorado Department of Human Services.

County Directors of Human Services (18 site informants, 2 per county): This person has general oversight of the TANF or Child Welfare program. This person usually is involved in higher level conceptual planning and developing the vision for the program, sometimes in securing and managing funding as well. This person usually is in charge of other staff working in TANF or Child Welfare.

Child Welfare/Colorado Works Leadership/Manager (36 site informants, 4 per county): This category of persons has direct oversight of case managers, caseworkers, technicians, and other client-serving staff. These persons serve in a leadership role and are responsible for the day-to-day management and work closely with the program and to oversee the activities of case workers, social workers, technicians, and other staff.

Child Welfare/Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff (54 site informants, 6 per county): This category includes people who carry out the daily activities of the program, such as a case manager, social worker, and other staff.

Allied Staff (36 site informants, 4 per county): This category of staff typically provides auxiliary services to TANF-CW recipients. This includes representatives from the State or County Housing department, Supplemental Nutrition Assistance Program (SNAP), Medicaid, and Child Care services.

Partners (27 site informants, 3 per county): Programs may have external/community-based public or private partners who assist with the program activities or its infrastructure in important ways. These could be organizations or specific people who provide various services, resources, and/or funding to the program.

Data Managers (9 site informants, 1 per county): This category of staff typically include staff who are in charge of overseeing the data management efforts (for TANF/child welfare services) for the county.

Potential informants who do not speak English will be excluded from participation; however, we expect that this will be rarely if at all applicable in this study. Further, in order to participate, individuals must have worked in the agency for at least one year – this is to ensure that site informants have had a period of time during which to become familiar with the agency roles and responsibilities. No other formal exclusion criteria will be employed.

B.2. Procedures for the Collection of Information

Data collection will be conducted by qualified individuals employed by the contractor, ICF International. Data collectors will have extensive training in data collection procedures, including survey and semi-structured interview administration. The following steps will be implemented by CDC to safeguard the objectivity of the evaluation: 1) all data collectors (also referred to as the case study team) will receive human subjects training; 2) documents will be developed to support data collection which contain standardized responses to common questions (i.e., a Frequently Asked Questions document with standardized responses to be utilized by data collectors); and 3) the contractor, with CDC oversight, will conduct site visits, will hold weekly or bi-weekly conference calls with the data collectors to provide oversight and discuss data collection procedures.

As the ICF International principal investigator on the IFC IRB protocol, Dr. Catherine Lesesne is responsible for overseeing the scientific and human subjects integrity of the study.

The case studies include a brief web-based survey and semi-structured interviews. The purpose of the case studies and all three types of data collection are to describe the extent and nature of current and past service integration between TANF and child welfare within nine Colorado county departments of human and social services and to inform the development and refinement of an Implementation Index, which is a tool which can be used to systematically determine the level of integration between welfare and child welfare service delivery. Each of these components will be described in turn.

Web-based survey. Approximately 12 individuals from each county (108 individuals total) will be asked to complete a brief web-based survey. This survey is an adapted measure previously fielded in these counties (Tungate, 2008); specifically, the survey developed by Tungate (2008) has been expanded to include additional items relevant to the current study particularly related to addressing the domains that we believe will be important in service of developing the Implementation Index. A brief web-based survey will also be administered to approximately 8 State-level staff to collect information about integration. The survey will be administered before the site visit by sending a secure link via email to those individuals identified by the site liaisons. An informed consent statement will be presented on the initial screen of the survey. The only personally identifiable information to be collected on the survey is the participants’ job title and length of service/time at current employer. E-mail addresses will also be obtained in order to send out the link to the survey. The survey data will also be housed in a database on encrypted, password protected electronic storage files.

Semi-structured Interviews. In addition to the brief survey, the two case study team members for each county and State will conduct 188 in-person (over the phone, if necessary) semi-structured interviews during a site visit with the key informants: County Directors of Human Services, State-level Administrators, Child Welfare/Colorado Works Leadership/Manager, Child Welfare/Colorado Works Case Manager, Caseworker, Technician, and Other Client-Serving Staff, Allied Staff, Partners, and Data Managers.

The only personally identifiable information that will be collected during the semi-structured interview is the participant’s name, job title, and length of service/time at current employer, which will only be used to frame the semi-structured interview. The contractor will make arrangements before the interview to conduct the interview in a quiet place that provides privacy (or over the phone, if necessary). Interviews will be audio recorded with permission and transcripts will be developed for each interview. The transcripts will be cleared of identifiers and housed in an ATLAS.ti database on encrypted, password protected electronic storage files.

Statistical concerns

This study uses a multiple case study design of 9 counties drawn from a convenience sample of counties with population over 100,000 and one State-level case study. There are no major statistical concerns related to the interviews and surveys as this is a case study. The selection of respondents will be carefully conducted to ensure the most appropriate individuals are identified and invited to participate. A variety of staff types in each county and the state are proposed for inclusion to ensure broad input and reduce potential for bias in case study findings.

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

All case study team members will receive extensive training on the case study protocols and protection of human subjects. The content of training will include: IRB approved procedures, including eligibility, enrollment, human subjects protection, semi-structured interview administration, and appropriate handling and management of data. All case study team members will be trained on how to follow-up on unexpected issues that may arise during interviews and what to do in the unlikely event that a respondent becomes uncomfortable with the interview. In addition, case study team members will be observed and given feedback during simulations of the semi-structured interview until they are able to perform each skill to the Principal Investigator’s satisfaction.

We expect that several aspects of the design of this study will maximize response rates:

• The survey is brief and takes approximately 15 minutes to complete.

• The semi-structured interviews were specifically designed for state- and county-level employees and take no longer than one hour to complete.

• Respondents will be asked only once to complete the survey and semi-structured interview; this collection request involves only one time point.

• The impact of this data collection on participants’ privacy is very low since no information that could link a participants’ name to his/her interview will be released to anyone outside of the project team (project director, project managers, site visitors and transcriber) and the risk of breach of privacy is minimal. The surveys are anonymous and the only personally identifiable information that will be collected during the semi-structured interview is the participant’s name, job title, and length of employment, which will only be used to frame the semi-structured interview. The contractor will make arrangements before the interview to conduct the interview in a quiet place that provides privacy. Interviews will be audio recorded with permission and verbatim transcripts will be developed for each interview. The transcripts will be cleared of identifiers and housed in an ATLAS.ti database on encrypted, password protected electronic storage files. The survey data will also be housed in a database on encrypted, password protected electronic storage files.

• Respondents completing the semi-structured interview will receive a $25 gift card in recognition of the time burden required to respond to the data collection request.

B.4. Test of Procedures or Methods to be Undertaken

The surveys and semi-structured interviews have been developed with the extensive input of expert consultants both internal and external to CDC and ICF International. In addition, the survey and semi-structured interviews are adapted from measures successfully fielded in Colorado counties (Tungate, 2008). Both before and subsequent to adaptation, all surveys, interviews and procedures were reviewed by both ICF International and CDC. Finally, we tested the case study protocol with a 9 county-level employees and as a result of this process, have refined our protocols and procedures for training.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

All instruments and procedures have been reviewed extensively by CDC. The following individuals have worked closely in developing the instrument and procedures that will be used, and will be responsible for data analysis:

• Natasha E. Latzman, Division of Violence Prevention, CDC

• Colby Lokey, Division of Violence Prevention, CDC

• Cathy Lesesne, ICF International

• Karen Cheung, ICF International

• Mary Sullivan, ICF International

• Pamela Day, ICF International

• Robert Stephens, ICF International

• Lucas Godoy-Garrazza, ICF International

• Freda Brashears, ICF International

References

Tungate, S. L. (2008). Welfare and child welfare collaboration. Dissertation Abstracts International Section A: Humanities and Social Sciences, 70(2-A), p. 706. Ann Arbor, MI: ProQuest Information & Learning.