1 PPW supporting statement 8-16-07 v2

SUPPORTING STATEMENT FOR THE

CROSS-SITE ASSESSMENT OF THE RESIDENTIAL TREATMENT FOR PREGNANT AND POSTPARTUM WOMEN (PPW), THEIR MINOR CHILDREN AND FAMILY PROGRAM

A. JUSTIFICATION

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT) is requesting approval from the Office of Management and Budget (OMB) for a revision of data collection activities of the Cross-site Assessment of the Residential Treatment for Pregnant and Postpartum Women (PPW), Their Minor Children and Family Program, hereinafter known as the PPW Program (OMB No. 0930-0269). The current approval expires September 30, 2007. Eight projects will be assessed in targeting substance abusing pregnant women and their minor children on the following two factors: (1) the mandate for collection and dissemination of findings for the PPW and (2) a monitoring system to measure project service capacity and treatment effectiveness.

The purpose of this submission is to modify the approved cross-site data collection effort based on six-month data collection experience with the six projects in the 2003 (first) cohort of this Assessment and feedback from project and assessment staff. The proposed modifications are: (1) to implement modifications to the instruments based on experiences gained during training on the cross-site process and instrument administration and data collection with the 2003 cohort; (2) to replace the 12-month post-intake data collection wave with a 6-month post-discharge data collection wave to ensure that post-discharge data is collected on all women (as some may still be in residential treatment at 12 months) and because it is important to collect post-discharge outcome data for all women ─ especially over a uniform interval (i.e., 6 months); (3) to increase the number of sites and participants involved in this Cross-site Assessment; and, (4) to increase the target population to ensure that the PPW program is more family-centered, as required in Congressional budget language for the PPW program for 2006.

Title V, Section 508, Residential Treatment Programs for Pregnant and Postpartum Women, of the Public Health Service Act, as amended, mandates the evaluation and dissemination of findings of residential treatment programs for pregnant and postpartum women. Though several Federal agencies have mandates to fund projects targeting substance abusing women and their minor children, evaluation data have shown that 10 percent or less of women treated in women-specific programs were pregnant or postpartum.¹ To address the needs of this underserved population, CSAT published a Request for Application (RFA) in 2006, which can be found in Attachment A. The eight projects were funded as a result of this RFA. To comply with Section 508, this Cross-site Assessment has implemented a systematic process of assessing accomplishments toward meeting the goals of the RFA. The Assessment appraises maternal and minor child health outcomes in eight projects in which SAMHSA and RFA goals are to improve the quality and availability of treatment through accountability, capacity, and effectiveness.

Some population-based data suggest that maternal substance abuse remains high, is typically undertreated even in women-specific treatment programs, and can have deleterious effects on maternal quality of life, birth outcomes, and child development. A National Institute on Drug Abuse (NIDA) study showed that children exposed to alcohol and illicit drugs are at-risk for birth defects, mental retardation, and later behavioral and learning difficulties.² Other studies reveal that children who are raised by drug abusing adults tend to exhibit a wide range of developmental, mental health and behavioral problems, and are themselves at higher risk for using alcohol and other drugs.³ The following data suggest the national magnitude of the problem:

• Nicotine and alcohol are the substances most widely used during pregnancy. National estimates suggest that 10.2 to 16.3 percent of women smoked during pregnancy in 2004 in the U.S.⁴ Approximately 10 percent of women were deemed to use alcohol during pregnancy, with about 2 percent binge drinking or drinking frequently in 2002.⁵

• In a large, multisite study (N =1,632), the incidence of substance use during pregnancy was as follows: tobacco (25%), alcohol (23%), marijuana (6%), methamphetamine (5%), and barbiturates (1%).⁶

• According to combined 2004 and 2005 estimates from the National Survey on Drug Use and Health (NSDUH), formerly the National Household Survey on Drug Abuse (NHSDA, in the month prior to the survey, 3.9 percent of pregnant females aged 15 to 44 used illicit drugs (i.e., marijuana, including hashish; cocaine, including crack; heroin; hallucinogens, including PCP and LSD; inhalants; or any prescription-type psychotherapeutic used non-medically), 12.1 percent used alcohol and 3.9 percent reported binge drinking.⁷

• CSAT is especially concerned about the high morbidity and mortality rates of African-American pregnant women and their infants. African-American pregnant women tend to use illicit drugs at a higher rate than any other population of pregnant women. Data from the combined 2004 and 2005 estimates from the NSDUH found that among pregnant women 15 to 44 years of age, 6.8 percent of African-American women reported illicit drug use in the month prior to the survey compared to 3.9 percent of white women.⁸

• The effects of illicit drug use by women during the prenatal period are well documented in the literature and include problems such as inadequate prenatal care,⁹ fetal death, premature delivery, congenital anomalies,¹⁰ and low birth weight infants.¹¹ The National Center for Health Statistics reports persistent racial/ethnic disparities in infant mortality. From 1980-2000, the infant mortality rate for babies born to African-American mothers was 14.0 per 1,000 live births while the rate for babies born to Caucasian mothers were 5.7 per 1,000.¹²

• Maternal alcohol use during pregnancy has been shown to have neurological and behavioral effects on minor children. The prevalence of infants born with Fetal Alcohol Syndrome (FAS) has been estimated at 0.2 to 1.5 per 1,000 live births. Other conditions associated with prenatal alcohol exposure are reported to be three times as common.¹³

• Maternal alcohol or other drug use impedes appropriate mother-child interaction, contributes to family dysfunction, impairs the child’s mental health, and deprives the child of social supports needed for early social and cognitive development.¹³

• Evaluation data showed that only 10 percent or less of pregnant or postpartum women treated for substance abuse/addiction and its complications utilize women-specific services targeted toward them.¹⁴

Outcome data reflect SAMHSA’s desire for consistency in data collected within the Agency. SAMHSA has implemented specific performance domains called National Outcome Measures (NOMs) to assess the accountability and performance of its discretionary and formula grant programs. These domains represent CSAT’s focus on the factors that contribute to the success of substance abuse treatment. The PPW program addresses the following performance domains:

• Abstinence from Drug / Alcohol Use

• Employment / Education

• Crime and Criminal Justice

• Family and Living Conditions

• Social Connectedness

• Access / Capacity

• Retention

• Perception of Care

• Use of Evidence-Based Practices

• Cost effectiveness (information obtained from the grant application).

It is well known that the use of drugs, alcohol, and tobacco has a negative impact on maternal health and pregnancy outcomes. Outcome data are used to measure the success of clinical treatment and recovery support services. The results are used to assess the need to continue SAMHSA’s targeted PPW programs, to design programs, to coordinate systems of care, and to provide assistance that ensures such programs can contribute appropriately to treatment and prevention of substance abuse among pregnant women and the prevention of health and educational problems among the offspring of these women.

SAMHSA/CSAT has established core standardized Government Performance and Results Act (GPRA) Client/Participant Outcome Measures (OMB No. 0930-0208) for all grant programs to report in order to capture the following essential client/participant-level information. The PPW projects are required to collect the following GPRA data for each mother that enters treatment.

1) Over the past year, whether there was an increase in the percentage of adults/adolescents that received services:

a) Who were currently employed or engaged in productive activities/attending school.

b) Who had a permanent place to live in the community/residing in a stable living environment.

c) Who had reduced involvement with the criminal/juvenile justice system.

d) Who had no use of illegal drugs or misuse of prescription drugs during the past month.

2) Retention in the program—determines the percentage of clients who completed the program or who left the program before completion and their status (discharge status).

3) Types of services received while in the program—to show the percentage of clients in the different types of treatment modalities.

4. Whether clients sought help from self-help groups to support their recovery.

PPW Program Background and Overview. In September 2003, SAMHSA awarded a cohort of six PPW grants for a period of three years, which constituted the first cohort – that is, the original applicants approved to participate in the Cross-site Assessment. Data were collected for the first cohort for the final six months of their funding periods, which did not provide sufficient information for the Report to Congress. Hence, in September 2006, SAMHSA awarded a cohort of eight PPW grants for a period of three years, which are the subject of this OMB submission. Participation was voluntary for the first cohort but is required by the RFA for this second (current) cohort.

Inclusion criteria: The PPW Program has several inclusion criteria which include women who are low-income (as defined by federal poverty definitions); age 18 and over; pregnant and/or postpartum (defined as the period after childbirth up to 12 months); and their minor children age 17 and under as well as their family members (i.e., partners and fathers of the children). These women and children have limited access to quality health services. Racial and ethnic minority women, especially African American women, have been identified as an important subpopulation by CSAT because they have been traditionally underserved and have had high morbidity and mortality rates. An addition in the Congressional budget language for FY 2006 recommends that SAMHSA explore ways to increase family-focused treatment capacity. Therefore, for the PPW Program, CSAT increased the target population to include the partners of the women and the fathers of the children, when deemed to be appropriate and beneficial, as well as extended family members of the women and children in treatment.

Length of residential treatment: CSAT recommends that the intensive residential treatment phase of the treatment process not exceed 12 months. However, PPW projects may select treatment phases ranging from 3 to 12 months as charted by the client’s individual service plan.

Program goals: The primary goal of the PPW treatment program is to provide cost-effective, comprehensive residential substance abuse treatment services for women and their minor children that can be sustained over time. The service system should address the individual needs of the target population and preserve and support the family unit, while creating a safe and healthy environment for family members.

Treatment outcomes: The treatment outcomes for the women include decreased alcohol and drug use; increased safe and healthy pregnancies; improved mental and physical health; improved family functioning; economic stability; improved quality of life; as well as decreased involvement in and exposure to crime, violence, sexual and physical abuse, and child abuse and neglect. The treatment program goals for the children are improved birth and developmental outcomes; prevention of and/or decreased use and abuse of alcohol and drugs; reduced effects related to maternal drug abuse; improved mental and physical health; and decreased involvement in and exposure to crime, violence, sexual and physical abuse, and child abuse and neglect.

2. Purpose and Use of Information

The primary purpose of the PPW Program is to provide cost-effective, comprehensive residential substance abuse treatment services for women and their minor children that can be sustained over time. To measure the achievement of this goal, CSAT is conducting this Assessment and program monitoring to measure the outcomes of treatment at each project on maternal and child outcomes, including maternal substance abuse, quality of life, infant birth outcomes, and early childhood development. CSAT will use this information to document and report the extent to which the goals of the RFA were achieved, as mandated by Congress.

The 2006 cohort will be gathering data during the three-year funding of the eight projects on approximately 963 mothers (age 18 and older) and their 3,852 minor children who consecutively enter treatment. Data on women and their minor children will be collected at six months following treatment discharge. (This data collection wave will replace the already-approved 12-months post-intake data collection for the PPW-specific tools.) Many of the data elements required for the Assessment are routinely collected by the projects as part of their own program management efforts as well as through the required GPRA data collection (OMB No. 0930-0208) for each mother. The eight projects are required to use the same data collection instruments to ensure cross-site comparability of the data.

This Cross-site Assessment seeks to show that from treatment intake to treatment discharge the following occurred:

1. Pregnant and postpartum women across the projects decreased their use and/or abuse of prescription drugs, alcohol, tobacco, and illicit and other harmful drugs (e.g., inhalants).

2. There was an increase in safe and healthy pregnancies, improved birth outcomes, and reductions in related effects of maternal drug abuse on their minor children.

3. The mental and physical health of women and minor children in these projects improved.

4. Women and minor children served by these projects improved their family functioning, economic stability, and quality of life.

5. There was decreased involvement with and exposure to crime, violence, sexual and physical abuse, and child abuse and neglect both in the role of victim and perpetrator.

6. The client’s quality of life improved, from the client’s perspective, related to health, social functioning, and environmental support.¹⁵

7. There was a decrease in barriers to accessing treatment and barriers to completing treatment.

RFA objectives. Two blueprints have been developed that are crucial to the overall management of the Cross-site Assessment and provide clear guidance for the implementation plan. The blueprints, one for women (Attachment B) and one for children (Attachment C), describe and crosswalk the methodology of the Cross-site process. The blueprints list each PPW RFA goal/objective and link them to Cross-site investigative questions and then to measures that determine outcome indicators, including SAMHSA NOMS and PPW Cross-site specific outcomes. The blueprints also present current literature (see Attachment D) to support the relevance and importance of the investigative questions and definitions provide consistency and guidance to the Assessment. The methods of measurement are presented, including reliability and validity of the instruments. The target groups and data collection periods are clearly delineated and linked to the PPW RFA goals.

An important part of this RFA objective is to involve family members of the women and children. Within 30 days of intake, CSAT will gather descriptive information on the level of involvement of fathers and father figures with each child and the man’s history of substance abuse (Child Data Collection Tool). At discharge CSAT will assess whether family counseling (Woman’s Discharge Tool), father-child parenting/bonding classes, and mother/father/child counseling/classes (Children’s Discharge Tool) were provided during treatment (as well as how often, where, and by whom). An item will be added to the Children’s Discharge Tool assessing who the child will live with upon discharge. The Family Recovery Support Services Tool will assess the level of services received by the women, the children, and family members at 6-months post-intake, at discharge, and at 6 months post-discharge followup.

Project-level RFA goals are designed to:

• Provide cost-effective and comprehensive residential substance abuse treatment services for women;

• Develop the project using a sustainability approach that encourages the continuation of these funded services; and

• Increase access of pregnant and postpartum women to treatment for alcohol, tobacco, and other drug use and/or abuse.

The following list presents each RFA objective, the assessment questions related to each, and the method of measuring attainment of the objective.

1. Provide cost-effective and comprehensive residential substance abuse treatment services for women, their minor children, and their families.

The achievement of this goal will be assessed by determining the number of women and their minor children served, bed utilization rates, types of services received, standard price of services, length of stay, and that the projects fall within the established CSAT cost bands for residential treatment. Data for these measures will be collected from GPRA Client/Participant Outcome Treatment Measures data (OMB No. 0930-0208) that are collected in “near” real-time.

2. Develop a program sustainability approach to encourage the continuation of the funded services.

The achievement of this objective will be assessed based on biannual reports, site visits/diagnostic assessments, and continuation application information that documents the implementation of long-range plans (a 3-5 year period into the future).

3. Determine if throughout the life of the PPW projects there is a decrease in the prevalence of alcohol and other drug use among pregnant and postpartum women and their minor children.

The achievement of this objective will be assessed based on self-reported patterns of use and/or abuse of substances using data collected from the GPRA Client/Participant Outcome Treatment Measures instrument (OMB No. 0930-0208) at intake, and at 6 months post-intake, at discharge, and, additionally for this Assessment only, at 6 months post-discharge. The Women’s Discharge Tool, collected at discharge, also will inform this objective by documenting pregnant women’s pregnancy trimester at admission.

4. Improve the mental and physical health of women.

The achievement of this objective will be assessed using the GPRA Client/Participant Outcome Measures instrument data and BASIS 24® data, both of which are administered at intake, 6 months post-intake, at discharge, and at 6 months post-discharge. Outcome indicators include documented existence and severity of psychiatric/mental health problems, and changes in these measures over the course of treatment and at followup.

5. Increase safe and healthy pregnancies.

This project outcome will be assessed with documented occurrence of health and wellness issues, levels of acuity, and changes over time on the Women’s Medical Record Audit. Records review of documentation related to physical health exams and followup-health care also informs this objective. This instrument will be completed at intake, 6-months post-intake, discharge, and at 6 months post-discharge.

6. Improve family functioning.

CSAT will record the family composition of the child’s primary residence and rate the quality of parenting care provided (and changes over time) on the Child Well-Being Scales, which are completed within 30 days of maternal intake GPRA or delivery, 3 months, 6 months, discharge, and 6 months post-discharge. In addition, CSAT will use the Ferrans and Powers Quality of Life Index to collect information from women and their partners/children’s fathers on overall family satisfaction including the woman’s satisfaction with her partner and the emotional support she receives from her family. This Index will be collected at intake, 6 months, discharge, and 6 months post-discharge for the women and within 30 days of the woman’s intake and at her discharge for the partners/fathers.

7. Improve the quality of life for alcohol-, tobacco-, and other drug-using, abusing, and dependent PPW.

Achievement of this objective will be assessed using items from the Ferrans and Powers Quality of Life Index, which measures satisfaction with the importance of five life areas: personal and family health factors, social support, socioeconomic factors, psychological/spiritual factors, and family functioning. Each of these quality of life indicators will be measured at intake, 6 months, discharge, and 6 months post-discharge.

8. Increase access to treatment for alcohol-, tobacco-, and other drug-using, abusing, and dependent PPW.

The achievement of this objective will be measured using the Allen Barriers to Treatment Instrument at intake, 6 months, discharge, and 6 months post-discharge. This instrument assesses personal beliefs, feelings, thoughts and other issues that frequently impede PPW from initially entering treatment. Changes in barriers to treatment entry and barriers to service use during treatment will be assessed.

9. Provide comprehensive therapeutic services for the minor (age 17 and under) children of substance-abusing women. Describe services these minor children receive while in substance abuse treatment with their mothers. Describe the demographic profile of the treated children.

The measurement of this objective will be assessed based on information collected from the Child Data Collection Tool, administered at intake, and the Children’s Discharge Tool, administered at discharge.

10. Improve birth outcomes and reduce related effects of maternal drug abuse on minor children. Specifically, improve the physical and mental health of child participants.

Physical Health: Key indicators for newborns delivered while the mother is in the residential treatment facility will be examined only at delivery (Newborn’s Medical Record Audit). These indicators are Apgar score, head circumference, length at birth, birth weight, weeks of gestation at delivery, and drug toxicology.

Key measures for all children will be measured by the Children’s Medical Record Audit, Women’s Discharge Tool (collected at discharge). These measures include improvement in physical health as evidenced by complete physical exams, routine laboratory testing, HIV/AIDS testing, and appropriate immunizations and prevention examinations. Data on the Children’s Medical Record Audit will be collected on all minor children within 30 days of the mother’s GPRA intake data collection or the child’s birth and again at 3 months, 6 months, discharge, and 6 months post-discharge.

Mental Health: Achievement of this objective will be measured using different instruments that measure appropriate developmental progression and achievement of milestones, number of personal and social strengths, and number of personal and social issues. The instrument administered depends on the child’s age as indicated below:

• Denver Developmental Screening Inventory II (ages 0 to 6 years, 0 days),¹⁶

• Middle Childhood Developmental Assessment Guide (ages 6 years, 1 day to 10 years), and

• Adolescent Childhood Developmental Assessment Guide (ages 11 to 17).

Administration of these instruments will occur within 30 days of the mother’s GPRA intake data collection or the child’s birth, 3 months, 6 months, discharge, and 6 months post-discharge.

11. Reduction in the related effects of maternal drug abuse on their minor children, including exposure to crime, violence, child sexual and physical abuse, neglect, and substance abuse.

Data to assess the attainment of this objective will be based on the Child Well-being Scales and the CRAFFT substance abuse screening instrument (for adolescents ages 11 to 17). These will be administered within 30 days of the mother’s GPRA intake data collection (or the child’s birth) and at 3 months, 6 months, discharge, and 6 months post-discharge.

Data collection instruments. Some of the data collection instruments are standardized instruments that have a long history of use in the substance abuse and child development fields and were chosen because they are working well in the field. Other instruments were developed specifically for the PPW Cross-site Assessment. CSAT will request that the eight projects routinely collect data using each of these instruments and will train appropriate program staff on their administration.

Based on experience with the 2003 cohort, CSAT proposes modifications to the instruments (summarized in Attachment E). Attachment F contains all data collection instruments to be used in the Cross-site Assessment. A new instrument will be added (Family Recovery Support Services Tool) to reflect two program change requirements in the 2006 RFA: (1) projects are to include family-focused treatment and (2) are to provide recovery support services during and post residential treatment. All instruments have been formatted for the Assessment to give them uniformity of appearance. The changes are highlighted on each instrument. Variable names and values have been added on the forms (in gray) to facilitate data entry and to simplify use of the data sets and output. Space for collecting tracking and administrative information has been added at the top of each form to include date, start and end time, IDs, data collection wave, interviewer and grant number. Having the tracking information will help minimize data collection and data entry errors.

As mentioned previously in this statement, the 12-months post-intake data collection wave will be replaced with a 6-month post-discharge data collection wave.

Provided next is a general description of standardized instruments that have established reliability and validity and are widely used in the field. Except for minor modifications to the formatting, which have been approved by the authors, no substantive changes are proposed to the standardized instruments.

• Allen Barriers to Treatment Tool (mothers) (Attachment F-1) ─ The purpose of this tool is to assess the perceived barriers that prevent women from obtaining and completing treatment services for alcohol and/or other drug problems. It measures treatment program characteristics; personal beliefs, feelings, or thoughts; and other issues and is currently being used in the substance abuse field in women’s treatment and NIDA-funded research at two major universities.

Proposed modifications: Minor formatting changes only, approved by the author. The instruction was modified from “Circle one number” to “Check one box” to reduce scoring errors and for consistency.

• BASIS 24® (mothers) (Attachment F-2) ─ The 24-item Behavior and Symptom Identification Scale (BASIS-24®), copyrighted by McLean Hospital, Belmont, MA, is a leading behavioral health survey that measures the change in self-reported symptoms and problem difficulty over the course of treatment. It was developed to assess outcomes from residential and outpatient mental health treatment. Like its predecessor (BASIS-32®), the survey measures the degree of difficulty experienced by the client during a one-week period on a five-point scale ranging from no difficulty to extreme difficulty. Scoring uses an algorithm that gives an overall score with six subscales for the following domains: Depression and Functioning, Relationships, Self-Harm, Emotional Lability, Psychosis, and Substance Abuse.

Proposed modifications: No changes will be made to this instrument, although it has been formatted to be consistent with the other instruments used in this study. CSAT proposes to change from BASIS-32®, which was used with the first cohort, to BASIS-24® because the BASIS-24® focuses more on alcohol and substance use items than the prior version, thus presenting a clearer picture of the substance use disorder. The BASIS-24® is the only version of the tool currently supported by McLean Hospital, which is applying all of their resources to this shorter instrument. Such resources include web-based scoring and the collection of national normative data (http://www.basissurvey.org.basis24/). Based on feedback from some projects that have prior experience using the BASIS-24®, it seems to be more valid for the current

substance abusing population for diagnostic and treatment planning than its predecessor.

• Child Well-Being Scales (mothers and all minor children) (Attachment F-3) ─ The purpose of this tool is to assess the quality of parenting children receive, with a particular sensitivity toward abuse and neglect. The scales were developed to evaluate programs of child welfare services funded by the Administration for Children, Youth and Families, Office of Human Development, DHHS.

Proposed modifications: No changes to the instrument. A data reporting sheet was developed to assist data collection. This sheet includes only those 13 (out of 43) scales that will be used in the Assessment and presents all response categories for the respondent to check off.

• CRAFFT (ages 11 to 17) (Attachment F-4) ─ This is a substance abuse screening instrument for adolescents. Receiving a score of two or higher indicates the need for further assessment.

Proposed modifications: As there was no instruction on the tool, the following was added: “Please check a YES or NO response to each of the following questions.”

• Denver Developmental Screening Inventory II (ages 0 to 6 years) (Attachment F-5) ─ The Denver II was copyrighted by W.K. Frankenburg and is a widely-used clinical screening instrument that measures appropriate developmental progression and achievement of milestones and a reduction of noted developmental delays and/or gaps.

Proposed modifications: No changes. As administration and scoring of this tool requires extensive training, a reporting form to facilitate correct data entry by staff unfamiliar with administration and scoring of this instrument was developed.

• Ferrans and Powers Quality of Life Index (mothers and their partners and children’s fathers) (Attachment F-6) ─ The purpose of this self-report tool is to measure the client’s quality of life. Specifically, it measures satisfaction with various aspects of life and the importance of those same aspects. This instrument measures satisfaction with/and importance of four areas in life. The website for this instrument is (http://www.uic.edu/orgs/qli/). This instrument will be administered to women at intake, 6 months, discharge, and 6 months post-discharge. This instrument will be administered to family members (i.e., women’s partners and children’s fathers) within 30 days of the woman’s intake and at the woman’s discharge.

Proposed modifications: The instruction was modified from “circling the number” to “checking the box” to reduce scoring errors and for consistency.

• Government Performance and Results Act Client/Participant Outcome Measures (GPRA) (OMB No. 0930-0208) (mothers) (Attachment F-7) ─ In addition to the required GPRA administration at treatment intake, discharge and 6-months post-intake, this instrument will be administered to the women in the PPW program at 6-months post-discharge to assess the substance use of the mothers after discharge from residential treatment. This new data collection wave will replace the already-approved 12-months post-intake data collection wave. This is particularly important, as GPRA provides important outcome data useful for measuring treatment effectiveness, particularly in the areas of continued alcohol and drug use.

Proposed modifications: No changes to the instrument.

The following instruments were developed to be specific to the needs of this Assessment. They were pre-tested (as described in Section B.4) then used successfully with the 2003 cohort. Proposed modifications to the instruments are highlighted on the copies provided in Attachment F.

• Child Data Collection Tool (mothers) (Attachment F-8) ─ The purpose of this tool is to collect comprehensive demographic and health information to create a profile of each minor child of mothers who enter residential treatment. Further, it is intended to assist the treatment field to identify resources needed for children and assist in the development of programs for these children. This tool is based on standard questions that address demographic characteristics, socioeconomic status, alcohol and other drug use, and health related issues.

Proposed modifications: The most modifications proposed in this OMB submission are to this tool and are described in detail in Attachment E. Add 13 items and split one item into two for clarity. Delete two items plus an item (c) that applies to all 30 items in Part 2. Condense seven items into three. Clarify item wording.

• Middle Childhood Developmental Assessment Guide (ages 6 to10) (Attachment F-9) and Adolescent Childhood Developmental Assessment Guide (ages 11 to 17) (Attachment F-10) ─ The purpose of these instruments is to ascertain the child’s development relative to his/her achievement of age-appropriate developmental milestones. These assessment guides were developed and validated in the “Bright Futures: Chartlines for Health Supervision of Infants, Children and Adolescents” program directed by Green and Palfrey, Georgetown University, and sponsored by Maternal and Child Health Bureau, Health Resources and Services Administration and the Center for Medicaid and State Operations, Health Care Financing Administration.

Proposed modifications: Add one item to each of the instruments to complement and clarify other items in the section; clarify language and scoring; and to minimize missing data, add one skip pattern to the Middle Childhood and “Not Applicable” response options to the Adolescent Childhood.

• Women’s Discharge Tool (mothers’ records) (Attachment F-11) and Children’s Discharge Tool (children’s records) (Attachment F-12) ─ The purpose of these instruments is to obtain a picture of what services the women and minor children received while participating in these PPW substance abuse treatment programs. They provide an overview of services received during treatment.

Proposed modifications – Women’s Discharge Tool: Add one item and discharge date, delete one item, split one item into two for clarity, and clarify language.

Proposed modifications – Children’s Discharge Tool: Add five items and discharge date, and clarify language.

• Women’s Medical Record Audit (mothers’ records) (Attachment F-13), Children’s Medical Record Audit (ages 3 months to 17 years) (Attachment F-14), and Newborn’s Medical Record Audit (newborns delivered while the mother is in treatment) (Attachment F-15) ─ The purpose of these instruments is to determine the quality of the data in the client’s medical record.

Proposed modifications: Add intake/delivery date; clarify instructions, wording, and response categories; and divide into three forms (based on target record) to simplify administration.

The following instrument was developed to be specific to the needs of this Assessment and is new to the assessment. It has not yet been pre-tested.

• Family Recovery Support Services Tool (mothers) (Attachment F-16) – The purpose of this instrument is to gather data on the recovery support services provided to women, children, and their families received during treatment and during the 6 months following treatment discharge. This reflects requirements added to the 2006 RFA for involving families in treatment and for providing recovery support services. This tool will be administered at 6 months post-intake, at discharge, and at 6 months post-discharge.

3. Use of Information Technology

Data collection using these paper-based instruments is conducted in the course of normal service delivery (as is the generally accepted assessment technique within clinical settings). Project staff will photocopy the instruments and send them to the CSAT-designated contractor for double key entry. Projects retain a copy of each instrument for their own clinical use. Current technology is used to manage, secure, store, clean and analyze the data.

With the exception of GPRA, web-based data collection is not practicable for several reasons. First, the development of a web-based data collection system is expensive, particularly considering the type of instruments included and the relatively modest number of participants responding to each. Second, it would not reduce burden. Data are collected as part of clinical practice and would need to be recorded and then entered into a computer. Third, submission of hard-copy instruments allows time for contractor staff to resolve errors before data entry occurs which ensures greater quality control.

4. Efforts to Identify Duplication

Information to be collected as part of this Cross-site Assessment with these eight residential projects is not available elsewhere. The battery of instruments to be used in this Assessment has not been collected previously with this population. These data are specific to the needs of this Assessment and not available elsewhere.

5. Involvement of Small Entities

The eight projects aim to treat approximately 20 to 40 clients per year (one aims to treat 20 clients, two aims to treat 30, and the rest aim to treat 40). Many of the questions are generally incorporated in instruments that are familiar to clinicians in the substance abuse treatment field. There is not a significant impact to these entities.

6. Consequences if Information is Collected Less Frequently

During this Assessment, the frequency of data collection from projects, women, minor children, and family members will be held to the minimum necessary to meet the needs of the Assessment. The data collection points for this Assessment are generally accepted intervals for assessing the effectiveness of substance abuse treatment.¹⁷ CSAT needs these data in order to address important issues in the Report to Congress.

The mothers will be asked to respond voluntarily at intake (just after their GPRA intake assessment), 6 months post-intake, discharge, and 6 months post-discharge. The data collection points for the children are within 30 days of maternal intake GPRA or delivery, 3 months and 6 months after the maternal GPRA intake, discharge, and 6 months post maternal discharge. The 3-month data collection wave is added for children because of the speed of development among minor children and because many of these children may experience changes in their living situations relative to the mother’s treatment entry. It is important to capture these changes in the data. Family members (i.e., women’s partners and children’s fathers) will be asked to voluntarily complete one instrument within 30 days of the woman’s intake and at the woman’s discharge.

7. Consistency with the Guidelines in 5 CFR 1320.5(d) (2)

This information collection fully complies with 5 C.F.R. § 1320.5(d)(2).

8. Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on October 31, 2006 (69 FR 63775). No comments were received.

The PPW Blueprints for Women (Attachment B) and for Children (Attachment C) that guide the Cross-site Assessment have been updated to reflect the proposed changes and were based on an extensive review of the literature (Attachment D). These were developed by Karen Allen, Ph.D., R.N., Chair, Department of Nursing, Andrews University. Dr. Allen is the lead researcher/consultant on the PPW Program and her areas of expertise include substance abuse and addiction, leadership/administration, and research. Her contact information is as follows:

Karen Allen, Ph.D., RN.

4137 Courtney Street

St. Joseph, MI 49085

Phone: (269) 471-3364

E-mail: [email protected]

Staff from the PPW projects provided input and feedback into the assessment process. See Attachment G for a list of project contact people. Projects provided such feedback during a focus group, meetings, conference calls, and training sessions by e-mail and telephone. Feedback addressed a wide array of topics relevant to the Assessment including instrumentation, item wording, data collection procedures, and data processing.

9. Payment to Respondents

Cash-equivalent incentives are used for all followup interviews conducted after the woman has left the residential treatment program and for all interviews of family members not in the treatment program with the woman. RFA recommendations state, that “the maximum allowable incentive is $20.00 or equivalent value in coupons, bus tokens, and personal care items per followup interview.” Cash equivalents are offered in lieu of cash payments. This remuneration is expected to aid in achievement of acceptable response rates.

Survey research literature suggests that monetary incentives have a strong positive effect on response rates and no known adverse effect on reliability. Substance abuse research has shown improved response rates when remuneration is offered to respondents. Substance abusers are typically a harder-to-reach population for whom out-of-pocket costs (e.g., transportation, child care) are significant barriers to participation. Therefore, transportation, child care, and/or home visits are provided by projects as needed for participation in the followup. In addition, it is expected that projects follow the recommendations of the RFA with regard to remuneration amounts.

Results from the 2001 National Household Survey on Drug Abuse (NHSDA) incentive experiment were reported by Eyerman and Bowman (2001) and Wright, Bowman, Butler, and Eyerman (2002). Key conclusions from their analyses are quoted below:¹⁸

• The $20 and $40 incentive payments each produced about a 10-point gain in overall response rates when compared with the $0 control group. The overall response rate was significantly higher for $40 than the $20 incentive within many of the subgroups addressed in the analysis.

• Both incentive payment groups more than paid for themselves due to decreased costs of followup and more productive screening resulting from the improved response rates.

• Some significant differences in prevalence rates were noted in comparisons between the $40 treatment and the control in some of the age, race, and historical response rate groups: two cases of significantly higher past month alcohol use and one case of significantly lower past month cigarette use.

• Persons who responded with incentives, but would not have responded without them, are different and have higher substance use than persons who would respond with or without incentives.

• Incentives motivate (or obligate) respondents to admit to substance use that they might not have admitted without the incentive.

10. Assurance of Confidentiality

Client treatment records in Federally-assisted alcohol and drug abuse programs are protected under 42 CFR Part 2 (OMB No. 0930-0092). SAMHSA and its contractors do not receive identifiable client records. Provider-level information is aggregated to, at the least, the overall grant program level.

The directors of all selected projects and all other potential respondents are assured that privacy is maintained throughout data collection (to the extent permitted by law) in all project-level descriptive data gathered periodically from project staff by CSAT staff. All data are closely safeguarded, and no institutional or individual identifiers are used in assessment reports, in which only aggregated data are reported.

Protection of the rights of assessment participants is assured through a combination of widely accepted survey practices. All PPW projects comply with applicable Federal and State laws and with ethical principals in the collection of information from and about persons enrolled in, or related to persons enrolled in, treatment. Among the rights commonly held for these types of studies are:

1. The right of informed consent/assent, which requires the assessment team to provide sufficient information about the assessment objectives, level of burden, and uses of participants’ information so that individuals may make an informed decision on participation;

2. The right to refuse to participate, which applies to the individual’s right to decline to participate at all in the Assessment or to decline to answer specific questions, without penalty or loss of benefits;

3. The right to privacy, which guarantees against invasions of privacy as well as the specific protections provided by the Privacy Act of 1974.

It is the responsibility of individual projects to ensure privacy of participant data. No system of records containing identifiers is maintained by CSAT or its contractors. Before submitting these data to CSAT, projects are instructed to delete all personal identifiers (such as names, addresses, phone numbers, Social Security Numbers, medical record numbers, etc.) from the data files. The projects also are directed to assign an identification number to each client strictly for the purposes of the Assessment. This number enables the contractor to keep track of individual client records in the absence of personal identifiers, and to link records over the course of the repeated submissions per client that take place as part of the time series design of the Assessment. However, the correspondence between the true identity of the client and the number assigned for assessment purposes are known only to the projects, who maintain parallel lists of the two types of client identifiers.

The Federal data collection affords no circumstance in which privacy of client data could be breached, since only anonymous information is received. It is the responsibility of CSAT to ensure that client data are reported only in aggregate form without linking information with a specific project.

SAMHSA is subject to the Privacy Act for the protection of data. Substance abuse treatment providers are subject to the Federal regulations for the privacy of alcohol and substance abuse patient records (42 CFR Part 2, OMB No. 0930-0092) which govern the protection of patient identifying data. In some cases, these same providers meet the definition of a HIPAA covered entity and are additionally subject to the Privacy Rule (45 CFR Parts 160 and 164) for the protection of individually identifiable data.

The PPW Program has been determined by the CSAT Director to fall under the SAMHSA Participant Protection Procedures. These procedures require each applicant to the RFA to provide information which is used to determine whether the level of protection of human subjects appears adequate or whether further provisions are needed according to standards set forth in 45 CFR Part 46 Protection of Human Subjects.

Adequate protection of human subjects is an essential part of an application and is carefully reviewed by the grant review panel. Applicants must report any foreseeable participation protection risks and the procedures developed to protect participants from those risks. Applicants must describe the selection of participants, consent/assent procedures, privacy procedures, and data collection including from whom the data are collected, the form of specimens, records, or data. In addition, projects must include a discussion of why the risks to subjects are reasonable in relation to the anticipated benefits to subjects and in relation to the importance of the knowledge that may reasonably be expected to result. The applications also are examined by a Federal Project Officer to determine whether these procedures are being met. The Project Officer works with applicants when the review panel has concerns or comments in order to enable award.

Although projects routinely obtain informed consent/assent from project participants, in which the women agree to allow information collected regarding them, CSAT has developed consent and assent forms to be used by the eight projects in this Assessment. (The forms can be found in Attachment H.) CSAT also provides needed oversight and training on issues relating to informed consent/assent for both women and minor children.

11. Questions of a Sensitive Nature

SAMHSA’s mission is to improve the quality and availability of prevention, early intervention, treatment, and rehabilitation services for substance abuse and mental illnesses, including co-occurring disorders, in order to improve health and reduce illness, death, disability, and cost to society. In carrying out this mission it is necessary for service providers to collect sensitive information, such as criminal justice involvement, use of alcohol or other drugs, as well as issues of mental health.

The data submitted by each project to CSAT as part of the client-level assessment are based in large part on data that most of the projects are already routinely collecting. This primarily includes data on client demographics, substance abuse and treatment history, services received, and client and child outcomes. In addition, projects frequently ask their clients about their experiences of physical, sexual, and emotional abuse, as well as the custody and living arrangements of their minor children and the client’s involvement with Child Protective Services.

11. Estimates of Annualized Hour Burden

The total annualized burden to respondents for all components of the PPW program is estimated to be 9,842hours. The annualized hourly costs to respondents are estimated to be $101,330. The burden estimates, summarized in the following tables, are based on the reported experience of the 2003 cohort. There are no direct costs to respondents other than their time to participate.

Table A-1 presents a summary of the overall total estimated annual response burden for this collection.

Table A-1. Total Annual Respondent Burden
Respondent	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Total for Mothers	321	---	12,840	---	4,289	$5.15	$22,089
Total for Family Members	642		642		161	$13.99	$2,252
Total for Minor Children	1,284	---	7,705	---	2,991	$0	$0
Total for Staff	8	---	7,712	--	2,401	$16.73	$76,989
TOTAL	2,255	---	28,899	---	9,842	--	$101,330

Note. Cells for “Responses per Respondent” and “Hours per Response” are blank because this is a summary table.

Table A-2 presents the individual estimated annual response burden for women for in-person interviews. It is estimated that the women earn minimum wage of $5.15 per hour (www.dol.gov/esa).

Table A-2. Annual Burden for Interviews of the Mothers
Instrument	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Child Data Collection Toola	321	4	1,284	0.75	963	$5.15	$4,959
Allen Barriers to Treatmentb	321	4	1,284	0.25	321	$5.15	$1,653
Ferrans and Powers Quality of Life Indexb	321	4	1,284	0.25	321	$5.15	$1,653
BASIS 24®b	321	4	1,284	0.17	218	$5.15	$1,123
Child Well-Being Scales (age 0 to 17)c	321	20 (5 times, < 4 settings)	6,420	0.33	2,119	$5.15	$10,913
Family Recovery Support Services Toold	321	3	963	0.25	241	$5.15	$1,242
GPRA Client/Participant Outcome Measures 6-mos. post-dischg.	321	1	321	0.33	106	$5.15	$546
Total for Mothers:	321	---	12,840	---	4,289	$5.15	$22,089

^aBased on intake interviews of 321 mothers regarding each of her estimated 4 children.

^bBased on interviews with 321 mothers at intake, 6 months, discharge, and 6 months post-discharge.

^cBased on interviews of 321 mothers (and observation of them interacting with their children) with regard to the setting in which each of her estimated 4 children lives. If all children live in the same setting, then the instrument is only completed once. This instrument is completed according to the children’s data collection schedule – that is, at intake/delivery, 3 months, 6 months, discharge, and 6 months post-discharge.

^dBased on 321 mothers at 6 months post-intake, at discharge, and 6 months post-discharge.

Table A-3 presents the individual estimated annual response burden for family members for in-person interviews. It is estimated that of the 321 mothers with four children in treatment with the mother, there are two family members (i.e., the women’s partner and/or father(s) of the children) that will be interviewed. The hourly wage of $13.99 was calculated based on weighted data from SAMHSA’s 2005 NSDUH respondents' personal annual income.

Table A-3. Annual Burden for Interviews of the Partners/Fathers
Instrument	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Ferrans and Powers Quality of Life Indexe	642	1	642	0.25	161	$13.99	$2,252
Total for Family Members	642	---	642	---	161	$13.99	$2,252

^eBased on 2 family members responding, on average, for each of 321 women.

Table A-4 presents the individual estimated annual response burden for children for in-person interviews and observation by staff of the child. It is estimated that the minor children do not work, and therefore, do not earn anything. Therefore, each estimated cost is $0.

Table A-4. Annual Burden for Interviews of the Minor Children
Instrument	Number of Respondents	Responses per Respondent	Total Responses	Hours Per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Denver Developmental Screening Inventory II (age 0 to 6 years)f	770	5	3,850	0.50	1,925	$0	$0
Middle Childhood Developmental Assessment Guide (age 6 to 10)g	257	5	1,285	0.33	424	$0	$0
Adolescent Childhood Developmental Assessment Guide (age 11 to 17)h	257	5	1,285	0.33	424	$0	$0
CRAFFT (age 11 to 17)h	257	5	1,285	0.17	218	$0	$0
Total for Minor Children:	1,284	5	7,705	---	2,991	$0	$0

^fBased on 60% of 1,284 minor children ages 0 to 6 at intake or delivery, 3 months, 6 months, discharge, and at 6-months post-discharge.

^gBased on 20% of 1,284 minor children ages 6 to 10 years at intake, 3 months, 6 months, discharge, and 6-months post-discharge.

^hBased on 20% of 1,284 minor children ages 11 to 17 at intake, 3 months, 6 months, discharge, and 6-months post-discharge.

Table A-5 presents the estimated annual response burden for one staff person at each of the eight projects to conduct a records review and to complete each of the instruments listed for the respondents in the PPW program. The 2005 mean hourly wage for substance abuse and behavioral disorder counselors is estimated to be $16.73 (www.bls.gov).

Table A-5. Annual Burden for Records Review by Staff
Instrument	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Women’s Medical Record Audit	8	120 (40 women, 3 times)	960	0.25	240	$16.73	$40,152
Children’s Medical Record Audit	8	600 (117 intakes; 1,284 followups , 3 times = 483)	4,800	0.25	1,200	$16.73	$20,760
Newborn’s Medical Record Audit	8	43	344	0.08	28	$16.73	$468
Women’s Discharge Tooli	8	40	320	0.58	186	$16.73	$3,112
Children’s Discharge Toolj	8	161	1,288	0.58	747	$16.73	$12,497
Total for Staff:	8	---	7,712	--	2,401	$16.73	$76,989

ⁱBased on treatment records review on all mothers at discharge. The instrument is completed for all women who entered treatment regardless of treatment completion rate.

^jBased on treatment records review on all minor children at discharge. The discharge instrument is completed for all minor children who entered treatment regardless of treatment completion rate.

13. Estimates of Annualized Cost Burden to Respondents

There are neither capital or startup costs nor are there any operation and maintenance costs.

14. Estimates of Annualized Cost Burden to the Government

The total average annual cost to the Federal government for the PPW Cross-site Assessment is estimated to be approximately $1,049,314. This includes $3.9 million annually in grants (approximately $500,000 per grant for a total three years) and a total of $49,314 in Federal costs associated with project monitoring and information dissemination. Included in the Federal monitoring costs are those costs that are incurred by the government in personnel costs for oversight of the Assessment by one CSAT employee (GS-13) for approximately 50 percent of their time for $44,993 annually. Additionally, costs are included that are incurred by the government for approximately 15 percent or $30,000 of a $200,000 annual technical assistance (TA) contract for the following activities:

• Train sites on data collection and data entry;

• Coordination with the PPW projects regarding evaluation efforts; and

• Development of reports, and documentation and dissemination of findings.

Another contractor is responsible for the following activities:

• Collection of project-level and client-level data from projects; and

• Data cleaning, preparation of data files, and statistical support.

14. Changes in Burden

Currently there are 335 burden hours in the OMB inventory. When preparing this submission it has come to SAMHSA’s attention that the hours were incorrectly recorded on the Notice of Action. The burden should have been (5,011 hours). SAMHSA is now requesting (9,842 hours). The increase of (9,507 hours are due to a (4,675 hour) adjustment for the recording error and 4,832 hours of a program change. While the number of women expected to be served decreased, the program change is due to an increase in the expected number of children per woman based on the 2003 cohort; an increase from 2.23 children per mother to 4 children per mother. Therefore, the number of hours children are interviewed increased from 2,174 hours to 2,991 hours and the number of hours women are interviewed about their children increased from 1,160 hours to 4,045 hours. The staff hours increased from 335 hours to 2,401 hours due to the time needed to record medical record audits and discharges. The Family Recovery Support Services Tool, an additional interview time period for the GPRA Client/Participant Outcome Measures and interviews of partners/children’s fathers have been added for 508 hours.

14. Time Schedule, Publication and Analysis Plans

16.a. Time Schedule

The annualized schedule below shows when activities are estimated to occur in the months after approval of this data collection assessment for the eight PPW grants in the second cohort.

Train sites & provide TA on data collection and data entry Month 1 & ongoing

Receive client data from sites Month 1 & ongoing

Data analysis Month 6 & ongoing

Validation of findings with projects Month 12, 24, 36 (or during

annual project meetings)

Report to Congress and publications End of grant period

16.b. Publication Plans

In addition to the required Report to Congress, presentations are made at PPW project meetings and conferences. The Report to Congress will be distributed to those interested in the role of treatment and prevention of substance abuse among pregnant and perinatal women and the prevention and amelioration of its effect on minor children.

16.c. Analysis Plans

The Cross-site Assessment design assesses the impact that the PPW program has had on the target population by examining the services received over time. The analysis focuses on a series of process questions, intermediate outcomes and distal outcomes for the participants in this program. Data are collected from mothers, their children, and their families over the 3-year duration of the grant program.

The information to be collected enables CSAT to describe the demographic characteristics of clients and to classify them at intake based on a series of assessment instruments that addresses such items as: client drug use; maternal and child health; child well-being; quality of life; and barriers to treatment. Basic frequency distributions and measures of central tendency and variability are employed where appropriate, to discern the overall distribution of the data and the participant characteristics. Additional process data are used to describe the eight projects regarding the description of interventions, staffing, utilization rates, and community context.

More specifically, the descriptive analysis primarily utilizes frequency distributions and counts from intake data, collected as part of the intake process in order to address such questions as:

1. How many clients were seen in this Assessment?

2. What were the demographic characteristics of the clients seen by these projects?

3. How many children were provided services and what were the services rendered?

4. What were the characteristics of the minor children in terms of the following areas:

   1. Health

   2. Quality of life

   3. Maternal bonding

5. What were the characteristics of the clients in terms of the following areas:

1. Employment Status

2. Housing Status

3. Criminal Justice Involvement

4. Recovery Support

5. Substance Use

Intermediate outcomes will be collected in terms of treatment completion and length of stay, as well as barriers to treatment experienced by the clients. These outcomes will be examined and measured within each PPW project for the purpose of program monitoring and project performance. In addition, these factors may, in the outcome analysis, be utilized as covariates.

In addition, outcome analyses utilize a pre and post measurement methodology. For the principal outcome items (e.g., drug use, maternal and child health, child well-being, quality of life, and barriers to treatment), the proportion of individuals showing improvement from intake to followup (0 to 3 months, 0 to 6 months, 0 to discharge) and discharge to 6 months post-discharge) will be calculated and aggregated at the program level. The followup interview data also will be described using frequency distributions and measures of central tendency in order to determine the distribution. Specifically, the percent of clients showing changes will be calculated on each of the projects’ categorical client outcome measures. For continuous items, mean differences will be used. Tables will be constructed to describe the change across projects on client/participant outcomes.

This limited data collection will not produce any nationally representative estimates. Data will be tabulated in a way that addresses the principal assessment objectives outlined in Section A.2. The data items collected will be analyzed and presented in reports using basic descriptive statistics for program monitoring reports that SAMHSA can utilize for performance review, improvement and oversight and for the Report to Congress on this specific program. If deemed necessary for CSAT-specific issues, the data will be examined at the project level. Results may be examined for subpopulations of interest within individual activities (e.g., by age or by gender).

1. Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

1. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The estimated universe of individual respondents includes women and their minor children, as well as family members of the women entering treatment per year across the eight grant projects. All clients entering treatment will be asked to participate in the Assessment. For each project, the starting point for data collection is at intake – that is, the time when the mother’s intake GPRA is completed. Women who leave the program at any time prior to treatment completion are targeted along with those who complete the program.

Based on other related projects currently reporting GPRA data to CSAT, it is expected that most clients will not refuse to participate in the program, thus PPW projects should not have a problem reaching the 80 percent response rates for intake or followup because of client refusal. Based on the information gleaned from projects’ applications, it is estimated that each woman will have four children with her in treatment.

In addition, as described in B.3, response rates will be maximized by maintaining contact with participants through social relationships and relatives as well as by the use of incentives for followup. CSAT will work with project directors and senior staff to assure that projects understand the need for accurate and timely followup of all clients.

Table B-1 summarizes the data collection schedule for each instrument, method of administration, by whom, and the source of the information.

Table B-1. Data Collection Schedule

Data Collection Instruments	Source of Data	Data Collector	Data Collection Method			Data Collection Time
			Interview	Self-Administered	Record Audit	Intake	30 days of Birth/ Mat. Intake	3-Mos	6-Mos	Discharge	6-Months Post-Discharge
Mother
Allen Barriers to Treatment Tool	Mother	Project Staff		▲		▲			▲	▲	▲
BASIS 24®	Mother	Project Staff	▲			▲			▲	▲	▲
Child Data Collection Tool	Mother (she asks others if DK)	Child Specialist, Medical Staff	▲		▲ (child’s)		▲
Child Well-Being Scales	Mother/child obs	Team/ Child Specialist	▲ (obs)				▲	▲	▲	▲	▲
Family Recovery Support Services Tool	Mother	Project Staff	▲		▲				▲	▲	▲
Ferrans & Powers Quality of Life Index	Mother	Project Staff	▲	▲		▲			▲	▲	▲
	Partner/ Child’s Father	Project Staff	▲	▲		▲				▲
Women’s Discharge Tool	Mother	Project Staff			▲ (mother’s)					▲
GPRA Client/Participant Outcome Measures	Mother	Project Staff	▲								▲
Women’s Medical Record Audit	Mother	Medical Staff			▲ (mother’s)		▲		▲	▲	▲
Child
Adolescent Childhood Dev. Assessment Guide	Child (11-17 yrs)	Child Specialist	▲ (obs)				▲	▲	▲	▲	▲
Children's Discharge Tool	Child (0-17 yrs)	Project Staff			▲ (child’s)					▲
Children’s Medical Record Audit	Child (3 mos-17 yrs)	Medical Staff			▲ (child’s)		▲(intake only)	▲	▲	▲	▲
CRAFFT	Child (age 11-17)	Project Staff	▲				▲	▲	▲	▲	▲
Denver Developmental Screening Inventory II	Child (mom) (0-6 yrs)	Child Specialist	▲ (obs)				▲	▲	▲	▲	▲
Middle Childhood Dev. Assessment Guide	Child (mom) (6-10 yrs)	Child Specialist	▲ (obs)				▲	▲	▲	▲	▲
Newborn’s Medical Record Audit	Child (< 3 months)	Medical Staff			▲ (child’s)		▲

NOTE. Interview = Staff elicit responses from client and complete tool. Self-administered = Clients who can read and write complete the tool on their own; otherwise staff administer tool. Record Audit = Staff elicit response from medical records. DK = Doesn’t know answer(s). Obs = Behavioral observation. Intake = mother’s intake GPRA.

Data for the Cross-site Assessment will be collected a maximum of five points in time: intake (or for children, within 30 days of the mother’s GPRA intake) or within 30 days of an infant’s birth; 3 months post-intake (for child instruments only); 6 months post-intake; discharge; and 6 months post-discharge. In some cases, the child’s primary residence may be with someone other than the mother, and the mother may not be able to provide the answer to all questions about a child. In these cases, program staff may support the mother in gathering that information from others in the child’s life. In many cases, this approach would have the added clinical benefit of helping the mother learn more about her child.

Data will be collected by intake workers, counselors, medical staff, child specialists, and/or other project staff, as is standard practice. In instances where participants are no longer in direct contact with the service provider, staff from the program will locate participants using a variety of tracking techniques. Followup interviews are conducted in-person since some instruments require observational techniques. (See Attachment I for sample tracking letters to be used in this Assessment.) CSAT will work closely with the projects to establish client tracking protocols, to assure maintenance of privacy during tracking, and to train project staff in tracking methodology using strategies that have worked effectively for GPRA data collection.

The instruments and associated interviewer instructions can be found in Attachment F. Blueprint tables in Attachments B and C list each assessment question, the source and rationale for use of each instrument, and the proposed data collection frequency.

Projects will be given TA and training in the administration of all aspects of the data collection procedures. CSAT and its contractors will provide training on all the instruments involved in the Assessment and assist projects in identifying appropriate persons to receive training. In addition, CSAT will provide for TA throughout the Assessment. For data abstraction from records review, CSAT will work with projects to develop procedures to abstract the required data elements from their existing files. In some cases, the instruments may be more appropriately administered in a language other than English. If such occasions arise, TA will be provided, if necessary, to ensure that foreign language assistance is available.

As shown in Table B-1, several different data collection methods will be used for the instruments, as follows:

• Behavioral observation supplemented by interview with the mother and/or child:

  • Child Well-Being Scales (mothers and all children) and

  • Denver Developmental Screening Inventory II (children ages 0 to 6).

• In-person interviews and/or paper and pencil questionnaires:

• Allen Barriers to Treatment Instrument (mothers),

• BASIS 24® (mothers),

• Child Data Collection Tool (mothers and medical record review),

• CRAFFT (ages 11 to 17),

• Ferrans and Powers Quality of Life Index (mothers and their partners and children’s fathers),

• Family Recovery Support Services Tool (mothers),

• Adolescent Childhood Developmental Assessment Guide (ages 11 to 17),

• Middle Childhood Developmental Assessment Guide (ages 6 to 10), and

• GPRA Client/Participant Outcome Measures (mothers).

• Staff review of client records:

  • Children’s Discharge Tool (children’s records, ages 0 to 17),

  • Women’s Discharge Tool (mothers’ records),

  • Children’s Medical Record Audit (children’s records, ages 3 months to 17 years),

  • Newborn’s Medical Record Audit (completed only at birth for infants born while the mother is in treatment), and

  • Women’s Medical Record Audit (mothers’ records).

2. Methods to Maximize Response Rates

Several methods will be used to retain participants, maximize response rates, and to optimize data completeness:

1. Clients will be asked to sign a consent or assent form during their orientation to the program. The intrinsic value of their participation in the data collection for their own treatment and for the future treatment of other women will be stressed.

2. Information will be gathered from the women on next of kin, close friends, or other emergency contacts. This information will be used when necessary to follow up on women who drop out of the program or who otherwise become difficult to reach and will be updated with each round of data collection, including discharge. Detailed contact information will be collected for each client and will be updated at each data collection stage.

3. Each client will be asked which contact methods are acceptable to her to arrange for the in-person followup interview, including best times to call, best days to call, and where the in-person interview can best take place. This information will be updated during all post-treatment contacts.

4. Clients will be reminded of the followup interview during the discharge interview. Project staff that makes regular contact with clients after discharge also will offer a reminder of the followup date at each contact.

5. Centralizing the data entry at one contractor will reduce the reporting burden on projects and improves the quality and completeness of data by allowing the contactor to resolve errors and inconsistencies in the data before the data set is finalized and analyzed.

6. Clients will be engaged in residential treatment to the maximum extent possible so that early dropout is minimized.

7. Cash equivalents will be offered by projects to participate as a means of retaining their cooperation with the followup data collection effort. Projects will be advised to offer a $20 cash equivalent to each participant who completes the followup data collection.

8. Transportation will be provided, if needed, for all clients who opt to complete the followup interview at the project address. If clients opt for an interview in a place other than the project location, project staff will transport themselves to that location.

9. Low-cost tracking procedures will be employed during the period between discharge and followup. GPRA tracking procedures will be used and have been shown to be effective.

CSAT will implement several strategies to assist the projects with followup activities. First, CSAT has conducted training for GPRA followup. The training program is designed to assist projects in learning about and conducting the followup at their sites and is offered to all projects. Individual project TA is made available for sites that need additional followup instruction. It is anticipated that these strategies will continue to improve the followup rates.

3. Tests of Procedures

• Allen Barriers to Treatment Instrument (mothers) ─ This tool has demonstrated good internal consistency reliability of (.87) and adequate face validity, content validity, and construct validity (Allen, 1994.) For the first cohort, the internal consistency was .92 for the overall instrument and .82 for the Treatment Program Characteristics Subscale, .75 for the Personal Feelings, Beliefs, or Thoughts Subscale, and .86 for the Specific Issues Subscale.

• BASIS 24® (mothers) ─ The psychometric properties of the BASIS 24® have been demonstrated for White, Latino, and African American clients in large inpatient and outpatient mental health and substance abuse treatment samples (Eisen, Gerena, Ranganathan, Esch, & Idiculla, 2006).¹⁹ Specifically, Chronbach’s alpha among these groups ranged from .87 to .91 for the overall summary score.

• Child Well-Being Scales (mothers and all minor children) ─ This widely-used, culturally sensitive scale has shown a canonical correlation of .72, indicating a high ability to discriminate neglectful from nonneglectful families (Eisen et al., 1999.) For the first cohort, the internal consistency reliability (Chronbach’s alpha) was .89.

• CRAFFT (ages 11 to17) ─ The documented reliability is .68 and criterion-related validity is .72 with strong scores for sensitivity (.80) and specificity (.86) (Knight et al., 2002.)

• Denver Developmental Screening Inventory II (ages 0 to 6 years) ─ The tool was normed on a sample of children who were delivered full-term and had no obvious developmental disabilities. The sample was diverse in terms of age, place of residence, ethnic/cultural background and maternal education. The test has good inter-rater and test-retest reliability (correlations of .90 or higher for most tests).

• Ferrans and Powers Quality of Life Index (mothers and their partners and children’s fathers) ─ Based on numerous studies, Chronbach’s alphas have ranged from .73 to .99 overall and .70 to .94 for the subscales. Test-retest reliability has also been good at .81 for a one-month interval. Face validity, content validity, and construct validity (by using factor analysis) have also been established (Ferrans & Powers, 1992). For the first cohort, the internal consistency was .94 for the overall instrument. The subscale alphas were adequate (i.e., .88 for Health and Functioning, .77 for Social and Economic, and .90 for .77 Psychological-Spiritual), except for the Family Subscale, which was low (.65).

A 6-month assessment with the 2003 cohort was recently concluded. To-date, all of the following instruments, except for the BASIS 24®, has been used. (The BASIS 32® was used instead of the BASIS 24®). Proposed modifications to these instruments were described in Section A.2. The following instruments are being used in the field by organizations other than SAMHSA:

• Allen Barriers to Treatment Instrument

• BASIS 24® Survey

• Child Well-Being Scales

• CRAFFT

• Denver Developmental Screening Inventory II

• Ferrans and Powers Quality of Life Index

The following instruments were developed for this Assessment. Their development is briefly described below:

• Child Data Collection Tool – Several iteration of this instrument were developed and shared with women’s treatment experts in the field. Initial drafts of this tool were updated based upon review and feedback from these experts. During administration to the 2003 cohort, questions, sensitivities to questions and more were addressed as well. The current submission of this instrument with this OMB statement has been refined based on all of the above. Preliminary scales developed using data from the 2003 cohort revealed Chronbach’s alphas from .66 to .78 for the following subscales: Educational (.66), Socioeconomic Status (.72), Legal (.74), Parental Relationships (.75), and Spiritual (.78).

• Family Recovery Support Services Tool – This tool was developed to capture recovery support services provided to target members specified in the 2006 RFA which included women, their children, and family members. The family members include fathers of the children and partners of the women, as well as extended family members of the women and children in treatment. Instrument development involved considerable input from experts in the field of providing substance abuse treatment to PPW and their minor children and experts in recovery support services. Such review included independent review, meetings, and a focus group. Response categories were based on the Women’s and Children’s Discharge Tools.

• Middle Childhood Developmental Assessment Guide and Adolescent Childhood Developmental Assessment Guide – These assessment Guides were developed and validated with four interdisciplinary panels of experts in infant, child, and adolescent health in the “Bright Futures” program and sponsored by Maternal and Child Health Bureau, Health Resources and Services Administration and the Center for Medicaid and State Operations, Health Care Financing Administration. In addition, they were reviewed by approximately 1000 child health practitioners, educators, and child health advocates across the country. These practitioners validated the interdisciplinary panel’s perspective on the reliability and validity of these measures for this population. Content validity has been established. The Middle Childhood Developmental Assessment Guide was used successfully with the 2003 cohort. The Adolescent Childhood Developmental Assessment Guide has not yet been administered as part of this Assessment because no adolescents were admitted into the study during the first cohort.

• Women’s Discharge Tool and Children’s Discharge Tool – These instruments are based on standardized discharge questions/categories that are gleaned from a records review by project staff. They were developed specifically for this Assessment based on standard items typically collected at discharge. Data are based on staff review of clients’ medical records. These instruments were pre-tested on nine participants and successfully used with the 2003 cohort.

• Women’s Medical Record Audit, Children’s Medical Record Audit, and Newborn’s Medical Record Audit – These instruments are based on elements that are typically found in medical records as part of standard medical practice. Responses are gleaned from a medical records review by medical staff. These were divided into three separate instruments, based on experience with the first cohort. The overall length of the instrument has not changed, but given the different data collection schedule of different participants, it is believed that this division of the instruments makes administration much easier and less confusing. In addition, based on the pattern of responding noted among the 2003 cohort, CSAT is reducing the number of response categories on the Women’s instrument to match those on the Children’s instrument.

4. Statistical Consultants

The individuals listed below reviewed statistical aspects of this Assessment and any differences were reconciled. The names and phone numbers of the statistical consultants are as follows:

Statistical Consultants for the PPW Cross-site Assessment
Name	Address	Contact Information
Karen Allen, Ph.D., R.N., FAAN	Andrews University 4137 Courtney Street, St. Joseph, MI 49085	Phone: 269-471-3364 [email protected]
Wendy Kissin, Ph.D. Senior Analyst	Westat, 1650 Research Boulevard Rockville, MD 20850	Phone: 301-294-3885 [email protected]
Andrea Kopstein, Ph.D. Lead Social Science Analyst	CSAT, Division of Services Improvement, 1 Choke Cherry Rd., Room 5-1103, Rockville, MD 20857	Phone: 240-276-1575 Andrea.kopstein@samhsa.hhs.gov
Project Officer for the PPW Cross-site Assessment
Linda White Young, M.S.W., LICSW Public Health Advisor	CSAT, Division of Services Improvement, 1 Choke Cherry Rd., Room 5-1118, Rockville, MD 20857	Phone: 240-276-1581 [email protected]

ATTACHMENTS

Attachment A: PPW Request for Application (RFA)

Attachment B: PPW Blueprint for Women and Their Families

Attachment C: PPW Blueprint for Children

Attachment D: Cited Literature for Women and Their Families

Attachment E: Proposed Modifications to Instruments

Attachment F: Data Collection Instruments

F-1 Allen Barriers to Treatment Instrument

F-2 BASIS 24®

F-3 Child Well-Being Scales

F-4 CRAFFT

F-5 Denver Developmental Screening Inventory II

F-6 Ferrans and Powers Quality of Life Index

F-7 GPRA Client/Participant Outcome Measures

F-8 Child Data Collection Tool

F-9 Middle Childhood Developmental Assessment Guide

F-10 Adolescent Childhood Developmental Assessment Guide

F-11 Women’s Discharge Tool

F-12 Children’s Discharge Tool

F-13 Women’s Medical Record Audit

F-14 Children’s Medical Record Audit

F-15 Newborn’s Medical Record Audit

F-16 Family Recovery Support Services Tool

Attachment G: Participants in Design Process

Attachment H: Consent and Assent Forms

Attachment I: Sample Client Tracking Letters

1 Clark, K., Dee, D., Bale, P., and Martin, S. (2001). Treatment compliance among prenatal care patients with substance abuse problems. American Journal of Drug and Alcohol Abuse, 27(1), 121-136.

2 See http://www.drugabuse.gov/drugpages/

3 See http://www.drugabuse.gov/drugpages/

4 http://www.cdc.gov/MMWR/preview/mmwrhtml/mm5350a4.htm

5 Martin, J.A., Hamilton, B.E., Sutton, P.D., Ventura, S.J., Menacker, F., & Kirmeyer, S. (Sept., 2006). Births: Final data for 2004. National Vital Statistics Reports, 2006, 55(1).

6 Arria AM, Derauf C, LaGasse L et al. (2006). Methamphetamine and other substance use during pregnancy: Preliminary estimates from the infant development, environment, and lifestyle (IDEAL) study. Maternal and Child Health Journal; 10(3); 293-302.

7 See http://www.oas.samhsa.gov/NSDUH/2k5nsduh/tabs/Sect7peTabs68to75.pdf

8 See http://www.oas.samhsa.gov/NSDUH/2k5nsduh/tabs/Sect7peTabs68to75.pdf

9 Arria AM, Derauf C, LaGasse L et al. (2006). Methamphetamine and other substance use during pregnancy: Preliminary estimates from the infant development, environment, and lifestyle (IDEAL) study. Maternal and Child Health Journal; 10(3); 293-302.

10 Aneja, A., Igbal, M.M., & Ahmed, K. (2006). The effects of amphetamine use during pregnancy and lactation. Directions in Psychiatry, 26(3); 237-251.

11 White, R., Thompson, M., Windsor, D., Walsh, M., Cox, D., & Charnaud, B. (June 2006). Dexamphetamine substitute-prescribing in pregnancy: A 10-year retrospective audit. Journal of Substance Use, 11(3); 205-216.

12 Infant Mortality and Low Birth Weight Among Black and White Infants – United States, 1980-2000. (July 2002). Center for Disease Control MMWR Weekly, 51(27); 589-92. See http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5127a1.htm

1³ www.cdc.gov/ncbddd/fas/fassurv.htm

13 McNichol, T., and Tash, C. (2001). Parental substance abuse and the development of children in family foster care. Child Welfare, 80(2), 239-257.

14 Clark, K., Dee, D., Bale, P., and Martin, S. (2001). Treatment compliance among prenatal care patients with substance abuse problems. American Journal of Drug and Alcohol Abuse, 27(1), 121-136.

15 There is evidence that on a broad scale, treatment improves the quality of life. In 1990, the National Institutes of Health working group framed a working definition of “Quality of Life” in the context of treatment for alcoholism and addiction. The three areas are clinical status (or health), negative aspects of life from the client’s perspective, and social functioning and environmental support. Ferrans, C. (1990). Development of a Quality of Life Index for patients with cancer. Oncology Nursing Forum, 17, 15-21.

16 Years and days are specified for the Denver II and the Middle Childhood Developmental Assessment Guide to indicate at what specific age the Assessment changes from using the former to the latter with a child age 6. The reason for this level of precision is that the Denver II is standardized for use with children age 6 and under, and that children of this age should have matured sufficiently to be able to complete all Denver II tasks.

17 The Urban Institute. (2003). “Finding Out What Happens to Former Clients,” Series on Outcome Management for Nonprofit Organizations.

18 http://www.drugabusestatistics.samhsa.gov/NHSDA/2k1NHSDA/vol2/attachmentc.htm

19 Eisen, S.B., Gerena, M., Ranganathan, G., Esch, & Idiculla, T. (2006). Reliability and Validity of the BASIS-24© Mental Health Survey for Whites, African-Americans, and Latinos. The Journal of Behavioral Health Services & Research, 33(3,) 304-323.

24