FASD_DxInt Supporting_Statement3.25.2010

SAMHSA’s Fetal Alcohol Spectrum Disorders (FASD) Center for Excellence

FASD Diagnosis and Intervention Project

THE SUPPORTING STATEMENT

A. Justification

1. Circumstances of Information Collection

Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Prevention (CSAP) requests Office of Management and Budget (OMB) approval for the data collection on the Fetal Alcohol Spectrum Disorders (FASD) Center for Excellence FASD Diagnosis and Intervention project. The purpose of the FASD Center for Excellence (CFE) is to prevent alcohol-exposed pregnancies among women of childbearing age and pregnant women, and improve the quality of life for individuals affected by FASD.

This data collection effort requests clearance for the following eleven data collection instruments (Attachment I).

1. Screening and Diagnosis Tool

2. Positive Monitor Tracking

3. Services Child Is Receiving at the Time of the FASD Diagnosis

4. Services Planned and Provided Based on the Diagnostic Evaluation

5. Services Delivery Tracking

6. End of Intervention/Program Improvement Measure: Case worker

7. End of Intervention/Program Improvement Measure: Parent

8. End of Intervention/Program Customer Satisfaction with Service

9. Outcome Measures: Children Ages 0–7

10. Outcome Measures: Children/Adolescents Ages 8–18

11. Client Participation Tracking Form

The FASD Center for Excellence (CFE) has been established as a result of legislative mandates from the Children's Health Act of 2000 (P.L. 106-310) The data outlined in this document will enable the SAMHSA FASD Center for Excellence to monitor the delivery and quality of the services provided. Section 501 (d)(4) of the Public Health Service Act requires that the Secretary of DHHS, acting through the Administrator shall “. . . assure that the Administration conduct and coordinate demonstration projects, evaluations, and service system assessments and the activities necessary to improve the availability and quality of treatment, prevention, and related services”.

The purpose of this data collection effort is to monitor the integration of the Diagnosis and Intervention projects into existing service delivery organizations. The FASD Center for Excellence will integrate the Diagnosis and Intervention projects at eight sites. A description of the intervention project and the data collection forms to be used is described below. In integrating this intervention the Center for Excellence intends to monitor and collect process level information on screening criteria used, type of FASD diagnosis, and type and amount of service provided to children. In addition improvement in performance will be measured at different time points by calculating the percent increase or decrease in a school attendance, expulsions, housing stability, and placement stability.

The Diagnosis and Intervention project is designed to screen children for an FASD, provide diagnostic evaluations to children who screen positive, and provide integrated intervention services to children diagnosed with an FASD. The goal of the FASD Diagnosis and Intervention project is to foster developmentally appropriate behaviors and improve overall quality of life in children/adolescents affected by an FASD and their families. The sites included would be state agencies, juvenile court systems, and local community programs that serve children age birth to 18, many of whom are at risk of having been exposed to alcohol in-utero.

All of the eight subcontractors work with vulnerable minors whose needs are met by many different service delivery systems. Thus, agencies have established practices to keep the individual level information of these children private. Proceedings and records in juvenile court (child protection and delinquency) cases are kept private in most jurisdictions. Agencies and courts are charged with adhering to Federal, State, and Tribal HIPAA Privacy regulations relating to consent for services. Each agency protects to the extent of the law of those who seek FASD services.

Comprehensive literature reviews performed by the FASD Center for Excellence reveal a dearth of evidence-based practices in the field of FASD diagnosis and treatment. As a result, there will be some variability in the FASD diagnostic criteria used among the eight sites. Children and adolescents (birth to 18 years) who enter the service delivery system at each of the sites will be administered the Screening Form (Form A). All sites will use the same criteria when screening children /adolescents for an FASD based on the criteria approved by the Expert Panel on FASD Screening in Juvenile Courts. Requirements for a positive FASD screen include such criteria as confirmed prenatal exposure, a face rank of 3 or 4 using FAS Photographic Screening Tool, existence of a sibling who has previously been diagnosed with an FASD, or has a birth mother with confirmed drug or alcohol history at some point other than pregnancy, and CNS or developmental abnormality and note in medical record indicating dysmorphia (Form A). Children and adolescents with a positive screening result will be referred for a full diagnostic evaluation. Sites will use one of four existing criteria to diagnose children/adolescent with an FASD: the 4-digit diagnostic code developed by the Washington State FAS Diagnostic and Prevention Network; MN Diagnosis criteria; Institute of Medicine Guidelines; or CDC Guidelines (Form A). Children ages birth through 3 who do not fulfill the screening criteria will be placed in a positive monitor category (Form B).

All sites are responsible for developing an individually tailored service delivery plan once children/adolescents are diagnosed with an FASD. Toward this end, sites’ case managers will document the services children/adolescents are receiving at the time they are diagnosed (Form C). Services planned and provided based on the results of the FASD diagnostic evaluation will be recorded (Form D). Process information such as session dates, frequency of rescheduled appointments, and barriers to services (lack of transportation or childcare, etc.) will also be recorded throughout the course of the intervention for each child/adolescent (Form E).

Outcome data will be collected by case managers at multiple points throughout the intervention. All children who are diagnosed will be assessed (1) at baseline, within 30 days of developing the service delivery plan; (2) every six months after the service plan is implemented; (3) at the end of probation for adjudicated children/adolescents and at the end of the intervention for all other children/adolescents; and (4) six and twelve months after the end of the intervention. Outcome measures will vary according to the child/adolescent’s age, with separate questionnaires for participants ages 0–7 (Form I) and participants ages 8–18 (Form J). The outcome measures include school attendance, school disciplinary measures, number of residential placement changes, change in level of placement, and adjudication measures. Sites that work with children in preschool or day care will track the number of preschool or day care placement changes every 6 months.

Other end of intervention outcome measures include a brief form to record the case manager’s perception of the child/adolescent’s improvement as a result of receiving services for an FASD (Form F); an identical form that records the parent or guardian’s perception of the child/adolescent’s improvement (Form G); and a customer satisfaction form that will be completed by the child/adolescent’s parent or guardian (Form H).

Client attrition from the program and any known reasons for attrition will be recorded using a Client Participation Tracking form (Form K).

The integration of the FASD Diagnosis and Intervention project at each of the eight sites will advance the field of FASD by learning more about the screening and intervention services that work for specific high-risk populations in juvenile court systems, state, and local settings. As is consistent with federal efforts to address issues of accountability, capacity, and effectiveness, data collection is designed to monitor the integration of the FASD Diagnosis and Intervention project by measuring changes in developmentally appropriate behavioral domains (placement stability in daycare or pre-school programs, consistent school attendance, lack of school disciplinary incidents, etc.) at different time points as children/youth progress through the intervention.

2. Purpose and Use of Information

The practical utility of this new project is to collect data from sites to monitor the integration of FASD screening and diagnostic evaluations, as well as intervention services provided to children/adolescents ages birth to18 who are diagnosed with an FASD.

Purpose of Data Collection

The basic purposes of data collection are to 1) determine whether the interventions were provided and document the details of the intervention as they are integrated into service delivery, and 2) assess the extent to which clients achieved the desired outcomes (consistent school attendance, placement stability within day care or preschool programs, etc.). In order to assess these broad goals, specific questions have been devised and are provided below:

• How many children/adolescents enter the program and are identified as needing a diagnostic evaluation based on a positive screening result?

• How many children/adolescents are diagnosed with an FASD?

• How many children/adolescents participate in FASD intervention services?

• What intervention services are most commonly provided to children/adolescents diagnosed with an FASD? What are the most commonly recommended service frequencies and quantities?

• What are the most common barriers to service provision, and to what extent do these barriers affect service delivery?

• How many children/adolescents participate in end of intervention and follow-up outcome measures?

• What is the case managers’ and parents/guardians’ perception of client improvement as a result of participating in FASD services?

• To what extent are positive results that are achieved at the end of the intervention maintained at six- and twelve-month follow-up?

• Is the project being implemented as intended (fidelity)?

GPRA Measures

For this project, National Outcome Measures (NOMs) will be collected for the following domains: Access/Capacity, School Attendance, and Stability in Housing.

Dissemination

Findings from this data collection effort will be disseminated in a way that addresses the various needs of multiple stakeholders and maximizes the multiple uses to which the findings may apply. The findings from this data collection effort can only be replicated at similar settings and population groups. The findings cannot be generalized to the population at large or to populations of children /adolescents. Since this program is being integrated at a setting where children are treated for FASD, its findings are representative of only that setting. Findings from this study will be disseminated to multiple stakeholders. Stakeholders with active interests in the FASD Center for Excellence include:

• FASD program staff in CSAP charged with implementing the program, monitoring adherence, and quality improvement;

• Other federal agencies with substantive or evaluative interests overlapping with those SAMHSA such as the Centers for Disease Control and Prevention (CDC) and the National Institute of Alcohol Abuse and Alcoholism (NIAAA);

• State leaders for development and implementation of FASD State Plans;

• Juvenile Courts: National Council of Juvenile and Family Court Judges and Office of Juvenile Justice and Delinquency Prevention.

• The CFE Expert Panel; and

• Individuals with FASD, families, service providers and researchers concerned about appropriate and timely prevention and intervention that is delivered in a culturally competent and inclusive manner.

3. Use of Information Technology

Data will be collected directly by staff at each of the participating sites during face-to-face interaction with children/adolescents. The case managers administering the forms will enter responses onto paper forms or directly into a standardized web-based database to reduce burden on staff and data will be stored electronically in secured files. All data will be kept private through standard procedures to protect privacy including unauthorized access. Access to data on the server is username and password protected. Burden for data collection at sites is eased by the use of a standard database for data entry which is web based.

4. Efforts to Identify Duplication

The data collection proposed for this evaluation is not available elsewhere, is not duplicative, and is seen as critically valuable for assessing the effectiveness of the FASD Diagnosis and Intervention project. It is important to stress that individual respondents are not being asked to provide information which has been collected elsewhere.

5. Involvement of Small Entities

There is no significant involvement of small entities.

6. Consequences if Information Collected Less Frequently

All participants in the site’s programs are screened for an FASD; participants with a positive screening result are referred for a diagnostic evaluation. Intervention services are provided for children who are diagnosed with an FASD. Service delivery and changes in developmentally appropriate behavioral domains are tracked throughout the course of the intervention. Behavioral changes are also assessed every six months once the service plan is implemented, at the end of the intervention, and six and twelve months after the end of the intervention. The reason for tracking all process related data provided at the intervention and other outcome data every six months through the intervention rather than at the beginning and end of intervention is to ensure that data is available on participants all through the intervention up to the point services are provided. We do not anticipate losing many participants or collecting information less frequently because interventions services for FASD are not widely available. Since the intervention plan and service delivery is individualized based on each child’s diagnosis, parents understand the importance of receiving such service.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on November 17, 2009, Vol. 74, No. 220, Pages 59190-59191. No written comments were received.

Below are summaries of the outside consultations (separate from the publication of the notice in the Federal Register) regarding proposed data collection instruments.

• First, the contractor developed initial draft versions of the forms. The initial draft was developed by reviewing existing data collection forms used for similar types of intervention. The individual sites provided information of data collections forms used in their settings. Forms were then finalized to ensure information was clearly presented and collected in least burdensome manner.

• Second, the SAMHSA FASD Center for Excellence sought consultation outside of the agency from its evaluation subcontractor at the Human Services Research Institute while developing the proposed forms. The evaluation subcontractor incorporated measures that were necessary to meet federal reporting requirements and assisted with the development of data collection procedures to minimize respondent burden while maximizing the collection of data. Evaluators at the Human Services Research Institute who were consulted while developing the instruments include:

Virginia Mulkern, Ph.D

Senior Vice President

(617) 876-0426 ext. 2315

Deborah Potter, Ph.D

Senior Research Specialist/Subcontractor Project Director

(617) 844-2506

Sandra Richman, MSW

Research Analyst III

(773) 484-7939

9. Payment to Respondents

Respondents will not receive additional monies to collect or report these data.

10. Assurance of Confidentiality

SAMHSA retains final authority to conclude whether or not the human intervention activities fall under the regulations protecting human subjects. The law 45 CFR 46 101(c) allows for SAMHSA agency heads to “adopt such procedural modifications as may be appropriate from an administrative standpoint” to waive IRB review. Such modification has occurred (IRB requirement was waived by SAMHSA). SAMHSA concurred that these projects are not research projects, but programs that add FASD prevention services to existing service delivery organizations. Hence consent for participation in the FASD program is obtained as part of the services provided by each of these service delivery organizations and the consent process is as described. Each client is administered the screening form to determine eligibility in the program.  If eligibility is confirmed, the client is provided with information about the program and invited to participate.  Each program uses the consent form to go over the program components, expectations, and assurances.  If the client agrees to participate, the form is signed by the client and the clinical supervisor.

Nonetheless, participation in this intervention is voluntary. The minimal requirement for the participants would be to respond to questions on the screener. All other forms are voluntary and not responding to the questions on those forms does not affect participation. The client service agreement form and the consent form administered by each of the sites inform respondents about the purpose of data collection, that their information will be kept private and that they are free to skip any question that they do not wish to answer. Each of the participating sites has informed consent documents (Attachment II) that are slightly different because they are integrating the Diagnosis and Intervention program into their existing service delivery systems. To further ensure privacy of individual responses, all data will be reported at the aggregate level so that individual responses cannot be identified; no data will be reported at the individual participant level.

Furthermore, data will be collected to meet the criteria of a “limited data set” as defined in the Privacy Regulations issued under the Health Insurance Portability and Accountability Act (HIPAA), (HIPAA Privacy Rule, 45 C.F.R. _ 164.501) [45 C.F.R. 164.514(e)(4)(ii)]. Each of the sites will sign a Data Use Agreement with the Center for Excellence so that they may share data on clients participating in the program with the Center for Excellence and the evaluation subcontractor. As outlined in the HIPAA regulations, this agreement will allow the subcontracts to disclose “a limited data set” to be used for this project. The limited data set will protect the privacy of the clients since it will exclude the release of all of the following types of data:

• Names

• Postal address information, except town or city, State, and zip code

• Telephone numbers

• Fax numbers

• Email Addresses

• Social Security Numbers

• Medical record numbers

• Health plan beneficiary numbers

• Account numbers

• Certificate/ license numbers

• Vehicle identifiers and serial number, including license plate numbers

• Web URLs

• IP addresses;

• Biometric identifiers, including finger and voice prints

• Full face photographic images and any comparable images

Data which are permitted with a Data Use Agreement and which will be collected include:

1. Dates such as admission, discharge, service, DOB

2. State of residence

3. Age in years, months or days

Any data which could be used to identify an individual will not be released by the Center for Excellence or its evaluation subcontractor. In order to further ensure privacy and comply with HIPAA, both the Center for Excellence and its evaluation subcontractor will use the information in a manner that is consistent with the Data Use Agreement and will employ safeguards to ensure that respondent information is protected. The contractor and its evaluation subcontractor will design reporting templates for the grantees that explicitly exclude all protected information and only include data to be reported for the program. Should any information which is not covered by the Data Use Agreement be released, the violation of HIPAA will be reported to SAMHSA. No direct identifiers will be included in order for the data to be considered a “limited data set.” A summary of the actions which will be taken in order to comply with HIPAA follows:

When creating a unique identification code, ensure that the code does not contain information that can be used to identify the individual.

The data collected by the sites constitutes a limited data set. A data use agreement can therefore be used while collecting data for evaluation purposes.

11. Questions of a Sensitive Nature

Many questions in these surveys are sensitive in nature, as they ask about children’s psychiatric diagnoses, academic records, and use of health services. In addition, given the cultural belief that drinking while pregnant is harmful to the developing fetus, parents may find it risky to disclose this information when their children are being screened for an FASD. The purpose of collecting this sensitive information is to address substance abuse during pregnancy; the Federal Government has already established its interest in collecting this type of data. Only data that are necessary for monitoring progress of the project will be collected.

Respondents will participate and use standard informed consent processes which are already in place at the participating agencies. Since all of the participating agencies have informed consent documents (Attachment III) that they currently use in their programs, slightly different consent forms will be used by each participating agency.

12. Estimates of Annualized Hour Burden

The first component of the total hourly costs is the estimated wages of participating clients. Because all clients will be under the age of 18, case managers and other service providers will complete questionnaires on their behalf. The cost of service providers’ time is covered by the subcontract funds that sites were awarded from SAMHSA (section 14); therefore, the hourly wage costs and total hourly costs of forms that are completed on behalf of children are listed as “N/A” in the burden hours table (Forms A, B, F, I, and J).

Children’s parents or current legal guardians will fill out two brief outcome measures at the end of the intervention (Forms G and H). Because the eight sites tend to serve low-income populations, we estimate that approximately 80% of clients will be unemployed and that the remaining 20% will receive minimum wage (currently $7.25/hour).

Four additional forms (Forms C, D, E, and K) are provider forms that are completed by case managers. While these forms are listed in the “Estimated annualized burden hours table,” the estimated costs of completing this form are not factored into the total hourly estimates. Rather, these costs (which include SAMHSA salaries and contractors’/evaluators’ salaries) are included in the federal contract costs, which are outlined separately in section 14.

Estimated annualized burden hours (Age Groups Separated)

Instrument / Activity		No. of Respondents	No. of Responses per Respondent	Total Number of Responses	Average Burden per Response	Total Burden Hours per Collection	Hourly Wage Cost	Total Hourly Cost ($)
	Screening Measures: Children 0–7
Form A. Screening and Diagnosis Tool		1400	1	1400	.17	238	N/A*	N/A
Form B. Positive Monitor Tracking		450	1	450	.03	14	N/A*	N/A
	Service Delivery Measures: Children 0–7
Form C. Services Child Is Receiving at the Time of the FASD Diagnosis		750	1	750	.17	128	N/A*	N/A
Form D. Services Planned and Provided Based on the Diagnostic Evaluation		750	1	750	.33	248	N/A*	N/A
Form E. Services Delivery Tracking		750	12	9000	.08	720	N/A*	N/A
	Outcome Measures: Children 0–7
Form F. End of Intervention/Program Improvement Measure: Case worker		750	1	750	.02	15	N/A*	N/A
Form G. End of Intervention/Program Improvement Measure: Parent		750	1	750	.02	15	$7.25	$87.00
Form H. End of Intervention/Program Customer Satisfaction with Service		750	1	750	.03	23	$7.25	$133.40
Form I. Outcome Measures (children ages 0–7)		750	5	3750	.08	300	N/A*	N/A
	Other Measures: Children 0–7
Form K. Lost to Follow-up (Client Participation Tracking form)		135	1	135	.03	4	N/A*	N/A
	Screening Measures: Children/Adolescents 8–18
Form A. Screening and Diagnosis Tool		100	1	100	.17	17	N/A*	N/A
	Service Delivery Measures: Children/Adolescents 8–18
Form C. Services Child Is Receiving at the Time of the FASD Diagnosis		50	1	50	.17	9	N/A*	N/A

Form D. Services Planned and Provided Based on the Diagnostic Evaluation		50	1	50	.33	17	N/A*	N/A
Form E. Services Delivery Tracking		50	12	600	.08	48	N/A*	N/A
	Outcome Measures: Children/Adolescents 8–18
Form F. End of Intervention/Program Improvement Measure: Case worker		50	1	50	.02	1	N/A*	N/A
Form G. End of Intervention/Program Improvement Measure: Parent		50	1	50	.02	1	$7.25	$5.80
Form H. End of Intervention/Program Customer Satisfaction with Service		50	1	50	.03	2	$7.25	$11.60
Form J. Outcome Measures (children/adolescents ages 8–18)		50	5	250	.08	20	N/A*	N/A
	Other Measures: Children/Adolescents 8–18
Form K. Lost to Follow-up (Client Participation Tracking form)		15	1	15	.03	1	N/A*	N/A
TOTAL		7,700	49	19,700	--	1821	--	$237.80

*costs included in section 14

Estimated annualized burden hours (Age Groups combined (0-18years))

Instrument / Activity		No. of Respondents	No. of Responses per Respondent	Total Number of Responses	Average Burden per Response	Total Burden Hours per Collection	Hourly Wage Cost	Total Hourly Cost ($)
	Screening Measures: Children 0–18
Form A. Screening and Diagnosis Tool		1500	1	1500	.17	255	N/A*	N/A
Form B. Positive Monitor Tracking		450	1	450	.03	14	N/A*	N/A
	Service Delivery Measures: Children 0–18
Form C. Services Child Is Receiving at the Time of the FASD Diagnosis		800	1	800	.17	137	N/A*	N/A
Form D. Services Planned and Provided Based on the Diagnostic Evaluation		800	1	800	.33	265	N/A*	N/A
Form E. Services Delivery Tracking		800	12	9600	.08	768	N/A*	N/A
	Outcome Measures: Children 0–18
Form F. End of Intervention/Program Improvement Measure: Case worker		800	1	800	.02	16	N/A*	N/A
Form G. End of Intervention/Program Improvement Measure: Parent		800	1	800	.02	16	$7.25	$92.80
Form H. End of Intervention/Program Customer Satisfaction with Service		800	1	800	.03	25	$7.25	$145.0
Form I and J. Outcome Measures (children ages 0–18)		800	5	4000	.08	320	N/A*	N/A
	Other Measures: Children 0–18
Form K. Lost to Follow-up (Client Participation Tracking form)		150	1	150	.03	5	N/A*	N/A
TOTAL		7,700	25	19,700	-	1821	-	$237.80

13. Estimates of Annualized Cost Burden to Respondents

There are no capital, start up, operational, or maintenance service costs related to this data collection activity.

14. Estimates of Annualized Cost to the Government

SAMHSA/CSAP has planned and allocated resources for efficient management and effective use of the information to be collected, including the processing of the information in a manner which will enhance, where appropriate, the utility of the information to agencies and the public. The total cost of this data collection effort, being performed under a Task Order, is $ 1,671,175.80.

SAMHSA/CSAP estimates the GS-15 Government Project/Task Order Officer (GPO/TOO) principally involved in the oversight and analysis of this contracted evaluation will spend on average approximately 0.5% of her time (0.25 hours weekly) overseeing various components of this project. On an annualized basis this would be the equivalent of $756 in federal employee personnel costs (based on an annualized GS-15 salary of $121, 000).

Therefore, the contract costs and the personnel costs associated with data collection, including SAMHSA and project specific costs (but not indirect costs associated with the participation of clients), are expected to total $1,671,931.80

15. Changes in Burden

This is a new project.

16. Time Schedule, Publication and Analysis Plans

The current contract for the FASD Diagnosis and Intervention project ends 5/31/2012. A report is to be delivered by this date to CSAP. The Table below outlines the project schedule with timelines for data collection, analysis, report delivery and presentation to stakeholders.

Activity Planned Start Time

Train programs on submitting data to Contractor	OMB approval + 2 weeks
Transmit baseline data from FASD screening and diagnosis	OMB approval + 3 weeks
Transmit monthly process and outcome data	OMB approval + 4 weeks
Develop quarterly data reports	OMB approval + 12 weeks to coincide with project reporting periods, each year, of: February 1 May 1 August 1 November 1
Submit data for Subcontractor to produce final report and recommendations	Submit data to Contractor and Subcontractor by 1/15/2012 (Subcontractor’s report due 3/30/2012)
Deliver Final Files	End Of Contract
Send project documentation	End Of Contract

Analysis Plan

Most survey items are closed-ended questions. Respondents will be given the opportunity to add clarifying comments regarding their responses. These will be written down by trained case managers or other service providers.

Construction of measures

Specific measures will be constructed using items from the relevant questionnaires, as described below. Many process and outcome measures will be reported as frequencies.

• The number of children/adolescents with a positive screening outcome will be calculated using question 10 in the “Screening and Diagnosis Tool” (Attachment I.A).

• The number of children/adolescents diagnosed with an FASD will be calculated using question 16 in the “Screening and Diagnosis Tool.”

• The number of children/adolescents who participate in FASD intervention services will be calculated by counting the number of completed “FASD Services Planned and Provided based on the Diagnostic Evaluation” forms (Attachment I.D) that include at least one service component for which the survey item “amount of service units provided” is greater than 0.

• Frequencies of the intervention services most commonly provided to clients will be calculated using the “service component” item in the form “FASD Services Planned and Provided based on the Diagnostic Evaluation” (Attachment I.D).

• Frequencies of the survey item “reason service was not provided as planned,” which is asked in the “Services Delivery Tracking” form (Attachment I.E), will be run in order to understand the most common barriers to service provision and the extent to which these barriers impact service delivery.

• Case managers’ and parents’ perception of client improvement will be assessed by running frequencies of the question about overall improvement that is asked in the “End of Intervention/Program Improvement Measure” (Attachments I.F and I.G); parents’ overall satisfaction with the services received will be assessed by running frequencies of the satisfaction question in the “Customer Satisfaction with Service” form (Attachments I.H).

• The number of outcome measures forms (Attachments I.I and I.J) completed at each of the designated assessment periods will be counted in order to calculate the number of children/adolescents who complete the end of intervention, 6-month follow-up, and 12-month follow-up assessments.

• The number of school days attended, the number of housing and school placement changes, and the number of petition offenses and adjudicated charges will be summed as relevant for each child and then aggregated in order to report the extent to which positive results that are achieved at the end of the intervention are maintained at six-month and twelve-month follow-up among the intervention population (Attachments I.I and I.J).

Statistical Analyses

Descriptive analyses (frequencies) will be used to understand the basic demographic characteristics of the children in the programs, as well as the characteristics of the interventions received (types of services). Furthermore, the mean number of all services each child received will be calculated, as well as the mean number of specific types of services.

In addition, SAMHSA requires data on the NOMS variable to be collected at different time points during the intervention in order to determine change over time. The purpose of using the inferential statistics in not to draw any conclusions about the effectiveness of the intervention but to determine if there was a decrease/increase in the percentage in key measures like school attendance, housing stability, or placement stability.

The sample table below illustrates some of the inferential data analyses about the change in percentage over time and differences between a state program and community program.

		Number of preschool/day care placements in the past six months for children ages 0–4 per 100 children*
	t-test	Baseline response	Post-Intervention response
State Agencies
Community Programs

*Children with school changes because of a required move will be excluded from the count (e.g., child aged out of the program).

Fidelity

The extent to which the programs have integrated FASD screening, diagnosis, and intervention services into their existing programs will be assessed through qualitative analysis of the sites’ implementation plans and/or the program manuals (secondary data which does not require additional data collection from the programs.).

Unique Identifier

Each client will be assigned a unique identifier that meets HIPAA requirements for privacy. Most sites will adopt the subcontractor’s proposed convention for a multi-level identification number in the format of “xxxzzz-yyyy” (with “xxxzzz” being a randomly generated 6-digit number and “yyyy” being the “check-digits” which are the sum of xxx+zzz to guard against incorrect data entry of follow-up data), such as 123456-0579. In practice, the lists of unique identification numbers generated by the subcontractor may be sub-divided within the participating agencies so that individual providers are assigned specific numbers to guard against clerical errors. Sites that elect to use an alternate strategy will select a format which conforms to HIPAA requirements.

17. Display of OMB Expiration Date

The expiration date for OMB approval will be displayed.

18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Eight sites from across the U.S. have been selected to integrate the Diagnosis and Intervention projects into its current service delivery system (See Attachment III for a complete list of participating sites). The sites were selected based on a proposal submission and review process. The selection criteria for Juvenile Courts, Local, and State agencies were slightly different. The review of proposal was conducted by the review team using the following selection criteria for Juvenile Courts: 1) Juvenile Court Capacity and Need 2) Experience 3) Target Population 4) Capacity of the proposed FASD Diagnostic Center or Team 5) Task Force and Needs Assessment Plans 6) Strategy/Sustainability 7) Brief Implementation Plan 8) FASD Screening, Diagnosis, and Intervention 9) Evaluation and Data Collection Plan 10) Experience 11) Project Management. The review of proposal was conducted by the review team using the following selection criteria for Local agency: 1) Program Capability and Need 2) Experience of Organization 3) Population Size and Age Groups 4) FASD Diagnosis and Intervention 5) Task Force and Needs Assessment Plans 6) Strategy and Sustainability 7) Brief Implementation Plan 8) FASD Screening, Diagnosis, and Intervention 9) Evaluation and Data Collection Plan 10) Experience 11) Project Management. The review of proposal was conducted by the review team using the following selection criteria for State agency: 1) Program Capability and Need /Experience of Organization 2) Population Size and Age Groups 3) Local Service Organization(s) 4) FASD Diagnosis and Intervention 5) Task Force and Needs Assessment Plans 6) Strategy and Sustainability 7) Brief Implementation Plan 8) FASD Screening, Diagnosis, and Intervention 9) Evaluation and Data Collection Plan 10) Experience and 11) Project Management. Eight sites were chosen because they received acceptable scores on the selection criteria. All children who are seeking care through the selected sites will be screened. All of those who meet the criteria for a positive FASD screen will be referred for a diagnostic evaluation. As such, no sampling will be conducted; the complete universe of eligible participants will be invited to participate.

The findings from this data collection effort can only be replicated at similar settings and population groups. The findings cannot be generalized to the population at large or to populations of children in the 0-7 or 8-18 years age group.

2. Information Collection Procedures

Copies of the survey instruments are included in Attachment I as Appendices A through K. As described above, the sampling plan is purposive. Each survey instrument states the purpose of data collection, their right not to participate, and that their information will be kept private.

Screening and Diagnosis (Forms A and B)

The “Screening and Diagnosis Tool” (Form A) is a screening tool that determines whether children screen positive for an FASD, and whether children who are referred for a diagnostic evaluation are diagnosed with an FASD.

The “Positive Monitor Tracking” form (Form B) is a tracking form used to monitor children ages birth through 3 who fulfill some of the criteria but not enough to warrant a positive screening result. These children are placed in a positive monitor category. This form is also used to record the final screening result for children who could not be screened as positive or negative for an FASD during their initial screening appointment.

Service Delivery (Forms C, D, and E)

The “Services Child Is Receiving at the Time of the FASD Diagnosis” form (Form C) is a provider form that is used to gather information about services that are already being provided to the child at the time of the FASD diagnosis. Form C is completed once for clients who are diagnosed with an FASD.

The “Services Planned and Provided based on Diagnostic Evaluation” form (Form D) is a provider form that is used to record the type and frequency of services recommended based on the results of the diagnostic evaluation, the type and frequency of services that are actually provided, and any reasons that services could not be provided as planned. Form D is completed throughout the course of the intervention for clients who are diagnosed with an FASD.

The “Services Delivery Tracking” form (Form E) is a provider form that is used to record specific dates of services, frequency of rescheduled appointments, and any reasons that services could not be provided as planned. Form E is completed throughout the course of the intervention for clients who are diagnosed with an FASD.

Outcome Measures (Forms F, G, H, I, and J)

The “End of Intervention/Program Improvement Measure: Case Manager” form (Form F) records the case manager’s perception of the child’s overall improvement as a result of participation in FASD services. The “End of Intervention/Program Improvement Measure: Parent” form (Form G) records the parent or legal guardian’s perception of their child’s overall improvement as a result of participation in FASD services. The “End of Intervention/Program Customer Satisfaction with Service” form (Form H) records the parent or legal guardian’s overall satisfaction with the FASD services provided to their child. Forms F, G, and H are administered once at the end of FASD services.

The “Outcome Measures (Children Ages 0–7)” form (Form I) and “Outcome Measures (Children/Adolescents Ages 8–18)” form (Form J) are administered as age-appropriate to participants who are diagnosed with an FASD. The forms are used to track behavioral changes in school attendance, school disciplinary incidents, residential placement stability, and probation violations. The forms are administered at baseline (within 30 days of developing the service delivery plan), every six months after the service plan is implemented, upon completion of probation or service delivery, and both six months and twelve months after the end of the intervention.

Lost to Follow-up (Client Participation Tracking form) (Form K)

The “Lost to Follow-up” form (Form K) is used to track children and adolescents’ participation in the program from eligibility through follow-up. In instances where the child/adolescent refuses to participate or the participant can’t be located, the case manager will record the relevant information on the Lost to Follow-up form.

3. Methods to Maximize Response Rates

The evaluation subcontractor will work with contractor to assist the sites in creating appropriate evaluation plans. Response rates have been estimated at 80% across all of the participating sites. The subcontractor’s strategy to help sites maximize their response rates and minimize burden on respondents relies upon establishing rapport with clients from the start of the first session and includes more specific strategies such as:

1. Creating and maintaining a tracking form which includes Client ID numbers, participant identifying information, and checkboxes to denote whether or not client completed each step of the intervention (screening, monthly appointments, etc.) This will help to closely track participants in order to identify and follow up with individuals with missing data.

2. Leaving a reminder voice mail or sending a reminder email to participants a few days before their upcoming appointment, to reduce the chance of a missed appointment.

3. Case managers keep in touch with the court system review such as probation hearing review or child welfare review.

4. Tests of Procedures

All survey instruments were reviewed by the subcontractor evaluation team during the fall of 2008. All instruments were pilot tested at 8 sites (one per site). Subsequent revisions were made to these forms in the spring and fall of 2009 based on their feedback. Some of the questions on placement of the child and screening criteria were modified based on feedback from the sites. The screening criteria for the 0-7 year olds was modified by adding a sub item “growth deficits” to the screening criteria. The positive monitor identification criteria was added to clearly identify those that fall in the positive monitor category and an option to check off the results of the screening was also added. The Lost to Follow-up form was also added based on feedback from the sites. The form was renamed as “Client Participation Tracking form” to clarify its purpose.

5. Statistical Consultants

Data collection will be conducted at the eight sites that are integrating the Diagnosis and Intervention project. The evaluator for the FASD Center for Excellence Vinitha Meyyur, Ph.D. will oversee the data collection at the sites and the data analysis performed by the subcontractor. Data analysis will be performed by the subcontractor, Human Services Research Institute under the supervision of the Project Director, Virginia Mulkern, PhD.

Contractor: Phone

Vinitha Meyyur Ph.D. Northrop Grumman Corporation 301-527-6512

Sub Contractors:

Virginia Mulkern, Ph.D. Human Services Research Institute 617-876-0426

Deborah Potter, Ph.D. Human Services Research Institute 617-844-2506

Sandra Richman, MSW Human Services Research Institute 773-484-7939

SAMHSA Project Officer:

Patricia B. Getty, Ph.D.

Acting Director

Division of Systems Development

Center for Substance Abuse Prevention

Substance Abuse & Mental Health Services Administration

One Choke Cherry Road, #4-1027

Rockville, MD 20857

(240) 276-2577

(240) 276-2580 (FAX)

[email protected]

 

List of Attachments

Survey Forms

I A Screening and Diagnosis Tool

2. Positive Monitor Tracking

3. Services Child Is Receiving at the Time of the FASD Diagnosis

4. Services Planned and Provided Based on the Diagnostic Evaluation

5. Services Delivery Tracking

6. End of Intervention/Program Improvement Measure: Case worker

7. End of Intervention/Program Improvement Measure: Parent

8. End of Intervention/Program Customer Satisfaction with Service

9. Outcome Measures: Children Ages 0–7

10. Outcome Measures: Children/Adolescents Ages 8–18

11. Client Participation Tracking Form

Other Attachments

II Informed Consent Forms

III List of FASD Diagnosis and Intervention Sites

19