OMB_ss B_nitt-aware_06-23-16

NOW IS THE TIME (NITT)—Project AWARE (Advancing Wellness and ResiliencE in Education) EVALUATION

SUPPORTING STATEMENT

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1 Respondent Universe and Sampling Methods

The NITT-Project AWARE evaluation will use a combination of census and sampling approaches to collecting process and outcomes data through the instruments at the center of this OMB application, while using existing State Educational Agency (SEA) grantee and project evaluation data, teacher survey data, student survey data, and school information systems data. The procedures and source for collecting data are listed in Table 6.

Using a census approach, the targeted universe is all NITT-Project AWARE grantee projects (N = 20) and participating Local Educational Agencies (LEAs; N = 62¹). All SEA grantees and their LEA partners have agreed to participate in NITT-Project AWARE evaluation data collection activities as a condition of funding.

Core Project Staff. Core staff will be selected as respondents to the AWARE Activities Inventory annual questionnaires and AWARE stakeholder interviews under a census approach. Core staff include those knowledgeable about the process of developing and delivering Project AWARE activities designed to enhance the mental health service array for students and youth; support mental health literacy and ability to detect and appropriately respond to signs of mental illness among students by adults who interact with them; support early identification of mental health needs among students; and promote school and community safety and wellness. Core staff include the Project Director, SEA-level project coordinator(s), and each LEA coordinator. These staff will complete the Web-based questionnaires to review and update relevant sections of the Implementation Activities Catalog via telephone and during in-person site visits.

A census of all 20 NITT-Project AWARE grantee project directors, any identified SEA-level project coordinators, and an LEA coordinator from each partner LEA (N = 62) is necessary due to the heterogeneous nature of the NITT-Project AWARE programs. These programs encompass a wide variety of organizational types and structures that are implementing a range of interventions with various outcome goals. The variety between the programs makes it critical to the evaluation to capture the details of each program to be able to answer the evaluation questions and assess which program characteristics and mix of interventions are associated with better outcomes for this youth/young adult demographic group and types of communities.

Other AWARE Stakeholders. A sampling approach for the Collaborative Partner Interview, conducted annually via phone and in-person includes nomination by the grantee project directors to identify key informant. Approximately eight SEA-level stakeholders will be interviewed in each SEA, annually beginning in mid-2016 and continuing until mid-2019. These stakeholders will represent key AWARE initiative partners and will include, at a minimum, representatives from youth and family perspectives; and partners representing state mental/behavioral health, state criminal/juvenile justice agencies, and other identified partner agencies at the state level, such as state Medicaid directors and child welfare, early childhood, and faith-based organizations.

School coordinators. A point of contact will be identified in each school participating in the AWARE national evaluation. These 432 schools will be selected through (1) a stratified random sample of all schools prioritized for AWARE grant activities in the LEA-partner communities, and (2) matched comparison schools. Schools within LEAs that have been prioritized as AWARE schools, and matched comparison schools will be stratified based on size and school type. The national evaluation team will work with each of the 20 SEA grantees to determine which schools are prioritized for AWARE activities during the grant period. These schools will form the universe for a stratified random sample of approximately 6 treatment middle and high schools per LEA. Once the treatment sample has been selected, the national evaluation team will use propensity score matching approaches to identify suitable comparison schools in each LEA. The comparison schools will be contacted and requested to participate in the national evaluation. Based on initial discussions with the SEA grantees and assessments of the data capabilities already in place in the partner LEAs, we expect to successfully recruit an average of 2 comparison schools in 30 of the LEAs. A point of contact at each school will be identified in collaboration with the LEA project coordinator and the school to respond to the school-level questionnaire.

School-level data. Data from school information systems will be collected from this school sample, which includes an expected 432 schools derived from the stratified random sample of project AWARE and comparison schools. School-level data will also include aggregated data abstraction of existing teacher school climate and school safety surveys and student surveys.

Teachers. All teachers in sample schools across all grantees comprise the respondent universe for the Teacher Mental Health Literacy Survey. The survey will be distributed to a sample of approximately 24 teachers per school from a stratified sample of treatment and comparison schools per LEA. Thus, the final potential sample size is 10,368 teachers.

Students. Focus groups will be conducted with approximately 8-10 students in each of four schools from 20 LEAs (one LEA will be identified for each SEA grantee), for a total of 1,000 students from 80 schools. Data will be aggregated at the school level. Focus groups will last approximately one hour. Parental permission will be obtained prior to data collection as well as student assent (see Attachment 9). These data will be collected during the 2016-2017 and 2018-2019 school years.

B.2 Information Collection Procedures

NITT-Project AWARE cross-program evaluation data collection will be centralized and coordinated through the NITT-Project AWARE data collection team leader and liaison assigned to each SEA grantee. This liaison will be the SEA grantee’s primary point of contact for all questions about data collection, reporting, and feedback. Furthermore, these liaisons will help collect data. For example, all process evaluation telephone interviews will be conducted by the RTI liaison assigned to that SEA grantee.

The NITT-Project AWARE evaluation approach will use various data collection strategies across the range of respondents, including perspectives of stakeholders, students, and the system that serves children and youth in the grantee communities. Due to the large number of relevant stakeholders at multiple levels of analysis, many types of instruments and modalities are being used; however, the evaluation has been designed to minimize burden for each stakeholder type. The type of data collection, respondent/source, mode, administration frequency and expected sample size are summarized in Table 6. This table is organized by primary data collection methodologies:

• Web-based instruments,

• in-person data collection, and

• secondary data abstraction.

Table 6. NITT-AWARE Data Collection Procedures by Source of Information

Type of Data Collection	Respondent/ Source	Mode	Administration Frequency	Expected N (per administration)
Primary data collection
SEA Leadership Questionnaire	SEA Project Director/ Project Coordinator	Telephone/ in-person interview	Annual: 2016 through 2019	20
LEA Coordinator Questionnaire	LEA Coordinator	Telephone/ in-person interview	Annual: 2016 through 2019	62
School Coordinator Questionnaire	School Coordinator	Telephone/ in-person interview	Annual: 2016 through 2019	432
SEA-Collaborative Partner Survey	SEA-level Collaborative Partners	Web-based survey	Annual, 2016 – 2019	480
LEA-Collaborative Partner Survey	LEA-level Collaborative Partners	Web-based survey	Pre-post: 2016 and 2019	930
Collaborative Partner Interviews	SEA-level Collaborative Partners	Telephone/ in-person interview	Pre-post: 2016 and 2019	160
Teacher Mental Health Literacy Survey	Teachers in sampled schools	Web-based survey	Pre-post: 2016 and 2019	10,368
Student Focus Groups	Students in sampled schools	Focus group	Pre-post: 2016 and 2019	1,000
Secondary Data Abstraction*
School Information Systems Data Abstraction	Sampled schools	Secondary data transfer	Annual: to cover school years 2014-2015 through 2018-2019	125
Student Survey Data Abstraction	Sampled schools	Secondary data transfer	Annual: to cover school years 2014-2015 through 2018-2019	125
Teacher School Climate and School Safety Survey	Sampled schools	Secondary data transfer	Annual: to cover school years 2014-2015 through 2018-2019	125

* Secondary data abstraction will be collected at the school level for all sample schools (N = 432), but the number of respondents is calculated based on whether the school information systems and existing student and teacher surveys are consistent across SEAs and/or LEAs, or whether they vary from school to school. Based on preliminary discussions with the grantees, we estimate that five SEA grantees will be able to provide data for all sample schools in the SEA (N = 5 SEA respondents), the data will be provided from LEAs in ten of the SEA grantees (N = 30 LEA respondents), and the remaining five SEA grantees will have school information systems and surveys that differ at the school level (N = 90 school respondents). Therefore 125 respondents will provide the secondary data that covers the 432 sample schools.

Web-based Instruments

Each Web-based data collection instrument will follow the procedures outlined below. Web-based instruments include the SEA Collaborative Partner Survey, the LEA Collaborative Partner Survey, and the Teacher Mental Health Literacy Survey.

E-mails will be sent to the designated respondents to inform them that data collection has started. A link to enter the Web-based survey will be included in the e-mail, as well as information on the background, purpose, types of questions and length of the survey. When respondents complete the survey a “thank you” e-mail will be sent automatically. Nonresponders will be sent predefined, automated weekly e-mails as needed, to remind them to complete their survey. If the survey is not completed within 3 weeks, the grantee will be contacted via telephone by the assigned liaison.

The NITT-Project AWARE evaluation team will develop user manuals for accessing and navigating the online data collection systems and question-by-question and frequently asked question (FAQ) guides to help respondents accurately complete the surveys. Grantees will also be provided training webinars to: 1) walk through the online data collection systems, and 2) review the instrument data collection procedures and answer questions. Within the online data collection system, all manuals, guides, and training webinars will be archived and accessible to respondents for reference at any time.

Availability is important in any data collection system, especially one employed by grantees around the country, including multiple time zones. The online system will be maintained in an available state as much as possible to allow grantees access for viewing their specific data, as well as to give the NITT-Project AWARE team, grantees, and SAMHSA access to reports.

Providing a robust system that is simple and easy to use across all areas is also critically important. To achieve this, the contractor will implement user-friendly features across all functional areas, taking into account the needs of both SAMHSA and grantees. Additionally, every page of the online data system will have a “Help” or “Support” link clearly labeled, which will allow the respondent to access the following support resources:

1. Search. More comprehensive than a list of FAQs and more organized than a support forum, the search feature will offer a “layered information” approach so that respondents can search by keyword and then drill down to view material at increasing levels of detail. It will be a curated and easily searchable source of information including items such as

• user guides,

• data collection protocols,

• training materials and webinars,

• question-by-question guide, and

• FAQs.

1. Contact Us. Respondents may request assistance either by calling a provided toll-free number or sending an e-mail request, as desired. The toll-free line will be routed to an e-mail system that is checked regularly by members of the training and technical assistance team. Staff responding to technical assistance requests will be trained in use of the system and have ready access to the full support system. Training and technical assistance team staff will monitor all submitted tickets to ensure timely response and resolution of technical assistance requests.

Grantee Telephone and In-person Interviews

Interviews will be conducted with grantee leadership and coordinators on an annual basis to compile and maintain the Activities Inventory, and with collaborative partners to collect information about the collaborative structure and processes. Interviews will be conducted primarily by telephone, but may be conducted on-site during site visits, depending on the timing of the visits, which are scheduled to facilitate student focus groups.

Respondents to the interviews will be contacted by the NITT-Project AWARE evaluation via e-mail with telephone follow-up to set up a mutually convenient time for the interview during regularly scheduled business hours. Before conducting the telephone interviews, the evaluation team will review grant applications (submitted to SAMHSA by each grantee and given to the evaluation team by SAMHSA) and other documents (e.g., Collaboration and Coordination Plan, Evaluation Plan, Implementation Plan, and annual progress reports) that detail the characteristics of the program and abstract information relevant to the evaluation (e.g., project structure, interventions) so that liaison staff conducting the interviewers are familiar with the grantee. This pre-abstracted information will be used to prepopulate some interview questions to reduce respondent burden. For instance, a list of the grantee’s activities will be abstracted to the Activities Inventory, and used as a guide for interviewees to update and correct the Inventory as indicated.

Once the interview is scheduled, the contractor will provide the participant with an electronic version of the consent form and the partially prepopulated interview guide and a toll-free, passcode-protected telephone conference number. Before beginning the telephone interview, consent will be requested to record the interview to confirm, if needed, the accuracy of noted responses. A senior evaluator from the contractor’s evaluation team will lead the respondent through the interview while a junior evaluator will record responses and take notes. After the interview, the interviewer and note taker will review the responses for accuracy. Any areas of discrepancy will be validated with the recording (if consented by the respondent); once the responses are considered final, the recording will be deleted. An electronic version of the telephone interview will be maintained on a password protected, secure server accessible only to the contractor’s evaluation team. After the interview, the interviewer will send an e-mail thanking the respondent for his or her participation.

This procedure will be followed for the follow-up data collection instruments as well.

A procedures manual will be developed for the administration of the telephone interviews and training will be provided to all interviewers and note takers to walk through interview procedures and questions.

In-person Interviews Conducting Grantee Site Visit

Prior to beginning a site visit discussion, the respondents, including both project staff and key informant/stakeholder, will be read and provided a copy of the site visit consent form that informs participants of their rights, including the right to not answer any question, and asks for their written consent to participate in the discussion and for the discussion to be recorded. Recordings will be used to ensure that information is correctly captured from multiple interviews, that information has been consistently captured, and to correct and clarify brief written notes as needed and part of data quality assurance process. Recordings will only be accessible to the contractor and will be stored on password-protected secure servers and destroyed once de-identified notes are completed. Information collected by the site visit interviews will be reported only in aggregate and individual respondents will not be identified.

Student Focus Groups

Student focus groups will be conducted by national evaluation project staff in one LEA associated with each of the 20 SEA grantees. The groups will be held during site visits in school year 2016-2017 and again in school year 2018-2019.

Prior to beginning a focus group discussion, the participants, including both youth/young adults and family/adult allies, will be read and provided a copy of the focus group consent form that informs participants of their rights, including the right to not answer any question or participate in any discussion, and asks for their written consent to participate in the discussion and for the discussion to be recorded. If a participant does not provide verbal assent consent to participate then that individual will be excused from the group. Consent will have to be provided by parents or guardians for any youth participants under the age of 18 years. Recordings will be used to ensure that information is correctly captured from multiple focus groups, that information has been consistently captured and to correct and clarify brief written notes as needed and part of data quality assurance process. Recordings will only be accessible to the contractor and will be stored on password-protected secure servers and destroyed once de-identified notes are completed. Information collected by the site visit interviews is only reported in aggregate and individual respondents will not be identified.

B.3 Methods to Maximize Response Rates

Grantees are required to participate in all NITT-Project AWARE evaluation activities by the terms and conditions of the NITT-Project AWARE grant award. The NITT-Project AWARE evaluation team will employ a number of strategies to maximize response rates.

Web-based Surveys

As described above, the NITT-Project AWARE evaluation will develop user manuals for accessing and navigating the online data collection systems and question-by-question and FAQ guides to help respondents accurately complete the instruments. Grantees will also be provided training webinars to introduce the NITT-Project AWARE evaluation, walk through the online data collection systems, and review data collection procedures and instruments. Within the online data collection system, all manuals, guides, and training webinars will be archived and accessible to respondents for reference at any time.

For online Web-based surveys, respondents will be sent automated, predefined e-mails to remind them of completion deadlines. Specifically, the following reminder schedule will be followed:

1. Start of Data Collection: At the start of data collection when the Web-based survey is available for the participant to complete, grantees and learning laboratory staff previously nominated will be sent automated e-mails to inform them that they were nominated by the project director to complete the survey and of the start of data collection. E-mails will include a link to the survey as well as login and password the participant will need to access the survey.

2. At Survey Completion: The data submission system will automatically send a “Thank You” e-mail when respondents complete the survey.

3. Weekly After the Start of Data Collection: The Web-based system will automatically send an e-mail to participants who have not completed the survey reminding them that data collection has started and asking them to complete the survey at their earliest convenience. The e-mail will include the evaluation toll-free number and e-mail address should the participant have any questions or need to speak with the evaluation team liaison.

4. One Week Before the Stated Data Collection Deadline: The evaluation team liaison will call the designated participant alerting them that the data collection deadline is approaching and offer any necessary assistance with completing the Web-based survey.

5. Data Collection Deadline: The Web-based system will automatically send an e-mail to nonresponders and their grantee alerting them that the data collection deadline has passed. When a nonresponder is a grantee, the SAMHSA Grantee Project Officers (GPOs) will also be notified.

6. Two Weeks After Stated Data Collection Deadline: NITT-Project AWARE will notify the GPO, who will either e-mail or request a telephone call with grantees (or with learning laboratories and their respective grantees) who have not completed their survey 2 weeks after the stated deadline. Grantees will be expected to monitor their learning laboratories compliance.

In addition, respondents to the Teacher Mental Health Literacy Survey will be offered a $20 gift card for completing the survey at each administration. As described in Section A.9, incentives can many times be an effective tool (especially but not exclusively at the individual level)—assuming OMB’s continued receptivity to the use of monetary incentives in this setting. Teachers in particular are reluctant to take on additional voluntary activities that are not written into their job description. This incentive will be instrumental in obtaining completed teacher surveys for this study because teachers are the target of numerous requests to complete surveys on a wide variety of topics from state and district offices, independent researchers, and the Department of Education (Policy and Program Studies Service and NCES). Further, the teachers’ school days are already quite busy, potentially requiring them to complete surveys outside school time. There are also in some localities collective bargaining agreements that do not allow teachers to complete surveys during school time.

Telephone and In-person Interviews

For the following telephone or in-person interviews, the initial e-mail invitations will provide a thorough explanation of the study and its importance, the reasons the participant is being asked to participate, and means by which they can contact the evaluation team for additional information, including a toll-free telephone number and project specific e-mail.

• SEA Project Director/ Coordinator Questionnaire

• LEA Project Coordinator Questionnaire

• School Coordinator Questionnaire

• Collaborative Partner Interview

The evaluation team will aim to identify the most convenient time for the participant to complete the telephone or in-person interview. Before the interview, participants will also be provided the interview topics so they will be knowledgeable about the types of information to be collected. Nonresponders to the initial e-mail invitation will be sent weekly follow-up reminder e-mails. If needed—although the evaluation team does not anticipate that it will be necessary—the follow-up reminder e-mails will include the grantee’s GPO.

Student Focus Groups

The evaluation team will collaborate with school coordinators in each school to identify methods of obtaining parental consent and student assent to participate in the focus groups. The focus groups will be held at a time that does not interfere with school academic activities, and held in an environment to maximize student privacy and comfort expressing their opinions in a focus group setting.

Comparison School Participation

An important component of the NITT-Project AWARE-SEA evaluation is a matched school comparison study which will be conducted in a subset of the LEAs to minimize burden on grantees. The evaluation team will work to identify appropriate comparison schools in the LEAs participating in Project AWARE-SEA and recruit those schools to participate in the comparison group substudy. The substudy will require all treatment and comparison schools to complete annual surveys about programs in place to achieve AWARE goals, coordinate with the national evaluation team to administer a mental health literacy survey with a sample of teachers in the school, and assist in recruiting students to participate in focus groups. The identified point of contact (or multiple points of contact) in each treatment and comparison school will also work with the national evaluation team to provide access to the school information system and existing surveys that measure school climate and school safety. These measures will allow the comparison of Project AWARE (“treatment”) schools with comparison schools regarding change in school-level mental health literacy, mental health service access and receipt, student coping skills and resiliency, school climate, and school safety.

To assist in recruitment and increase participation rates, a $250 gift card will be provided to each comparison school for each year of their participation, $50 of which will go to the coordinator responsible for gathering and reporting data (where allowed by school regulations).  The gift card will also serve as a thank you for the time staff in the school invest as these schools are not included in NITT-Project AWARE-SEA grant activities and thus do not benefit from professional development, program implementation support, or technical assistance from the state or district related to programs. These incentive amounts are comparable to those provided in other OMB-approved studies and evaluations in similar educational settings with similar data collection approaches (e.g., respondent type, burden), for example, the Progress in International Reading Literacy Study (OMB No. 1850-0645), the High School Longitudinal Study Field Test First Follow Up and Main Study (OMB No. 1850-0852), and the Middle Grades Longitudinal Study–Item Validation Field Test (OMB No. 1850-0911).  

B.4 Test of Procedures

Contractor and subcontractor staff completed the Web-based surveys and interview instruments, either in paper-pencil form or within word processing software. These staff members have experience with evaluation initiatives, as well as calculating survey lengths. It is likely that the Web-based versions of the surveys below will take less time than the paper version tested to generate the estimates in this section, as skip patterns will be automated and some items will be prepopulated automatically after initial responses.

The SEA-Collaborative Partner Survey takes 28 minutes to complete, including time for reading the survey instructions and consent and responding to survey questions.

The LEA-Collaborative Partner Survey takes 35 minutes to complete, including time for reading the survey instructions and consent and responding to survey questions.

The SEA-level, LEA-level, and School Coordinator Questionnaires take an average of 56 minutes to complete, including time for reading the survey instructions and consent and responding to survey questions.

The Student Focus Group conducted during the grantee site visit will take approximately 60 minutes to complete; this includes time for introductions, directions and informed consent, and time for the focus group discussion. The focus group discussion will end after 60 minutes, regardless of whether all discussion items have been covered.

The Teacher Mental Health Literacy Survey takes approximately 30 minutes to complete.

The Collaborative Partner Interview is estimated to take 1 hour to complete per response.

The secondary data abstraction protocols, to collect annual information from existing School Information Systems, Teacher School Climate and School Safety Survey, and Student Surveys will take approximately one and a half hours for each respond to compile and upload the requested information.

B.5 Statistical Consultants

As noted in Section A.8, SAMHSA has consulted with an expert panel on the NITT-Project AWARE evaluation plan, data collection procedures, and analysis plans. These experts will continue to provide advice and feedback throughout the course of the evaluation through annual panel meetings. In addition, the contractor team comprises several experts who have been involved in the development of the NITT-Project AWARE data collection and analysis plans and will be directly involved in data collection and statistical analysis. Also, SAMHSA advisors will be consulted throughout the evaluation on various statistical aspects of the design, methodological issues, and data analysis. Table 7 provides details of these team members and advisors.

Table 7. Statistical Consultants for the NITT–Healthy Transitions Evaluation

Name & Role in Evaluation	Title & Address	Contact Information
NITT-Project AWARE Evaluation Staff
James Trudeau, PhD NITT Evaluation Project Director	Senior Research Social Scientist Center for Justice, Safety, and Resilience RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-485–7751 E-mail: [email protected]
Heather Ringeisen, PhD NITT Evaluation Deputy Project Director	Director Center for Behavioral Health and Development RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-541-6931 E-mail: [email protected]
Duren Banks, PhD NITT-Project AWARE Evaluation	Senior Research Criminologist Center for Justice, Safety, and Resilience RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-541-8026 E-mail: [email protected]
Antonio Morgan-Lopez, PhD NITT-Project AWARE Analysis Team Lead	Principal Scientist Risk Behavior and Family Research RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-316–3436 E-mail: [email protected]
Suyapa Silvia, PhD NITT-Project AWARE Data Collection Team Lead	Director Center for Education Assessment RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-541-8026 E-mail: [email protected]
SAMHSA Advisors
Nainan Thomas, PhD Contracting Officer’s Representative	Public Health Advisor CMHS, SAMHSA 1 Choke Cherry Road, Room 6-1099 Rockville, MD 20857	Phone: 240-276-1744 E-mail: [email protected]
Kirstin Painter, PhD Alternate Contracting Officer’s Representative	Public Health Advisor CMHS, SAMHSA 1 Choke Cherry Road, Room 6-1040 Rockville, MD 20857	Phone: 240-276-1932 E-mail: [email protected]
Beda Jean-Francois, PhD	Social Science Analyst CBHSQ, SAMHSA 1 Choke Cherry Road, Room 2-1012 Rockville, MD 20857	Phone: 240-276-0370 E-mail: [email protected]

REFERENCES

Arseneault, L., Walsh, E., Trzesniewski, K., Newcombe, R., Caspi, A., & Moffitt, T.E. (2006). Bullying victimization uniquely contributes to adjustment problems in young children: A nationally representative cohort study. Pediatrics, 118, 130–138.

Burns B.J., Costello E.J., Angold A., Tweed D. et al. Children’s Mental Health Service Use Across Service Sectors, Health Affairs, Vol. 14, No. 3, 1995: 149-159.

Flannery, D. J., Wester, K. L., & Singer, M. I. (2004). Impact of exposure to violence in school on child and adolescent mental health and behavior. Journal of Community Psychology, 32, 559–573.

Kataoka S., Zhang L., Wells K. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. American Journal of Psychiatry. 159:1548–1555.

Milan, A.J., Furr-Holden, C.D.M, & Leaf, P.J. (2010). Perceived school and neighborhood safety, neighborhood violence and academic achievement in urban school children. The Urban Review, 42, 458-467.

RAND. (2001). Mental health care for youth: Who gets it? How much does it cost? Who pays? Where does the money go? Retrieved from http://www.rand.org/content/dam/rand/pubs/research_briefs/2005/RB4541.pdf

Trudeau, J., Williams, J., & Murray, S. (2010, January). Safe Schools/Healthy Students Initiative cross-site evaluation: Final report. Prepared for the Office of Juvenile Justice and Delinquency Prevention, U.S. Department of Justice.

U.S. Department of Health and Human Services. (2001). Mental Health: Culture, race and ethnicity - A supplement to Mental Health: A Report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Public Health Services, Office of the Surgeon General.

LIST OF ATTACHMENTS

1. AWARE Planning and Implementation Activities Inventory (AWARE Activities Inventory)

2. State Educational Agency Collaborative Partner Survey (SEA-CPS)

3. Local Educational Agency Collaborative Partner Survey (LEA-CPS)

4. Collaborative Partner Interview Guide

5. School Information Systems Data Abstraction Protocol

6. Teacher Mental Health Literacy Survey

7. Teacher School Climate and School Safety Survey Data Abstraction Protocol

8. Student Survey Data Abstraction Protocol

9. Student Focus Group Protocol

1 One of the SEAs has partnered with 5 LEAs, while the remaining 19 SEA grantees have each partnered with 3 LEAs.

6