Supporting Statement

Pretesting Messages and Materials for the

Caring for Every Child’s Mental Health Campaign

(2007–2009)

Supporting Statement

1. Justification

The Substance Abuse Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services is seeking OMB approval for focus group questionnaires previously approved under the generic clearance OMB No. 0930-0196. These data collection tools will be used to conduct focus groups with three different audience types: (a) school superintendents, (b) school principals and (c) school board members.

In 2004, the Caring for Every Child’s Mental Health Campaign conducted a comprehensive situational analysis to identify social marketing strategies and tactics to best support system of care communities as they move toward sustainability beyond Federal funding. In this analysis, system of care communities overwhelmingly identified educators as one of the most important partners in a system of care. This finding was reinforced in 2006 when the Campaign Resource Group (CRG) identified community educators as a critical partner in helping system of care communities reach sustainability.

The situational analysis and the CRG also identified developing customizable low-literacy materials for community educators as one of the most important Campaign tasks. “Community educators” refers to individuals without formal training as teacher, but play important roles in educating others in their communities. Examples of community educators include parents, caregivers, and caring adults, as well as youth with serious mental health needs. According to the situational analysis and the CRG, there is a significant need for easy-to-read, low-literacy materials that (1) parents, caregivers, and caring adults can use to explain mental health and illnesses to children and youth, (2) youth can use to explain their mental health needs to their peers and caring adults, and (3) youth can use to empower themselves to play a more active decision-making role in the type of care they receive.

1. Circumstances of Information Collection

The Center for Mental Health Services (CMHS) of the Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting OMB approval for two new Moderator’s Guides and one Depth Interviewer’s Guide for use in focus groups to pretest messages and materials for the Caring for Every Child’s Mental Health Campaign:

1. MODERATOR’S GUIDE: Campaign Messages/Visuals/Themes (School Superintendents, School Principals, or School Board members or equivalent);

2. MODERATOR’S GUIDE: Campaign Materials (School Superintendents, School Principals, or School Board members or equivalent); and

3. Depth Interviewer’s Guide: Depth interview questions (School Superintendents, School Principals, or School Board members or equivalent).

CMHS is responsible for the Comprehensive Community Mental Health Services for Children and Their Families Program, now referred to as the Children’s Mental Health Initiative (CMHI). Communities funded through this program must establish and operate one or more systems of care for making a range of specified mental health services available to each child. A system of care is a coordinated network of community-based services and supports that are organized to meet the challenges of children and youth with serious mental health needs and their families. Families and youth work in partnership with public and private organizations to design mental health services and supports that are effective, that build on the strengths of individuals, and that address each person’s cultural and linguistic needs. A system of care helps children, youth, and families function better at home, in school, in the community, and throughout life.

CMHS provides its funded communities with a range of support, including assistance with social marketing strategies through its Caring for Every Child’s Mental Health Campaign. The focus groups will be used to assess messages and materials that will help funded communities reach out to and partner with local school systems. The information gained will provide guidance to CMHS in developing messages and materials that will appeal to and be effective with this audience.

Effective, concise messages and materials are essential to the success of the Caring for Every Child’s Mental Health Campaign, enabling funded communities to:

• Reduce the stigma associated with mental illness and promote mental health;

• Increase the likelihood that children and youth with serious emotional and behavioral disturbances and their families are appropriately served and treated;

• Increase awareness of mental health needs and services for children and youth among mental health providers, funded communities, intermediary groups/organizations, and the public;

• Demonstrate to communities that the mental health needs of children and youth with serious emotional and behavioral disturbances and their families are best met through utilization of systems of care; and

• Build capacity within funded communities to sustain services and support to children and youth with serious emotional and behavioral disturbances and their families.

The messages and materials to be pretested are intended to help communicate what a system of care is; how systems of care work in meeting the mental health needs of children, youth, and families; the benefits that children, youth, and families receive when schools partner with systems of care; and the benefits school systems receive when they partner with a system of care. The audience for these messages and materials has been determined to be the primary decision makers within school systems who are capable of creating, or significantly influencing the creation of formal partnerships with systems of care. Specific messages and materials to be pretested are included with the attached Moderator’s and Depth Interviewer’s Guides.

2. Purpose and Use of Information

The overall goal for the proposed focus groups is to obtain feedback on messages and materials in development that can be effective with school system decision makers. The objective is to ensure that the Campaign’s messages, themes, and visuals are clear and effective and that they meet the needs of the SOC communities to communicate and partner with their local school systems in meeting the needs of children and youth with serious mental health needs and their families.

CMHS proposes two sets of focus groups and one set of depth interviews:

Set 1: Messages/themes/visuals. A total of nine focus groups will be held with School Superintendents, School Principals, and School Board Members in at least three communities to explore which core messages of systems of care resonate with the education audience. For example, we know that systems of care and the education system share common goals, such as achieving higher academic standards and accountability, lower absenteeism, fewer behavior problems, more time for teaching and learning, and increased parent involvement.

Specific messages we may test with these audiences include:

• Systems of care are an additional resource for getting mental health services and supports for students.

• Systems of care help improve communication among all those serving children and families (i.e., juvenile justice, child welfare, mental health providers, primary care, etc.), which allows educators to better understand and more effectively meet a student’s needs.

• Systems of care can coordinate a wide range of services and supports for students, including education services, which may reduce the administrative burden on educators.

School Superintendents—One focus group will be held with Superintendents in communities with formal partnerships between schools and a system of care; one focus group will be held with Superintendents in communities where there is a system of care, but no formal partnership exists with the school system; and one focus group will be held with Superintendents in communities where there is no system of care. Please note that at least one of these focus groups will either take place at a national conference where several superintendents are attending or via phone conference.

School Principals—One focus group will be held with Principals in communities with formal partnerships between schools and a system of care; one focus group will be held with Principals in communities where there is a system of care, but no formal partnership exists with the school system; and one focus group will be held with Principals in communities where there is no system of care.

School Board Members—One focus group will be held with Board members in communities with formal partnerships between schools and a system of care; one focus group will be held with Board members in communities where there is a system of care, but no formal partnership exists with the school system; and one focus group will be held with Board members in communities where there is no system of care.

Set 2: Specific materials. A total of nine focus groups will be held with School Superintendents, School Principals, and School Board Members in at least three communities to gather feedback on the prototype materials, which have been developed in response to suggestions from the audiences in the earlier stage of testing. The discussion will focus on visual elements of the prototype—including color, fonts, photographs or artwork, and more—and their appropriateness and ability to communicate effectively with the audience. Similarly, the specific language used on the cover or the interior pages of the prototype material will also be open for discussion.

School Superintendents—One focus group will be held with Superintendents in communities with formal partnerships between schools and a system of care; one focus group will be held with Superintendents in communities where there is a system of care, but no formal partnership exists with the school system; and one focus group will be held with Superintendents in communities where there is no system of care.

School Principals—One focus group will be held with Principals in communities with formal partnerships between schools and a system of care; one focus group will be held with Principals in communities where there is a system of care, but no formal partnership exists with the school system; and one focus group will be held with Principals in communities where there is no system of care.

School Board Members—One focus group will be held with Board members in communities with formal partnerships between schools and a system of care; one focus group will be held with Board members in communities where there is a system of care, but no formal partnership exists with the school system; and one focus group will be held with Board members in communities where there is no system of care.

Set 3: Depth interviews. The purpose of depth interviews is to get more information than any one person would have time to share in a group setting. A total of 54 depth interviews will be held in at least three different communities. In each of the three or more communities, interviews will be conducted with individuals having three levels of understanding of systems of care:

School Superintendents—Six depth interviews will be held with Superintendents in communities that have formal partnerships between the school systems and a system of care; six depth interviews will be held with School Superintendents in communities where there are systems of care, but no formal partnerships exist with the school systems; and six depth interviews will be held with School Superintendents in communities where there are no systems of care.

School Principals—Six depth interviews will be held with Principals in communities that have formal partnerships between the school systems and a system of care; six depth interviews will be held with Principals in communities where there are systems of care, but no formal partnerships exist with the school systems; and six depth interviews will be held with Principals in communities where there are no systems of care.

School Board Members— Six depth interviews will be held with Board members in communities that have formal partnerships between the school systems and a system of care; six depth interviews will be held with Board members in communities where there are systems of care, but no formal partnerships exist with the school systems; and six depth interviews will be held with Board members in communities where there are no systems of care.

Notes and audio recordings from the focus groups and depth interviews will be analyzed for positive and negative reactions to messages and materials, including reactions to specific images, words, or phrases. They will also be analyzed for indicators of whether messages were clearly understood and indicators of the participants’ knowledge, attitudes, and beliefs about serious emotional and behavioral disturbances, and of systems of care. While focus group results and depth interview responses cannot be generalized to an entire population or community, the analysis will look for insights into the subtleties and nuances of audience reactions and will highlight any common themes or concerns that emerge in several different groups as potential indicators of how materials and messages will be received.

The results will be used to refine Campaign messages, themes, and visuals and to guide further development of the materials tested. The results will also help guide development of future materials. With better understanding of audience reactions, writers and graphic designers will be able to choose words and images that are most appealing and meaningful to that audience and program managers will be able to plan materials that are likely to be most effective and useful to funded communities.

3. Use of Information Technology

Focus groups: The information will be gathered in person by an experienced focus group moderator who will meet with groups of up to 12 respondents. Electronic information collection is not feasible for a focus group, which generates findings through group discussion and interactions.

Depth interviews: Depth interviews will be conducted in-person by an experienced interviewer and writer. When in-person interviews are not possible, depth interviews will be conducted over the telephone. Electronic collection is not feasible for an interview, which generates findings through tone, nonverbal communication, and respondent effect.

4. Efforts To Identify Duplication

The information needed is specific to the Campaign’s messages and materials and is not collected anywhere else.

5. Involvement of Small Entities

This project will not have a significant impact on small businesses or entities.

6. Consequences if Information Collected Less Frequently

Each respondent will be asked to respond only once.

7. Consultation Outside the Agency

The instruments have been submitted to two experts for review:

Dana Carr

Program Specialist

Office of Safe and Drug-Free Schools

U.S. Department of Education

400 Maryland Avenue, SW

Room 3E100

Washington, DC 20202-6450

Tel: 202–260–0823

Fax: 202–260–7767
E-mail: [email protected]

Sandra Keenan

Education Resource Specialist

Technical Assistance Partnership for Child and Family Mental Health

1000 Thomas Jefferson Street, NW

Washington, DC 20007

Tel: 202–403–5324 
Fax: 202–403–5007
E-mail: [email protected]

8. Payment to Respondents

Respondents will not receive payment.

9. Assurance of Confidentiality

Focus group participants will be identified only by first name. No personal identifiers will be collected.

10. Questions of a Sensitive Nature

There are no questions of a sensitive nature.

11. Estimates of Annualized Hour Burden

The focus groups will have an average of 10 participants each for a total cost burden of $6,075, as shown below.

Moderator’s Guide/Interview Question List	No. of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost
Messages/Visuals/Themes	90 (9 groups of 10 participants each)	1	90	1.5	135	$20	$2,700
Materials	90 (9 groups of 10 participants each)	1	90	1.5	135	$20	$2,700
Depth Interviews	54 individual interviews	1	54	.5	27	$25	$675
TOTAL	234	-	234	-	297	-	$6,075

12. Estimates of Annualized Cost Burden to Respondents

There are no costs to respondents associated with either capital and startup efforts or operation and maintenance of services for this project.

13. Estimates of Annualized Cost to the Government

The cost to CMHS of the contract task to collect this information is $90,096. The cost of the time of the project officer is $1,730 (40 hours @ grade 13) for a total cost to the Government of $91,826.

14. Changes in Burden

This is a new project.

15. Time Schedule

This project is anticipated to begin in November 2007, depending on receipt of OMB approval. Focus groups and depth interviews will take place in winter 2008. The analysis and report will be completed by September 2008.

2. Statistical Methods

1. Respondent Universe and Sampling Methods

There will be 61 sites funded as systems of care communities by the Comprehensive Community Mental Health Services Program for Children and Their Families in the time schedule proposed for this project. From this respondent universe, CMHS will select nine sites that have demonstrated strong partnerships with schools and nine sites in need of increasing partnership activities with schools in their service area to meet their program goal per data from the National Evaluation Program. These 18 sites were also selected for their capacity to provide a cross-section of participant characteristics (e.g., racial/ethnic populations, geographic location, population density, years of experience with SOC, and types of local partners). In addition to their affiliation as system of care communities, these funded sites also have programmatic affiliation with SAMHSA through the Federal/National Partnership; this affiliation will serve as a forum in which personal contact can be made between those recruiting and potential focus group participants

2. Information Collection Procedures

Identification and Contacting of Participants

CMHS will identify and contact participants by working with individual systems of care, SAMHSA Federal/National Partnership (some members represent school systems), and professional organizations representing School Administrators, Principals, Superintendents, etc. Social marketing technical assistance specialists maintain close contact with each of the system of care communities, professional associations, and SAMHSA Federal/National Partnership members.

Conducting Focus Groups

Focus groups will be led by an experienced focus group moderator with knowledge of social marketing, CMHI, and systems of care. The protocol is outlined in the Focus Group Moderator’s Guide (Attachments 1 and 2).

Conducting Depth Interviews

Depth interviews will be conducted by an experienced interviewer and writer with knowledge of social marketing, CMHI, and systems of care. The protocol is outlined in the Depth Interviewer’s Guide (Attachment 3).

3. Methods To Maximize Response Rates

Recruitment will rely on the access to respondents via personal contacts that have developed between the systems of care communities and the service providers in their communities. All respondents will be closely affiliated with systems of care, the SAMHSA Federal/National Partnership, and the various professional associations. This “informal” means of communication has yielded a response rate of approximately 78 percent when utilized in previous projects where professionals were asked by their peers to participate in similar information gathering activities. To maximize response rates, focus groups and depth interviews will be held during normal working hours, perhaps at national conferences or during routine meetings. Furthermore, CMHS’s social marketing technical assistance experts will work with systems of care, professional organizations, and the SAMHSA Federal/National Partnership to help them recruit participants, providing them with briefing materials about the purpose of the focus groups and depth interviews, as well as tips on recruitment. In addition, to ensure that the sites understand and support the proposed focus groups and depth interviews, CMHS will be discussing them at meetings and conference calls in the months prior to this effort.

4. Tests of Procedures

Questions similar to those in the Moderator’s Guides will be asked in a small discussion group of less than nine people. Questions similar to those in the Depth Interviewer’s Guide will be asked via in-person or phone interview with one individual at a time.

5. Statistical Consultants

Project Officer: Lisa Rubenstein, SAMHSA/CMHS/CAFB, 240–276–1927

Contractor/Project Director: Maria Rodriguez, Vanguard Communications, 202–331–4323

List of Attachments

Attachment 1: MODERATOR’S GUIDE: Campaign Messages/Visuals/Themes—School Superintendents, School Principals, and School Board Members

Attachment 2: MODERATOR’S GUIDE: Campaign Materials—School Superintendents, School Principals, and School Board Members

Attachment 3: Depth Interviewer’s Guide—School Superintendents, School Principals, and School Board Members

9