OMB_Part A FINAL PBHCI 6 24 2016

Evaluation of the Primary and Behavioral Health Care Integration Grant Program

Supporting Statement

A. Justification

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ) is requesting approval from the Office of Management and Budget (OMB) to conduct the evaluation of the Primary and Behavioral Health Care Integration (PBHCI) grant program.

The Center for Behavioral Health Statistics and Quality is requesting clearance for ten data collection instruments and forms related to the implementation and impact studies to be conducted as part of the evaluation:

1. PBHCI grantee director survey (Attachment A)

2. PBHCI frontline staff survey (Attachment B)

3. Telephone interview protocol (Attachment C)

4. On-site staff interview protocol (Attachment D)

5. Client focus group guide (Attachment E)

6. Electronic data collection tool for grantee registry/electronic health records (EHRs) (Attachment F)

7. Consent to share contact information with the research project (Attachment G)

8. Initial client letter for physical health exam and health assessment (Attachment H)

9. Consent form for client physical exam and health assessment (Attachment I)

10. HIPAA form for client physical exam and health assessment (Attachment J)

11. Consent form for client focus group (Attachment K)

12. Client health assessment questionnaire (Attachment L)

a. Statement of need for a rigorous evaluation of PBHCI

As the largest federal effort to implement integrated behavioral and physical health care in community behavioral health settings, SAMHSA’s PBHCI program offers an unprecedented opportunity to identify which approaches to integration improve outcomes, how outcomes are shaped by the characteristics of the treatment setting and community, and which models have the greatest potential for sustainability and replication. SAMHSA awarded the first cohort of 13 PBHCI grants in fiscal year (FY) 2009, and between FY 2009 and FY 2014, SAMHSA funded a total of seven cohorts comprising 127 grants. An eighth cohort, funded in fall 2015, included 60 new grants.

The PBHCI grants are covered under the requirements of P.L. 103-62, the Government Performance and Results Act (GPRA) of 1993; Title 38, section 527, Evaluation and Data Collection, as well as 38 CFR section 1.15, Standards for Program Evaluation. The GPRA requires federal government agencies to evaluate their performance on a regular basis, and the analyses will include items reported as part of GPRA compliance.

HHS’s Office of the Assistant Secretary for Planning and Evaluation conducted the first evaluation of PBHCI in partnership with SAMHSA. OMB approval for this evaluation (OMB No. 0990-0371, expiration September 30, 2014). The first evaluation included only the first 56 grantees funded in FY 2009 and FY 2010, and the evaluation ended in 2013. The evaluation showed that PBHCI grantees implemented programs that were successful in various ways, such as in building integrated, multidisciplinary teams that offer an array of integrated primary, behavioral health, and wellness services. It also identified a number of positive effects of integration on several physical health outcome domains.

Despite promising findings, however, the first evaluation was limited in sample size and the duration of follow-up, and the grant program was still in its infancy and did not yet include the more robust integration requirements of later grant announcements. As PBHCI has evolved with changing grant requirements, a more comprehensive evaluation is necessary to understand how different integration approaches being used by the grantees affect a wider array of outcomes, including behavioral health, quality of life, perceptions of care, and social functioning. Moreover, ongoing evaluation of this large grant program is consistent with recent recommendations from the Government Accountability Office (GAO)¹ that SAMHSA and other agencies conduct formal program evaluations of federal programs targeting people with serious mental illness (SMI) to inform agency strategic planning and program management. This study will provide important feedback on program design, execution, and effectiveness as part of SAMHSA’s evaluation efforts.

In accordance with the current evaluation contract timeline, the evaluation will take place over 60 months, beginning September 1, 2015, and ending September 1, 2020. Data collection for the evaluation will not begin until SAMHSA receives final OMB approval. The data collection described in this request will build upon the first evaluation and provide essential data on the implementation of integrated primary and behavioral health care, along with rigorous estimates of its effects on health.

b. Overview of study design and evaluation questions

To learn about the effectiveness of the PBHCI program, the study team will use a mixed-methods approach.

The evaluation has three components: (1) an implementation study, which will focus on the services that grantees provide and the extent to which programs are implemented as planned, (2) an impact study, which will focus on the effect of the PBHCI programs on quality of care and health status relative to comparison clinics that are not implementing integration activities, and (3) a comparison across PBHCI cohorts to assess how various changes in PBHCI program requirements relate to client outcomes. Table A.1 summarizes the cohorts that SAMHSA plans to include in each component of the evaluation.

Table A.1. Cohorts Included in Each Component of the Evaluation

Evaluation component	Cohorts I–V	Cohorts VI–VII	Cohort VIII
Implementation		X	X
Impact			X
Comparison across cohorts	X	X	X

The study will address five evaluation questions about the implementation and impacts of the PBHCI program:

1. What integrated care structures and clinical practices do PBHCI grantees implement?

2. What integrated care services do PBHCI clients receive, and what are clients’ perceptions of care?

3. What is the impact of PBHCI on physical health, behavioral health, and social functioning outcomes compared with clinics without PBHCI programs?

4. What are the links between integrated care structures, processes of care, and outcomes among current PBHCI clients?

5. Are expanding grantee requirements in the PBHCI program associated with differences in integrated care structures, processes, and outcomes?

2. Purpose and Use of Information

The purpose of the PBHCI grants is to improve the overall wellness and physical health of people with SMI, including people with co-occurring substance use disorders, by helping communities coordinate and integrate primary care services into publicly funded, community-based settings providing mental health and behavioral health care. This data will help SAMHSA assess whether integrated primary care services produce improvements in the physical and mental health of the SMI population receiving services from community-based behavioral health agencies. Each proposed data collection instrument is described below, along with how, by whom, and for what purpose the collected information will be used. Table A.2. provides additional detail about how the content areas in each data collection instrument will be used to answer the evaluation’s key questions.

a. Qualitative data

PBHCI grantee director and frontline staff surveys (Attachments A and B). In years 2 and 4 of the evaluation, the evaluation team will conduct two brief web-based surveys of cohort VI–VIII grantees. The surveys will gather information from (1) the grantee administrator and (2) selected frontline staff who are implementing integration activities and providing services to PBHCI clients. The grantee director survey consists of 64 questions and the frontline staff survey includes 33 questions.

The surveys will be used to systematically gather information about (1) the structures and processes used to support integrated care and (2) the successes and challenges of integration. To facilitate comparisons with earlier cohorts, the evaluation team will use streamlined versions of many of the items that were included in the first PBHCI evaluation’s survey of cohort I–III grantees. The surveys will also be used to assess domains that are relevant to the grant requirements for later cohorts, including questions about the delivery of evidence-based practices (EBPs) and cultural competency training.

Telephone interview protocol (Attachment C). During years 2 and 4 of the evaluation, the evaluation team will conduct telephone interviews with staff from cohorts VI–VIII. In year 2, the team will interview grantees from cohorts VI and VII, who are in the final years of their grant, and grantees from cohort VIII, who are in the initial stages of implementation. The year 4 interviews will focus on cohorts VII and VIII. The team may include some grantees in both phases of the telephone interviews to follow up on how implementation issues develop over time. In most cases, the respondents will be program directors. Additional program staff with knowledge of the topic of interest may also be interviewed.

The evaluation team plans to interview two or three people from each of 10 programs in each of the two years. Telephone interviews will be semistructured and tailored to the grantee’s program characteristics as needed. Questions will address the main structural components of the program, as reported in reports and through the web survey, and issues that arise in the implementation of the program over time. Interviews will be conducted by researchers and/or research associates who will have reviewed the available data on the program before conducting the interview. One interviewer and one note-taker will participate in each interview. In most cases, the respondents will be program directors. Additional program staff with knowledge of the topic of interest may also be interviewed.

The telephone interviews will be used to explore identified implementation challenges and successes in greater depth than can be done through more structured methods. The interviews will also offer an opportunity to explore issues of sustainability, adherence to recovery-oriented principles, and cultural competence, which are not systematically reported through grantee quarterly reports.

Site visit interview protocols. During years 2 and 3, the evaluation team will conduct site visits to 10 strategically selected grantees, with each visit lasting about two business days. Sites will be selected based on data collected from grantee proposals and quarterly reports, plus responses to the first wave of the staff web survey. Each site visit will be completed by two‑person teams who will interview staff and clients using two data collection instruments:

On-site staff interview protocol (Attachment D). This semi-structured instrument will consist of groups of questions from which interviewers will select based on respondents’ position in the clinic, their role in providing primary care services, and the features of the clinic. For example, protocols for peers and case management staff will include questions about their training, certification, specific functions, and billing opportunities and requirements. Questions for a range of clinical care staff will be used to systematically derive information on care coordination, information sharing, referrals and follow-up, perceived level of integration within and across partner services, unmet client needs, implementation of wellness EBPs, and more.

Table A.2. Data Collection Activities, by Data Source

Data source	Mode, timing, and respondent	Evaluation questions (EQs)	Domain	Analysis
Qualitative data sources
Grantee web survey	In years 2 and 4 of the evaluation, the evaluation team will conduct a brief web-based survey of cohort VI–VIII grantees.	EQ1, EQ2, RQ4	(1) Behavioral health, physical, and wellness services available at the clinic (including the provision of specific EBPs relevant to cohort VIII); (2) dimensions of integration (colocation, shared structures and systems, integrated practice, and culture); (3) population health management approaches; (4) perceived implementation successes; (5) challenges and contextual factors that have influenced implementation	Descriptive analyses
Telephone interviews	During years 2 and 4, telephone interviews will be conducted with program leaders from selected active PBHCI sites. In year 2, the team will interview grantees from cohorts VI and VII, and in year 4, the team will interview grantees from cohorts VII and VIII.	EQ1, EQ2	(1) Program structure, resources, and features; (2) cooperation/collaboration across mental health and primary care; (3) evidence-based practice use	Descriptive analyses
Site visits	During years 2 and 3, the evaluation team will conduct site visits with 5 strategically selected grantees (10 grantees total across years 2 and 3), each lasting about two business days.	EQ1, EQ2	Staff interviews: (1) interviewee role; (2) program structure; (3) screening and referral; (4) registry/tracking; (5) cooperation/collaboration across mental health and primary care; (6) evidence-based practices; (7) care management and continuity of care; (8) wellness, illness prevention, and other self-management support; (9) client (and family) involvement; (10) electronic capabilities; (11) performance monitoring; (12) women’s and minority health cultural competency; (13) implementation	Descriptive analyses
			Client focus groups: (1) awareness of primary care services, (2) types of services, (3) availability of services in other clinics, (4) perception of clinic, (5) remaining barriers, (6) preferences for care
Quantitative data sources
Grantee registry/EHR data	The evaluation team will ask cohort VIII grantees to extract a limited set of variables from their clinic registries and/or EHRs.	EQ3, EQ4, EQ5	(1) Physical health, (2) behavioral health, (3) quality of life, (4) wellness EBPs, (5) care management	Hierarchical linear models
Comparison group data	The evaluation team will collect data in years 2–4 from a comparison group of clients with SMI who receive care from clinics that are not funded by PBHCI and have engaged in few or no integration activities.	EQ3	(1) Physical health, (2) behavioral health, (3) quality of life	Descriptive, difference-in-differences

Client focus group guide (Attachment E). During the site visits, the evaluation team will conduct focus groups with clients to understand their experiences with integrated care. The focus groups will be conducted by one moderator and one note-taker. The discussion guide will consist of open-ended questions that address the following topics:

• Clients’ awareness of primary care options

• Types of primary care services that clients use

• Perceptions of the PBHCI clinic as a source of primary care services

• Remaining barriers to primary care

• Clients’ preferences for primary care

• Client experiences with wellness EBPs

As with the telephone interviews, the evaluation team will use data collected through site visits to explore implementation challenges and successes through in-depth case studies. Data collected during site visits will be analyzed qualitatively to illustrate the importance of the implementation issues, the barriers that some grantees face, and the solutions they may have found. Information from the site visits and focus groups will also help to inform additional items the team may add to the second web survey.

b. Quantitative data

Grantee registry/EHR data. The evaluation team will ask cohort VIII grantees to use an electronic data collection tool (Attachment F) to extract a limited set of 42 variables from their clinic registries and/or EHRs using an extraction tool. These variables will provide information on service use, which the evaluation team will use to construct two types of measures: (1) measures of service use and (2) measures of whether the service use is indicative of receiving a basic package of services indicative of integrated care. The evaluation team will use these data to quantify the extent to which PBHCI clients have regular contact with their providers and are engaged in wellness activities. The evaluation team will also use the data extracted from registries/EHRs to calculate intermediate process measures that may reflect the delivery of integrated care.

SAMHSA will likely be most successful obtaining this information from cohort VIII grantees given the more stringent grant requirements regarding development and use of EHRs for this cohort. The evaluation team will ask grantees to submit their data extracts via a secure SharePoint site about once a quarter. The team will provide grantees with technical assistance to help them extract the data and will also provide feedback on the completeness of their data submissions.

c. Primary data collection from comparison group

The evaluation team plans to collect data from a comparison group of clients with SMI who receive care from clinics that are not funded by PBHCI and that engage in few or no integration activities. The team will collect the same physical health, behavioral health, and quality-of-life indicators for comparison group clients that PBHCI grantees are required to capture for their clients, as a condition of the grant. Data from the comparison group will be collected at baseline and one year later to facilitate comparisons with the data captured for PBHCI clients using the client health assessment questionnaire (Attachment L) consisting of 25 questions. The evaluation team anticipates recruiting up to 10 clinics for the comparison group and recruiting up to 250 clients per clinic for baseline assessments. At each comparison clinic, a qualified health provider will be designated to receive and advise clients of their physical exam results. The evaluation team will also work with clinics to retain clients for the follow-up assessment.

d. Timeline for the data collection

The evaluation is expected to be completed in five years, with four years of data collection. Table A.3 shows the schedule of data collection activities.

Table A.3. Timeline for the Data Collection

Data source	Cohort VI (2013–2017)a N = 7	Cohort VII (2014–2018) N = 25	Cohort VIII (2015–2019) N = 60
Web-based grantee director and frontline staff surveys	2016	2016, 2018	2016, 2018
Telephone interviews (10 grantees in each of the two years)	2016	2016, 2018	2016, 2018
Site visits (5 grantees in each of the two years)		2017, 2018	2017, 2018
Registry/EHRb data collection			2016–2019
Primary data collection from comparison sites			2016–2019

^a Years in parentheses indicate the performance period of the grants in the cohort.

^b Registry/EHR data are collected quarterly

3. Use of Information Technology

Web-based surveys will be the primary mode of data collection for the grantee director and frontline staff surveys. These surveys will enable respondents to complete the data collection instrument at a location and time of their choice, and its built-in editing checks and programmed skips will reduce response errors.

Registry/EHR data will be collected from the grantees through a secure SharePoint site. Although SAMHSA will specify the required data elements, SAMHSA will accept any format in which the data are provided in order to reduce the burden on the grantee. All data will be converted to a consistent format so that they can be combined with data submitted by other grantees and be suitable for analysis.

4. Efforts to Identify Duplication

In formulating the evaluation plan, SAMHSA has carefully considered how to minimize burden by supplementing existing data sources with targeted primary data collection. To this end, the evaluation incorporates the following approaches:

• Using data from existing grantee quarterly reports and sustainability plans while conducting supplemental primary data collection: : To the extent possible, information regarding grantee implementation and sustainability issues will be gathered through a review of the quarterly reports and sustainability plans that grantees have been preparing and submitting to their SAMHSA project officer over the normal course of the PBHCI program (these data collection efforts were approved under OMB No. 0930-0340). However, the level of detail and consistency of the information provided in these reports varies from grantee to grantee. To supplement data gathered from these reports, SAMHSA is requesting OMB clearance to conduct a web-based survey, site visits, and interviews. The survey will provide comparable data for analysis across all evaluation sites. The evaluation team will also use the information gleaned from visits to 10 grantees to further supplement the data from the quarterly reports and surveys; the questions asked during these visits have been tailored to different stakeholders to minimize the time that grantee staff must spend in interviews. Interviewers will only collect information that is not available through any other mechanism or source. Furthermore, the information collected during the visits will be distinct from the data gathered via any agency-monitoring activities that may occur during the grant period.

• Registry/EHR data: The data requested from clinic registries/EHRs are intended to complement data that clinics report to SAMHSA in the normal course of participation in the grant program and do not represent a duplication of effort.

5. Involvement of Small Entities

Grantees vary in size, from small entities to large provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required to effectively evaluate the PBHCI program. Further, the use of an external contractor for the client physical exam and health assessment questionaire is intended to reduce the burden on the comparison group clinics. Finally, a number of the grantees have contracted with external evaluators who are assisting with data collection and reporting.

6. Consequences if Information Collected Less Frequently

Each of the data sources provides information needed for the evaluation. If the data are not collected, the evaluator will not have adequate information to answer the five evaluation questions. The inclusion of all planned data sources is necessary to glean information about structure, process, and outcomes and to obtain a complete picture of the quality of care.

Telephone interviews and site visits will take place only twice during the evaluation. If they were not completed, the evaluator would not have adequate information to assess the structure and processes of care in place to answer evaluation questions 1 and 2. The web-based survey will provide information about the structure and processes of care across all grantees, but not with the level of detail afforded by site visits. The registry/EHR data will provide valuable information on individual service use that will be important in examining the relationship between client service use and outcomes. Comparison group data are necessary to assess the comparability of the intervention and comparison groups to ensure that they are not imbalanced on any key demographic or service use characteristics.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d)(2).

8. Consultation Outside of Agency

The notice required in 5 CFR 1320.8(d) to solicit public comments was published in the Federal Register on March 11, 2016 (81FR12917). No comments were received.

The PBHCI evaluation plan was developed in consultation with a nine-member technical expert panel (TEP) made up of specialists in evaluation design and implementation, mental health services research, and the integration of community-based behavioral health services and primary care services, along with consumers of mental health services who have evaluation experience. The TEP met once via webinar to review the evaluation and data collection plan and to respond to questions related to the evaluation design. A second TEP meeting will be held by phone with the purpose of soliciting comments on the findings. Table A.4 lists the TEP members.

Table A.4. TEP Members

Name	Title and affiliation
Anne Sullivan, M.D.	Commissioner, New York State Office of Mental Health
Joseph Parks, M.D.	Medical director, Missouri Medicaid
James Schuster, M.D.	Medical director, Community Care Behavioral Health Center
Margaret Park	Recovery specialist, wellness project manager, Allegheny County Department of Human Services
Jonathan Delman, Ph.D., J.D.	Physician, principal researcher, Department of Psychiatry, University of Massachusetts Medical School
Jurgen Unutzer, M.D., M.P.H.	Chair, department of psychiatry and behavioral sciences, University of Washington
Steve Bartels	Director of mental health and chemical dependency services and chair of the chiefs of psychiatry, Dartmouth Institute
Imara West, M.P.H.	Research scientist 3, department of psychiatry and behavioral sciences, University of Washington
Octavio Martinez, M.D.	Executive director/physician, Hogg Foundation

9. Payment to Respondents

An incentive of $25 in the form of a gift card to a local chain store (such as Target or Walmart) will be paid to all clients at the comparison clinics who participate in the baseline physical exam and health assessment. They will receive an additional $25 gift card at the one-year follow-up exam and health assessment. Focus group participants will also receive a $25 gift card to the same local chain store.

10. Assurance of Confidentiality

Web-based survey: The first page of both web-based surveys will include language noting the protection and privacy of respondents’ answers will be protected and kept private, to the extent of the law. Respondents will provide informed consent for the survey by checking a box at the bottom of the first page of the web instrument. The respondent will not be able to proceed to the survey unless the box is checked. All survey information will be maintained on a secure system and respondents will be identified by unique numeric IDs and not names or other identifying information.

Telephone interviews: Prior to the start of the interview, directors interviewed will be reminded that the information gained will be used for evaluation purposes only and will not be attributed to any individual.

Site visit interviews: Prior to the start of the interview, all staff interviewed will be reminded that the information gained will be used for evaluation purposes only and will not be attributable to any individual.

Client physical exam and health assessment: Clients, at the comparison clinics, who express interest in participating in the physical health exam and health assessment will be asked to complete and sign a consent form (Attachment G) allowing their center’s staff to release their name, home address, email address, and phone number to Mathematica’ and Onsite Health Diagnostics for the purposes of providing more information about the study and determining the client’s eligibility to participate. Mathematica will assure the respondent of the privacy of information collected in basic language in an advance letter (Attachment H) which will be mailed to each potential respondent about two weeks before they are contacted. The language used in the letter will be close to a 6^th grade reading level. In the introduction to the physical exam and health assessment, the respondents will be reminded about the voluntary nature of their participation and that information collected will meet all requirements of the Health Insurance Portability and Accountability Act (HIPAA) and the Privacy Act. All participants will be required to read and sign a consent form (Attachment I) and HIPAA authorization form (Attachment J) before beginning the physical exam and health assessment.

Focus groups: Prior to the start of the focus group, participants will be asked to read and sign a consent form (Attachment K) and members of the evaluation team will be available to answer questions.

All data collection instruments and supporting documents will be approved by the evaluation contractor’s Institutional Review Board.

11. Questions of a Sensitive Nature

The client health assessment for the comparison group will involve questions about medical history, mental health, diet, exercise, and any medications the client may take. Clients will also be asked to report on drug and alcohol use. These topics may be sensitive, but this information is necessary to answer the evaluation questions. Assessment and treatment of co-occurring substance use disorders are key components of the PBHCI program. In addition, the health assessment will involve questions about sexual orientation; experiences involving violence; and whether the client has spent time homeless, been hospitalized for mental health care, spent time in a facility for detox or substance abuse treatment, or spent time in a correctional facility. PBHCI grantees ask these same questions of their clients at enrollment into the PBHCI program and at regular follow-up intervals. Grantees collect these data using the OMB-approved TRansformation ACcountability (TRAC) National Outcomes Measures (NOMs) Client-Level Measures for Discretionary Programs Providing Direct Services tool (OMB No. 0930-0285). This tool was approved by OMB for use with clients of all SAMHSA discretional grant programs, including PBHCI.

SAMHSA will be collecting these data from comparison group for two reasons. First, this information will allow us to examine the comparability of the intervention and comparison groups to ensure they are not imbalanced in terms of key demographics or service use. Second, several of these items, such as the number of hospitalizations and receipt of substance abuse treatment, are key outcomes for our impact analyses.

12. Estimates of Annualized Hour Burden

Table A.5. provides estimates of the average annual burden for collecting the proposed information. Details on the time and cost burdens are provided below for each of the separate data collection activities.

• Web-based survey: For the director and front line staff surveys administered in 2016, SAMHSA anticipates an 85 percent response rate for both the PBHCI grantee director and grantee frontline surveys, for a total of 78 completed director surveys (92 directors x 0.85) and 782 frontline staff (92 grantee sites x 10 frontline staff x 0.85). SAMHSA estimates the web-based surveys will require an average of 30 minutes for the director survey (78 directors x 0.5 hours) and 30 minutes for the frontline staff survey (782 frontline staff x 0.5 hours). For the second survey administration conducted in 2018, after funding for the seven cohort VI grants end, SAMHSA anticipates an 85 percent response rate for both the PBHCI grantee director and grantee frontline surveys, for a total of 72 completed director surveys (85 directors x 0.85) and 723 frontline staff (85 grantee sites x 10 frontline staff x 0.85). SAMHSA estimates the web-based surveys will require an average of 30 minutes for the director survey (72 directors x 0.5 hours) and 30 minutes for the frontline staff survey (723 frontline staff x 0.5 hours).

• Telephone interviews with grantee directors: Sixty interviews with PBHCI grantee directors will be conducted (30 in year 2 and 30 in year 3), lasting about one hour each (60 directors x 1 hour).

• Grantee site visits: During each of the 10 grantee site visits, the following interviews will occur:

• Interview with the director lasting 2 hours (10 sites x 1 director x 2 hours)

• Interviews with 4 mental health providers lasting 1 hour each (10 sites x 4 mental health providers x 1 hour)

• Interview with 4 primary care providers lasting 1.5 hours each (10 sites x 4 primary care providers x 1.5 hours)

• Interviews with 2 care coordinators lasting 1.5 hours each (10 sites x 2 care coordinators x 1.5 hours)

• Client focus groups: SAMHSA estimates each focus group will take 45 to 60 minutes. Each group will consist of 10 to 12 clients. (10 comparison clinics * 12 clients * 60 minutes)

• Extraction of grantee registry/EHR data: SAMHSA estimates that the extraction of clinical registry/EHR data from grantees could require up to eight hours if done manually. It will be completed up to 11 times by all 60 grantees (60 x 11 x 8 hours).

• Physical exams and health assessments for comparison group clients: SAMHSA estimates that it will require an average of one hour to complete the exam and assessment with each client at the 10 comparison group clinics, including time to provide an introduction, complete the informed consent form, provide reassurance of confidentiality, and answer questions. The physical exam and health assessment will be completed at two points in the evaluation period (250 clients/site x 10 comparison group clinics x 1 hour at baseline) + (175² clients/site x 10 comparison group clinics x 1 hour at follow-up). SAMHSA estimates requiring eight hours to coordinate with the comparison group’s clinic directors to arrange and conduct the exams and health assessments (10 directors x 8 hours).

Table A.5. Estimated Annualized Burden Hours

Respondents/activity	Number of respondents	Responses per respondent	Total responses	Hours per response	Total hour burden	Hourly wage costa	Total hour cost ($)
Web surveys
Grantee director	78	2	149b	0.5	75b	$40	3,000
Grantee frontline staff survey	782	2	1,494c	0.5	747c	$40	29,880
Phone interviews
Grantee director	60	1	60	1.0	60	$40	2,400
Site visits
Grantee director—site interview	10	1	10	2.0	20	$40	800
Grantee mental health providers—site interview	40	1	40	1.0	40	$40	1,600
Grantee primary care providers—site interview	40	1	40	1.5	60	$40	2,400
Grantee care coordinators—site interview	20	1	20	1.5	30	$28	840
Focus groups
Focus group participants	120	2	240	1.0	240	$15	3,600
Extraction of grantee registry/EHR data
Extraction of grantee registry/EHR data	60	11	660	8.0	5,280	$40	211,200
Physical exam and health assessment
SMI clients—baseline physical exam and health assessment	2,500	1	2,500	1.0	2,500	$15	37,500
SMI clients—follow-up physical exam and health assessment	1,750	1	1,750	1.0	1,750	$15	26,250
Comparison group clinic director—coordinationd	10	1	10	8.0	80	$40	3,200
Total	3,720e		6,973		10,882		322,670

^a Hourly wage estimates are based on salary information provided in 10 PBHCI grant proposals representing mostly urban locations across the country and represent an average across responders of each type.

^b Cohort VI funding ends before the administration of the second survey. Total number of responses excludes the Cohort VI directors, who will not receive the second survey.

^c Cohort VI funding ends before the administration of the second survey. Total number of responses excludes the Cohort VI frontline staff, who will not receive the second survey.

^d Includes logistical coordination between the evaluation and site staff to conduct the physical exam and health assessment as well as oversight of client recruitment.

^e Excludes physical exam and health assessment follow-up respondents.

13. Estimates of annualized cost burden to respondents

There will be no capital, start-up, operation, maintenance, or purchase costs incurred by the sites participating in data collection for the evaluation.

14. Estimates of annualized cost to the government

It is estimated that two SAMHSA employees will be involved for 10 percent of their time. Annual costs of SAMHSA staff time are estimated to be $22,000. Additional costs are 100 percent of the contract awarded for the conduct of the PBHCI evaluation by SAMHSA ($6,764,368 over 5 years or an annualized cost of $1,352,874). The total estimated average cost to the government per year is $1,374,874.

15. Changes in burden

This is a new information collection request.

16. Time schedule, publication and analysis plans

The evaluation contract for the PBHCI grant program anticipates that aggregate results from the national evaluation will be incorporated in text and charts in the following publications:

• Four annual reports due August 2016, 2017, 2018, and 2019

• A final evaluation report describing evaluation data collection, analysis, and findings, due July 6, 2020

• Brief analytic reports of no more than five pages to be distributed through the SAMHSA Center for Behavioral Health Statistics and Quality Short Report mechanism or similar vehicles; these short reports are designed to provide brief updates using bullet points, figures, charts, and tables to be easily understood by a wide range of audiences

• About two ad hoc data requests (such as presentations, blog postings, and podcast interviews) per year in 2017 through 2020 as well as additional optional publications, presentations, and trainings as directed by SAMHSA

Table A.6. provides an overview of the evaluation tasks and in which years the tasks will be carried out.

SAMHSA may also incorporate the aggregate results from the cross-site evaluation into journal articles, scholarly presentations, and congressional testimony related to the outcomes of the PBHCI grant program.

Table A.6. Evaluation tasks time line

Evaluation time line	Base year	Option year 1	Option year 2	Option year 3	Option year 4
Development of evaluation plan and instrumentation	X
Expert panel meeting	X				X
OMB and IRB approval	X
Document review: grantee applications, Behavioral Health Integration Capacity Assessment, and quarterly reports	X	X	X	X	X
TRAC analysis	X	X	X	X	X
Recruitment of comparison clinics	X
Data collection from comparison clinics		X	X	X
Collection of grantee registry/EHR extracts		X	X	X	X
Grantee survey and telephone interviews		X		X
Site visits		X	X
Grantee technical assistance (focused on registry/EHR extracts)	X	X	X	X	X
Comparisons of cohorts	X	X	X	X	X
Impact analyses				X	X
Analyses of links between structure, process, and outcomes				X	X

17. Display of expiration date

The expiration date will be displayed.

18. Exceptions to certification statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

1 “Report to the Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, House of Representatives: Mental Health: HHS Leadership Needed to Coordinate Federal Efforts Related to Serious Mental Illness.” GAO-15-113. Washington, DC: GAO, December 2014.

2 SAMHSA expects a 30 percent attrition rate.

3