OMB Ss-b_7_23_15.

Behavioral Health Information Technologies Survey

SUPPORTING STATEMENT

2. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods

The respondent for this survey is a subset of grantees currently funded by three Centers within the Substance Abuse and Mental Health Services Administration (SAMHSA): the Center for Mental Health Services (CMHS), the Center for Substance Abuse Prevention (CSAP), and the Center for Substance Abuse Treatment (CSAT.) The Center Director for each of these Centers’ nominated grant programs to be included in the survey. This reflects the intent to sample programs that represent the various types of services and providers supported by SAMHSA and variations in current HIT capacity.

All active grantees in the program selected will be surveyed except for those in their first 3 months of their initial year. This group of grantees is to be excluded because they grantees are engaged in the start-up phase of grant operations, a period where staffing and operations are still under development.

The list of grant programs, numbers of grantees in each program, and the Center funding the grant program are listed in the table below.

Grant Programs, Grantees, and Center Funding
Grant Program	Estimated number of grantees	Center(s) Funding
Screening Brief Intervention and Referral to Treatment (SBIRT)	18	CSAT
Grants to Expand Care Coordination through the use of Technology-Assisted Care in Targets Areas of Need (TCE/TAC)	17	CSAT
Offender Re-Entry Program	13	CMHS/CSAT
Primary Behavioral Health Care Integration (PBHCI)	89	CMHS
National Child Traumatic Stress Initiative (NCTSI)	56	CMHS
Suicide Lifeline Crisis Center Follow-up	12	CMHS
Garrett Lee Smith Youth Suicide Prevention Program	56	CMHS
Minority AIDS Initiative (MAI)	113	CSAP

Measures will be taken to ensure a response rate of at least 75 percent. These efforts include:

1. Conducting a pilot of the survey among a small (nine) representative group of potential participants and analyzing the results using post-survey interviews. Please see Attachment 4 for the Pilot Survey Protocol and related documents.

2. Developing and implementing a survey promotion and launch strategy that leverages the influence of agency leadership

3. Developing and implementing effective follow-up processes and procedures to be applied uniformly to all “late” (post 2-weeks) responders. As detailed below in Section B.2.b Communication with Targeted Respondents we will send reminder emails at weeks 2, 5, and 7 and possibly including telephone calls during the final follow-up.

B.2. Information Collection Procedures

Survey design and construction, especially question selection was informed by the research questions the survey is intended to address and the domains identified to guide selection of HIT areas of interest. These domains include:

1. EHRs and practice management systems

2. Interoperability/health information exchange functionality

3. Telehealth

4. Mobile tools

5. Web portals

6. Consumer engagement tools

7. Personal health records

8. Dashboard/tools for integration of care

The Behavioral Health Information Technologies and Standards (BHITS) project team developed a comprehensive Data Dictionary, to ensure that the key terms related to technology aligned with published definitions of terms. The Data Dictionary is included as Attachment 1—Data Dictionary. It ensures consensus among project stakeholders regarding the meaning of various information technology terms. It is also the basis for “layman’s definitions” of key terms used in the survey. This ensures common and accurate understanding of the terms among the participants which supports the quality of data collected.

The survey questions were developed by collecting questions used in previous surveys into an Excel workbook. These served as the basis for developing questions specific to the intent and purpose of the BHITS survey. The source of each previous survey question was tracked along with the development of modifications to the question. These modifications tailored the questions to the survey areas of interest. Modifications were proposed and developed in a series of iterative teleconferences. To address gaps, some questions were crafted by the BHITS project team members. The final set of survey questions was methodically cross-walked to the Research Questions the survey will answer, and the subject Domains. The summary of this crosswalk, along with the rationale for including each question is included as Attachment 3—Survey Questions Crosswalk.

This approach supports both the quality and the usability of the data collected. The surveys reviewed for question sources included:

• HIT Adoption and Readiness for Meaningful Use in Community Behavioral Health, 2012 conducted by the National Council for Behavioral Healthcare (NCBH)

• National Survey of Substance Abuse Treatment Services (N-SSATS), 2013 sponsored by SAMHSA

• National Mental Health Services Survey (N-MHSS), 2014 sponsored by SAMHSA

• Assessing Adoption of Effective Health Information Technology Questionnaire, 2005 conducted by Medical Group Management Association (MGMA) Center for Research

• Community Chronic Care Network (CCCN): Technical Readiness, date unknown conducted by Pajaro Valley Community Health, Watsonville CA

• Community Clinics Initiative Information Management Assessment, 2005 conducted by the Community Clinics Initiatives (CCI), San Francisco, CA

• HIMSS Analytics Mobile Technology Survey, 2013 conducted by the Healthcare Information and Management Systems Society (HIMMS)

• Telehealth in State Substance Use Disorder (SUD), 2009, Conducted by the National Association of State Alcohol and Drug Abuse Directors (NASADAD)

Each of the eight domains were methodically addressed by organizing each of the survey questions by domain into one of 6 (six) Sections. These include:

• Section 1: Grantee Program Information and Characteristics

• Section 2: Grantee Program Information Technology Infrastructure

• Section 3: Use of Certified Electronic Health Record Technology in Mental Health and Substance Use Disorder Treatment and Prevention

• Section 4: Use of Telehealth (includes Telemedicine) in Mental Health and Substance Use Disorder Treatment and Prevention

• Section 5: Use of Mobile Technology (mHealth) in Mental Health and Substance Use Disorder Treatment and Prevention

• Section 6: Use of Social Media in Mental Health and Substance Use Disorder Treatment and Prevention

Each section of the survey was organized to support the effective and efficient implementation of “skip logic,” with stem questions located at the beginning of the section. The stem question determines whether a set of questions is relevant to the respondent. If not, the respondent is automatically forwarded, or “skipped” to the next relevant question or section. The minimum time for the average respondent to complete the survey is 20 minutes, with the maximum time of 30 minutes for a respondent who skips the fewest and therefore answers the most questions. The average response time is estimated to be 22 minutes.

The questionnaire was designed with the SAMHSA grant-funded program as the unit of analysis. That is, questions were directed at understanding the use of HIT in activities directly supported by the grant, rather than the use of HIT by the organization managing the division or department where the grant was being implemented.

The expectation is that the grant project director or their designee will be responsible for ensuring that the survey is completed and will be most familiar with the grant activities and associated HIT activities and concerns. This approach would not prevent the project director from seeking input from others in the organization or asking them to complete portions of the survey. Please see Attachment 1—Survey Instrument for a paper-based copy of the final version of the survey.

Data collection procedures include three key areas: (1) developing information required for the survey frame, (2) communicating with targeted respondents; and (3) Project Officer support. These key components are detailed below.

B.2.a. Information Development for the Survey Frame

The eight grant programs identified in Table 1 will be included in the survey. These eight programs were identified by the SAMHSA Center Directors of CMHS, CSAT, and CSAP. This approach ensured a diverse representation of SAMHSA grant programs for the sample frame.

The sample frame for the survey will be all grantee in each of the selected programs except those in the 90-day start-up or wind-down period. As noted earlier, newly funded grants in the start-up phase do not have sufficient experiences to report on. Along with information on the grant this should the list of grantees included contact information for the project director for each grant, an alternate point of contract with the grantee, typically the project manager, and the name of the SAMHSA Project Officer. This grantee information will be used to pre-fill basic information in the survey and we will use the project director information to distribute the survey.

The SAMHSA Project Officers are the primary source for verifying that contact names and information are accurate.

B.2.b. Communication with Targeted Respondents

Two keys to obtaining participation in a survey are to establish the importance of the information provided and to limit the amount of effort required by the respondent. Survey deployment strategy activities are intended to strengthen individual participant understanding of the nature of the survey and also to support the highest possible response rate. The deployment strategies are carefully timed around the schedule for survey distribution and collection.

As part of establishing the importance we will send lead letter via email to all potential respondents from a SAMHSA authority on the Agency’s letterhead. This lead letter will come under the signature of the lead of the HIT Strategic Initiative at SAMHSA, and this letter would go out approximately one week prior to the release date for the survey. The lead letter can be found in Attachment 5.

Limiting the amount of effort required by respondents includes keeping all communication clear and succinct along with limiting the time required to complete the survey. Any emails or other communication will be designed so that the reader can grasp the key elements and what is expected of them from the first few lines.

On the release date of the survey a customized email will go out to each potential respondent from BHITS project team members. This email will contain a link to the survey site. The email will identify SAMHSA as the sponsor of the survey and reference the lead letter sent the previous week. It will also contain information on how to obtain assistance if the respondent has questions or technical issues with completing the survey. This will include both an email address and a toll-free number for obtaining help should questions or issues arise. (These same contact points will also be listed on the first and last pages of the electronic version of the survey).

We will use a 40-60-day data collection period to allow sufficient time for respondents to complete the survey and our staff to undertake efforts to encourage this participation. During the first 2 weeks of data collection, the BHITS project team will conduct two to three webinars to present the instrument and respond to the FAQs prompted by the survey release.

Grantees who have not responded will be sent reminder emails at weeks 2, 5, and 7, and receive telephone follow-up. This schedule is designed to establish and maintain a positive relationship with the potential respondent, encouraging their continued engagement and interest while assuring them of the importance of their participation. The communication strategy will be evaluated for effectiveness, and modified if it does not effectively encourage a response. For example, instead of sending only an email at week 7, the BHITS project team may decide to also place telephone calls to each non-responder, asking them to expect the reminder email and reminding them it is not too late to respond.

B.2.c. Project Officer Support

The Project Officers for each of the sampled grantees, who know and are in frequent contact with the members of this group, will provide valuable support in encouraging grantees to respond. These Project Officers will communicate to their grantees the importance of the survey information to SAMHSA and how the information will be used. The Project Officer will be asked to engage the grantees on survey participation after the lead letter promoting the survey is released, but before the initial email from the BHITS project team. They will provide additional encouragement to non-responders to complete the survey, as part of their normal communications with the grantees. Their support will also underscore the high-level of importance the agency is placing on the data collection effort.

Project Officer support for the survey will come in addition to their regular workload and we will implement several strategies to encourage their assistance. The earlier that they can be involved in the survey process, the more likely they will understand the value of the information to SAMHSA. A prime opportunity is to encourage Centers to include the Project Officers of the selected grant programs in the Center review process that is scheduled for mid-February to mid-March. Feedback from Project Officers who regularly work with the targeted grant programs know these providers best and are well positioned to provide guidance on the relevance of the questions for this population.

During the actual data collection period, each Project Officer would be provided with regular information on the status of responses from his or her grantees, and suggestions for including mention of the survey during regularly scheduled interactions with the grantees. The BHITS project team would prepare bi-weekly reports, with response data organized by Project Officer and by grant program, listing each grantee and whether or not they have responded to the survey. The report will also indicate actions taken by the BHITS team to encourage responses.

B.3. Methods to Maximize Response Rates

Essential to the success of this data collection is attaining a high response rate to the survey itself. To do so, efforts will be undertaken to ensure clear and easy communication between respondents and survey administrators. As noted, SAMHSA will implement a comprehensive strategy to maximize the response rate, which will include the following methods:

1. Survey length will be kept to a minimum of 20 minutes and a maximum of 30 minutes.

2. The survey will be promoted by the program government Project Officers and by agency leadership. The importance of the survey will be identified and the value of the data to the respondents will be highlighted.

3. The survey itself will be deployed on the internet, to make completion of the survey faster and simpler for the participants.

4. A methodology for non-responder follow up will be developed and implemented using email and telephone. Non-responders will be tracked to individual SAMHSA programs to determine if one or more programs in particular have a high non-response rate. In this case, the government Project Officer’s assistance would be requested in encouraging the grantees in that program to respond.

5. Survey participants will have access to the final survey report. Research indicates that when data is made available to participants, the quality of data reported improves and the response rate increases.

6. The planning and administration of the survey will identify and address potential barriers to participation, especially those related to the web-based technology to be used.

Messaging, including scripts to guide follow-up efforts, will be reviewed by staff that specialize in communication, to help ensure clarity and effectiveness.

B.4. Tests of Procedures

In order to mitigate any undue burden and to maximize the effectiveness and utility of the survey instrument, a pilot test was conducted. The pilot tested whether the questions were clearly stated and that they captured the intended information. The results of the pilot were used to inform efforts to refine the wording of questions that were not clear and to clarify questions that were not understood. The pilot test was also used to determine the actual time of administration for the intended audience and to ensure that the target range was achieved.

The pilot was conducted with nine grantee project managers listed in the survey frame, with invitations to participate based on recommendations from Project Officers from the various grant programs to be surveyed. The grantee project managers who accepted the invitation were asked to complete the survey at a pre-arranged time, noting the total time for completion. Participants were provided with information concerning the nature and intent of the data collection effort, and a paper-based, loosely formatted tool for taking notes on issues as they emerged and their observations about the survey. Please see Attachment 4—Pilot Protocol, Schedule, and Documents for the Pilot Survey Protocol, schedule, and related documents used to test approaches to messaging.

Survey analyses will use univariate and bivariate presentations of the distribution of key variables in tabular format related to core areas of the survey: basic technology infrastructure, the current status of EHR adoption including grantees assessed barriers, the use of various telehealth approaches and mobile technologies, and how grantees use social media.

Cross-tabulations will examine these key variables by organizational characteristics including treatment focus (mental health, substance abuse, behavioral health), size of the organization defined by number of FTEs and budget, ownership, billing for services, and degree of linkage with primary care.

Data collected for this project will be valuable information that can inform the HIT literature for behavioral health. The reports or publications that come out of this project will not attempt to make any national estimates based on the information collected from this project. Instead, they will focus on the usability properties of the instrument. In addition, the data will help inform future performance reporting on HIT for SAMHSA programs.

B.5. Statistical Consultants/Individuals Collecting and/or Analyzing Data

The survey instrument was designed and developed under a contract with FEi Systems, Inc. (FEi), which has a subcontract with the National Council for Behavioral Health (National Council). The BHITS project team, which includes the National Council, is responsible for the data collection and analysis of the information collection.

The staff responsible for data collection and analysis are employed by the National Council and include Colleen O’Donnell, MSW, PMP, CHTS-IM and Sarah Perry, MPA. Both of these individuals have extensive experience in survey design, data collection and analysis, HIT, and HIE and the unique challenges faced by behavioral healthcare providers in the changing HIT and HIE landscape. In FY 2012, Ms. O’Donnell led the successful National Council technical assistance effort on the selection and implementation of certified EHR technology in support of more than 50 behavioral health providers implementing the SAMHSA HIT Supplement to the Primary and Behavioral Health Care Integration grant (PBHCI). In her role as the Policy and Practice Improvement Specialist for HIT and HIE, she continues to provide a wide range of technical assistance on this topic area to PBHCI program grantees and to National Council members. Ms. Perry brings the behavioral healthcare provider perspective on HIT and HIE, after playing a key role on the team adopting and implementing an EHR to successfully attest for Meaningful Use.

These individuals will initiate the data collection activities and supervise at least three National Council staff in the implementation of the non-responder follow-up plan. They will ensure the provision of bi-weekly reports on the survey response rate, and initiate interventions in the event that the response rate is not satisfactory.

SAMHSA Officers for this Project

Government Project Officer: Kathryn Wetherby

Public Health Advisor, Center for Substance Abuse Treatment

Substance Abuse and Mental Health Services Administration

Address: 1 Choke Cherry Road, Room 5-1056 Rockville, MD  20857

Daytime Phone: (240) 276-2899

Email: [email protected]

Government Task Lead: Thomas Clarke PhD

Social Science Analyst, Center for Behavioral Health Statistics and Quality

Substance Abuse and Mental Health Services Administration

Address: 1 Choke Cherry Road, Room 2-1002, Rockville, MD 20857

Daytime Phone: (240) 276-0493

Email: [email protected]

BHITS Project Personnel

Kazi Ahmed, PhD BHITS Project Director FEi Systems Address: 7175 Columbia Gateway Drive, Suite A Columbia, MD 21046 Daytime Phone: (571) 252-5504 Email: [email protected]	Colleen O’Donnell, MSW, PMP, CHTS-IM Policy and Practice Improvement Specialist National Council for Behavioral Health Address: 1400 K Street NW, Suite 400 Washington, DC 20005 Daytime Phone: (202) 684-7457 X 278 Email: [email protected]
Bill Luckey, PhD Evaluation and Reporting Lead FEi Systems Address: 7175 Columbia Gateway Drive, Suite A Columbia, MD 21046 Daytime Phone: (301) 467-6378 Email: [email protected]	Sarah Perry, MPA Practice Improvement Manager National Council for Behavioral Health Address: 1400 K Street NW, Suite 400, Washington, DC 20005 Daytime Phone: (202) 684-7457, ext. 291 Email: [email protected]

LIST OF ATTACHMENTS

Attachment 1—Survey Instrument

Attachment 2— Data Dictionary: Definitions of Key Terms and Concepts

Attachment 3—Survey Questions Crosswalk

Attachment 4—Pilot Protocol, Schedule, and Documents

Attachment 5—Sample Grantee Invitation Letter

Attachment 6- Screen Shots of Survey

Attachment 7- Public Comments

Attachment 8- Responses to Public Comments

8