Supporting Statement

SAMHSA User Feedback Survey – Medication-Assisted Treatment (MAT) Application

Office of Communications, Public Engagement Platform

Customer Feedback Surveys

A. Product/Activity to be Assessed

The Substance Abuse and Mental Health Services Administration’s (SAMHSA) Office of Communications (OC) Public Engagement Platform (PEP) is committed to expanding the availability of SAMHSA’s content for health professionals and the public, and enabling SAMHSA’s key messages and evidence-based materials to reach existing and new audiences. In alignment with the Federal Digital Strategy, SAMHSA OC is participating in an ongoing effort to make a growing proportion of its products available through an array of digital dissemination channels, such as mobile applications (apps)—namely, the Medication-Assisted Treatment (MAT) app. Feedback from app users is essential to enhancing the quality and utility of SAMHSA’s products and services. For this reason, SAMHSA’s OC is requesting Office of Management and Budget approval for the following customer satisfaction feedback surveys:

• The SAMHSA Medication-Assisted Treatment (MAT) App: User Feedback Survey (Attachment 1) will assess users’ satisfaction with the content and functionality of the MAT App. A link to a web-based survey will be embedded directly within the mobile app. It will include multiple choice questions, fill-in-the-blank items, and Likert scale items (extremely likely, likely, neutral, unlikely, not at all likely).

B. Brief Statement of Objectives

This survey will assess participant satisfaction with SAMHSA’s services and products. The data collected from the evaluation will include the following components:

• SAMHSA MAT App: User Feedback Survey

  • Specific satisfaction – responses will determine mean satisfaction across multiple items, indicating the extent to which participants are satisfied with aspects of the MAT App.

  • Personal satisfaction – participants’ comments and recommendations will provide rich, contextualized information about personal app usage.

OC and the contractor will use the data gathered by the survey to assess user demographics, determine the nature of app usage, and assess priorities for future app enhancements. To analyze the data, the contractor will compute descriptive statistics (e.g., weighted mean and frequency) for the Likert scale and summarize the open-ended comments.

C. Overview of Methods to Collect the Information

Data collection method.

• The contractor will administer the SAMHSA MAT App: User Feedback Survey electronically using a web-based survey. Mobile app users can access the embedded survey within the app. The survey uses commonly asked questions to solicit feedback regarding participant satisfaction with the mobile app.

Method for identifying respondents.

• All individuals who access the MAT App will be encouraged to complete the survey. Anyone who accesses the app will be eligible to complete the survey.

Proposed sample size and rationale.

• Based on the contractor’s experience with similar projects, the contractor estimates that approximately 1,750 unique individuals will use the contractor’s mobile apps monthly. All of these individuals will be able to complete this survey. Assuming that 1 percent of these individuals complete this survey, 18 mobile app users will complete this survey monthly /216 per year.

Planned frequency of information collection.

• The message encouraging individuals to complete the User Feedback Survey and the embedded link to the online survey will be consistently available for app users to access.

Time period over which the information will be collected.

• This survey will remain active for the foreseeable future, until the decision is made to remove the product from the site or to terminate data collection.

Expected response rate and plan for follow-up, if any, of nonrespondents.

• The contractor estimates that approximately 1 percent of app users will complete the survey. The contractor will embed a link to the survey within the app with text that encourages users to complete it. There will be no follow-up with respondents, unless specifically requested in the open-ended comments.

Expected ability to assess nonresponse bias using existing information.

• The contractor does not anticipate any nonresponse bias based on existing information.

Methods used to maintain customer privacy.

• Responses will be anonymous. The introductory text for the survey explains this for respondents. Respondents will reply to the surveys using a common survey link, so it will not be possible to identify respondents unless they self-identify in the open-ended comments. Since this uses the same link for everyone, it is assumed that the possibility of duplicate responses would be from the same person. However, SAMHSA does not anticipate duplicate responses.

D. Annual Response Burden Estimate

The following table provides information about the annual response burden estimated for this consumer feedback survey.

Respondent	Number of Respondents1	Number of Responses/ Respondent2	Hours/ Response in Minutes3	Total Number of Responses	Annual Burden Hours	Hourly Wage4	Annual Cost ($)
SAMHSA Medication-Assisted Treatment (MAT) App: User Feedback Survey	216	1	0.05	216	10.8	$71.90	777

¹ For the mobile apps, the number of respondents has been estimated using projections for app usage.

² For the purposes of estimating annual cost, the contractor assumes that community members will complete each survey only once.

³ The average burden per response was estimated based on independent review of the survey by contractor staff.

⁴ Contractor staff reviewed 2013 average salary estimates of occupations from the U.S. Department of Labor’s Bureau of Labor Statistics, at http://www.bls.gov/oes/current/oes_nat.htm, to calculate the mean hourly wage estimates. The mean hourly wage is estimated at $71.90 for MAT App respondents based on a mean yearly salary of $149,559.

E. Methods Used To Develop and Test the Survey

The contractor prepared the consumer feedback survey. The contractor circulated several draft versions that were critiqued by the contractor’s management team and other staff for appropriate content and clarity of the survey.

F. Federal Project Officer and Contractor

Philip Passarelli

Federal Project Officer

SAMHSA/OA/OC

5600 Fishers Lane, Room 18E35D

Rockville, MD 20857

Phone: (240) 276-2136

Fax: (240) 276-1870

Email: [email protected]

Ileana Quintas

Project Director and CEO

IQ Solutions, Inc.

11300 Rockville Pike, Suite 901

Rockville, MD 20852

Phone: (240) 221-4230

Fax: (240) 984-1473

Email: [email protected]

Stephanie Adams

Vice President, Health Intelligence

IQ Solutions, Inc.

11300 Rockville Pike, Suite 901

Rockville, MD 20852

Phone: (240) 221-4353

Fax: (240) 984-1473

Email: [email protected]

G. Project Statistician and Data Collection Entity

• The contractor will collect and compile data using a web-based survey. The use of compiled and analyzed results is at the sole discretion of OC.

List of Attachments

Attachment 1: SAMHSA Medication-Assisted Treatment (MAT) App: User Feedback Survey

5