Annie Text Messaging Survey Question Changes

Request for Approval under the “Generic Clearance for Improving Customer Experience: OMB Circular A-11, Section 280 Implementation”

(OMB Control Number:2900-0876)

TITLE OF INFORMATION COLLECTION: Annie Text Messaging Survey Question Changes

PURPOSE OF COLLECTION:

What are you hoping to learn / improve? How do you plan to use what you learn? Are there artifacts (user personas, journey maps, digital roadmaps, summary of customer insights to inform service improvements, performance dashboards) the data from this collection will feed?

The VA Office of Connected Care intends to ensure the quality of customer service and satisfaction provided to Veterans aligns with the VA Secretary’s priority. The survey will provide information on the Veterans’ satisfaction with the Annie Application (App), VA’s short message program that promotes patient self-care and can be used by Veterans with any mobile phone that supports Short Message Service (SMS) text messaging. The survey will be instrumental to the Office of Connected Care in their efforts to improve applications and health subscriptions and to ensure patient overall satisfaction with their experiences using the Annie application.

In order to capture the voice of the Veterans using the Annie application, the Veteran Experience Office (VEO) will leverage VSignals to collect feedback through a short, low burden customer experience survey delivered as a URL for distribution on the Annie App. The survey is completed via a web-based survey design and contains questions to identify customer satisfaction and customer service areas that may need improvement. The survey will in no way collect nor share personally identifiable information. The participant can choose to exit the survey at any time before submitting their survey response.

This is an update to a previously OMB approved survey instrument to remove questions related to ease of use that have been deemed redundant to analysis and add two questions related to location—state and medical center—to better understand location-based differences in the service. The survey is ongoing and continuous.

TYPE OF ACTIVITY: (Check one)

[ X ] Customer Research (Interview, Focus Groups, Surveys)

[ ] Customer Feedback Survey

[ ] Usability Testing of Products or Services

ACTIVITY DETAILS

1. If this is a survey, will the results of this survey be reported to Touchpoints as part of quarterly reporting obligations specified in OMB Circular A-11 Section 280?

[ ] Yes

[X ] No

[ ] Not a survey

2. How will you collect the information? (Check all that apply)

[ X ] Web-based or other forms of Social Media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain

3. Who will you collect the information from?

Explain who will be interviewed and why the group is appropriate for the Federal program / service to connect with. Please provide a description of how you plan to identify your potential group of respondents and if only a sample will be solicited for feedback, how you will select them(e.g., anyone who provided an email address to a call center rep, a representative sample of Veterans who received outpatient services in May 2019, do you have a list of customers to reach out to (e.g., a CRM database that has the contact information, intercept interviews at a particular field office?)

The survey will be offered to Veterans who have subscribed and used the Annie text message application for one of the available self-care reminder health subscriptions.

4. How will you ask a respondent to provide this information?

(e.g., after an application is submitted online, the final screen will present the opportunity to provide feedback by presenting a link to a feedback form / an actual feedback form)

Participants to the Annie system will be provided with a text link: “We want to hear about your experience using Annie. Your answers to these questions directly help us improve our services.”

Participants will voluntarily choose whether they want to click on the link and whether they want to participate after opening the survey. The participant can choose to exit the survey at any time before submitting their survey response.

5. What will the activity look like?

Describe the information collection activity – e.g. what happens when a person agrees to participate? Will facilitators or interviewers be used? What’s the format of the interview/focus group? If a survey, describe the overall survey layout/length/other details? If User Testing, what actions will you observe / how will you have respondents interact with a product you need feedback on?

The survey will be offered to Veterans who have used the Annie application for one of the self-care reminder health subscriptions. There will be the following statement asking the readers to take the survey:

“We want to hear about your experience using Annie. Your answers to these questions directly help us improve our services.”

Participants will choose whether they want to click on the link and whether they want to participate after opening the survey. The participant can choose to exit the survey at any time before submitting their survey response.

The activity will be an electronic survey that takes approximately five minutes to complete.

6. Please provide your question list.

Paste here the questions or prompts presented to participants in your activity. If you have an interview / facilitator guide, that can be attached to the submission and referenced here.

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Is this the code that was provided to you? Hidden

• Yes

• No

[ Logic: Question used for variable and will be hidden from the respondent ]

How would you rate your level of comfort with technologies such as smartphones and computers?

• Novice: I have limited experience and require assistance to navigate technology.

• Between Novice and Intermediate: I have some experience but still need occasional assistance.

• Intermediate: I am comfortable with basic operations and can solve some problems on my own.

• Between Intermediate and Advanced: I can handle most tasks and troubleshooting but still seek help for very complex issues.

• Expert: I am very comfortable with technology and can handle almost all complex tasks and troubleshooting independently

To use Annie, you must first register with the Annie System. This includes providing your mobile phone number and time zone. How did you first register to use Annie? (select only one option) Required

• A VHA staff person registered me

• I self-registered

• I am unsure

After your registration, you received messages from Annie through this subscription. A VHA staff person can assign a subscription, or you can self-subscribe to one. How did you begin to use this Annie subscription? (select only one option) Required

• A VHA staff person assigned it to me

• I self-subscribed

• I am unsure

Did a VHA staff person speak with you about the subscription and what to expect before you started receiving messages from Annie? Required

• Yes

• No

[ Logic: Question will not appear if the respondent selects “I self-subscribed” in the previous question ]

What was your preferred means of communicating with Annie? Required

• Text messaging

• Annie App for Veterans (Annie web app)

• Using the Annie App for Veterans (Annie web app) and receiving generic texts notifying me of new messages in the Annie App

Receiving and replying to Annie messages was easy. Required

1 - Strongly Disagree

2 - Disagree

3 - Neither Agree nor Disagree

4 - Agree

5 - Strongly Agree

Annie’s messages helped me to manage my health better. Required

1 - Strongly Disagree

2 - Disagree

3 - Neither Agree nor Disagree

4 - Agree

5 - Strongly Agree

How would you rate the number of days the subscription lasted? Required

• Too Many

• Somewhat Too Many

• Just Right

• Somewhat Too Few

• Too Few

How would you describe the number of messages you received from Annie? Required

• Too Many

• Somewhat Too Many

• Just Right

• Somewhat Too Few

• Too Few

I would have contacted my VA care team more if I didn’t get messages from Annie. Required

1 - Strongly Disagree

2 - Disagree

3 - Neither Agree nor Disagree

4 - Agree

5 - Strongly Agree

I felt more connected to the VA because of Annie’s messages. Required

1 - Strongly Disagree

2 - Disagree

3 - Neither Agree nor Disagree

4 - Agree

5 - Strongly Agree

Overall, I am satisfied with the messages I received from Annie. Required

1 - Strongly Disagree

2 - Disagree

3 - Neither Agree nor Disagree

4 - Agree

5 - Strongly Agree

In which state do you receive medical care? (Dropdown of States)

In which medical center do you receive care?” (Dropdown of Medical Centers)

If there were any Annie messages that were not helpful to you, please describe them.

Type in your response

How can we improve your experience with Annie?

Type in your response

Can VA contact you about your experience to learn more about your feedback?

• Yes, VA can contact me about my experience.

• No, I do not want VA to contact me about my experience.

[ Logic: Following Questions below will not appear if the respondent selects “No, I do not want VA to contact me about my experience.” in the previous question ]

Please provide your name, email, and phone number so our staff can contact you about your experience:

Name____________

Email Address___________________

Phone Number_________________

7. When will the activity happen?

Describe the time frame or number of events that will occur (e.g., We will conduct focus groups on May 13,14,15, We plan to conduct customer intercept interviews over the course of the Summer at the field offices identified in response to #2 based on scheduling logistics concluding by Sept. 10^th, or “This survey will remain on our website in alignment with the timing of the overall clearance.”)

The survey will be a continuous, ongoing survey to collect feedback after the Veteran uses the Annie Application for one of their self-care reminder health subscriptions.

8. Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants?

[ ] Yes [ X ] No

If Yes, describe:

N/A

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden Hours
Individuals	25,000 Annual	5 minutes	2083 hours
Totals	25,000 Annual	5 minutes	2083 hours

CERTIFICATION:

I certify the following to be true:

1. The collections are voluntary;

2. The collections are low-burden for respondents (based on considerations of total burden hours or burden-hours per respondent) and are low-cost for both the respondents and the Federal Government;

3. The collections are non-controversial;

4. Any collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the near future;

5. Personally identifiable information (PII) is collected only to the extent necessary and is not retained;

6. Information gathered is intended to be used for general service improvement and program management purposes

7. Upon agreement between OMB and the agency aggregated data may be released as part of A-11, Section 280 requirements only on performance.gov. Summaries of customer research and user testing activities may be included in public-facing customer journey maps.

8. Additional release of data will be coordinated with OMB.

Name and email address of person who developed this survey/focus group/interview:

Name: __Brian Brown__________________

Email address: _[email protected]__________

All instruments used to collect information must include:

OMB Control No. 2900-0876

Expiration Date: 02/28/2026

HELP SHEET

(OMB Control Number: 2900-0876)

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g. Comment card for soliciting feedback on xxxx)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Agencies should only collect PII to the extent necessary, and they should only retain PII for the period of time that is necessary to achieve a specific objective.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households;(2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected per row.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g. fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours: Multiply the Number of responses and the participation time and divide by 60.

12