Form CMS-10634 Information Interview

APPENDIX B
KEY INFORMANT INTERVIEWS

This page left blank for double-sided copying.

APPENDIX B: CMCS INTERVIEW

MATHEMATICA POLICY RESEARCH

CMCS: Formative stakeholder interview
I’d like to start by asking you about the coordination of the pilot mHealth program.
Text4baby coordination: State outreach activities
National and state initiatives using Text4baby
a. Beyond Voxiva; Healthy Mothers, Healthy Babies; and states, what other key stakeholders or partners
does CMCS engage with to implement the pilot mHealth program? (PROBE: Public? Private? National?
State? Local?)
•

What other CMCS or federal efforts do you consider when overseeing the implementation of the pilot
mHealth program? How do they influence the guidance and direction you give to the implementation
partners?

b. Please describe the types of activities that you engage in to facilitate coordination in the implementation
of the pilot mHealth program overall.
•
•
•
c.

Please describe the specific activities you engage in to provide support to Voxiva and Healthy
Mothers, Healthy Babies.
Please describe the specific activities you engage in to support pilot states.
Please describe the specific activities you engage in to provide support to any other partners.

What particular strategies have you observed that seem to promote coordination between
implementation partners and/or are helpful in supporting implementation?
•
•
•

What states in particular had an implementation that went particularly smoothly? What factors
contributed to this?
Did partners share a clear vision of the goals of the pilot mHealth program?
Did key players agree on how to shape the pilot mHealth program in each state?

d. What particular factors have you observed that seem to be barriers to coordination between
implementation partners and/or are least helpful to implementation?
•
•
•
•
•
•

What states in particular experienced more barriers to implementation? What factors contributed to
this?
Were there any technological barriers? What were they?
Were there difficulties with coordination or communication among key players, such as Healthy
Mothers, Healthy Babies; Voxiva; or the states?
Was there a lack of staff, budget, or resources? Which ones were barriers?
Were there holdups with decision making? What caused these delays?
Were there competing priorities among partners that made it difficult to implement the pilot mHealth
program? What were those competing priorities?

e. What strategies have you used to address these barriers, and how effective would you say that they
were?
Sustainability: Institutionalization and dissemination
f.

Thinking about CMCS’s roles and responsibilities in the pilot mHealth program, do you think that the
activities that CMCS engaged in could be transferred to the state or community level?
•

If yes, what agency or entity would be in the best position to take on this role?

PRA Disclosure Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-TBD
(Expires: TBD). The time required to complete this information collection is estimated to average 60 minutes per response,
including the time to review instructions, search existing data resources, gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form,
please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

B.3

APPENDIX B: CMCS INTERVIEW

MATHEMATICA POLICY RESEARCH

Use and potential of public-private partnerships for health
g. How important do you think the structure of a public-private partnership is in successfully implementing
the pilot mHealth program? What are the pros or benefits of such a partnership? What are the cons or
challenges?
Reach: State outreach activities
Now I would like to ask you specifically about outreach and promotion activities for the pilot mHealth program.
Outreach strategies and networks
a. From your perspective, what state outreach strategies seem to be most successful in reaching Medicaid
women? What has been the least successful? Why is that?
b. What else do you think could be done to increase the number of Medicaid women who sign up for
Text4baby?
Sustainability: Institutionalization and dissemination
The next questions are about sustaining and disseminating the pilot mHealth program.
Administrative, clinical, and technical support
a. Please tell me about the types of resources that CMCS has devoted to the pilot mHealth program.
(PROBE: Money? Staff? Time? Technology support?)
Costs/resources associated with promoting Text4baby
b. What resources do you believe are needed to implement the pilot mHealth program in the Medicaid
population on a national scale?
Perceived benefits of promoting Text4baby
c.

What are the benefits of CMCS’s involvement with the pilot mHealth program?
•
•

Are there other CMS initiatives that mHealth aligns with? If so, can you please describe these other
initiatives?
Does this effort overlap with any other CMS efforts? If so, can you please describe these other effort?

Factors for scale up of Text4baby
d. What factors do you think are needed to implement this pilot or a similar mHealth program in the
Medicaid population on a regional or national scale?
•
•
•

What types of infrastructure, stakeholder buy-in, and resources are necessary to expand a mHealth
program to a regional or national scale?
What types of advertising, marketing, or incentives do you believe are needed to promote Text4baby
or another mHealth program among Medicaid women on a regional or national scale?
What role do you think would be most beneficial for CMCS to play to support the uptake and
dissemination of a mHealth program?

Potential of mHealth technology in Medicaid population
e. What elements of this pilot mHealth program do you feel are replicable with other public health issues in
the Medicaid population?
f.

What elements of this pilot mHealth program need fine-tuning before they are applied to other public
health issues in the Medicaid population?

g. Do you think that these issues are specific to Text4baby, or to mHealth technology more generally?
•
•

What are the issues specific to Text4baby? What do you think can be done to address these issues?
What are the issues that apply to mHealth technology more generally? What do you think can be
done to address these issues?

h. Do you have any final thoughts or comments you’d like to share about Text4baby or the pilot mHealth
program in particular?

B.4

APPENDIX B: VOXIVA INTERVIEW

MATHEMATICA POLICY RESEARCH

Voxiva and Zero to Three: Formative stakeholder interview
Reach: State outreach activities
a. I’d like to start by asking you about the roles of Voxiva and Zero to Three in the implementation of
Text4baby as part of a pilot mobile health program supported by CMCS. Please tell me about your
partnership’s role in implementing Text4baby for this pilot.
Administrative, clinical, and technical support
•
•
•

What are the specific responsibilities of Voxiva and Zero to Three in implementing the pilot mHealth
program in the selected states?
Please discuss the specific resources states may request or “order” from Voxiva as a participant in
the pilot mHealth program.
Please describe the specific activities you engage in to support CMCS and states, including any
administrative and technical support.

Local partnerships to facilitate outreach and enrollment
b. Beyond CMCS and the state Medicaid agencies involved in the pilot, who are the other key
stakeholders or partners that you regularly engage to implement the pilot mHealth program?
•
•
•
c.

(PROBE FOR TYPES: Other state agencies, local partners, providers, outreach partners?)
What are the formal and informal mechanisms through which you and other partners collaborate?
Please describe the specific activities you engage in to support these partners.

What particular strategies have you observed that seem to promote coordination between
implementation partners (that is, CMCS, the pilot states, and any other key partners in the pilot mHealth
program)? Any others?

d. What particular factors seem to be barriers to coordination between implementation partners? Any
others?
•
•
•
•
•

Were there any technological barriers? What were they?
Were there difficulties with coordination or communication among key players? What were they?
Was there a lack of staff, budget, or resources? Which ones, and what types?
Were there holdups with decision making? Please share an example or two.
Were there competing priorities among partners that made it difficult to implement the pilot mHealth
program? Please describe what these were.

Sustainability: Institutionalization and dissemination
Use and potential of public-private partnerships for health
e. What strategies have you used to address these barriers? And how effective would you say the
strategies were?
f.

How important do you think the structure of a public-private partnership is in successfully implementing
the pilot mHealth program? What are the pros or benefits of such a partnership? What are the cons or
challenges?

PRA Disclosure Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-TBD
(Expires: TBD). The time required to complete this information collection is estimated to average 60 minutes per response,
including the time to review instructions, search existing data resources, gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form,
please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

B.5

APPENDIX B: VOXIVA INTERVIEW

MATHEMATICA POLICY RESEARCH

Reach: State outreach activities
Now, I would like ask you specifically about outreach and promotion activities for Text4baby and the pilot mHealth
program.
Outreach strategies and networks
a. Historically, what have been the main ways that your partnership has facilitated outreach and
subscription to Text4baby among the broader population of pregnant and postpartum women?
b. In CMCS’s pilot mHealth program, what strategies are you using for outreach to target women enrolled
in Medicaid?
•
•
c.

Are any of these strategies new ones? Which ones?
How do outreach efforts vary across the four pilot states?

Which outreach strategies are the most effective in increasing awareness of and subscription to
Text4baby (for the general population and for Medicaid women, specifically)?

Local partnerships to facilitate Text4baby outreach and enrollment
d. Which outreach strategies are least effective in increasing awareness of and subscription to Text4baby
(for the general population and for Medicaid women specifically)?
e. What is Voxiva or Zero to Three doing to help monitor or evaluate the effect of outreach efforts for
CMCS’s mHealth pilot?
f.

How much oversight do Voxiva and Zero to Three exercise over local outreach efforts among partners
in the pilot? In your opinion, should this oversight be increased? Decreased? How should it be
increased/decreased?

g. Other than CMCS’s mHealth pilot, what other organizations or agencies do you work with in the states
participating in the pilot?
•

Please describe the activities that you engage in with these organizations and the particular purposes
and area for collaboration.

Education: Text4baby content
Text4baby content for all subscribers
a. Generally speaking, how often are messages updated or added to the Text4baby protocols? What
circumstances motivate this?
b. How has the frequency of revisions and additions of messages been different for the pilot mHealth
program? Have the changes been more frequent or less frequent than they are for Text4baby in
general? What circumstances motivate this?
c.

In general, what is the process for developing, updating, and reviewing messages for Text4baby? Who
is involved in each step of this process? What kind of testing is done to ensure cultural and linguistic
appropriateness?

Text4baby content for Medicaid subscribers
d. How has the process for developing content for the pilot mHealth program been different? Have the key
players been different? In what way? Has the process to ensure cultural and linguistic appropriateness
been different? How so?
Evolution of messages
e. How has Text4baby content evolved over time? How do you know when it is time to update message
content?
•

Do you use evidence-based practices to guide updates?

•

Do you listen to requests from consumers?

•

Do you respond to feedback from providers and medical research?

•

Do you periodically review all of the messages and look for areas to revise or expand upon?

•

Any other methods?
B.6

APPENDIX B: VOXIVA INTERVIEW

f.

MATHEMATICA POLICY RESEARCH

What content areas, if any, are lacking or could be improved for all Text4baby users? What topics, if
any, do you think are not covered in enough detail that would be pertinent to all users?

g. What content areas, if any, are lacking or could be improved specifically for the Medicaid
population? What topics, if any, that are not currently covered in sufficient detail would be particularly
helpful for Medicaid beneficiaries (for example, more information about social services that are available
in subscribers’ communities)?
Sustainability: Institutionalization and dissemination
The next few questions are about the sustainability of Text4baby and applying the Text4baby approach to other
public health and health education issues within the Medicaid population.
Costs/resources associated with promoting Text4baby
a. What types of resources has your partnership devoted to the pilot mHealth program? (PROBE: Money?
Staff? Time? Technology support?)
•

Where has funding for these contributions come from?

•

Are these funding sources likely to run out? If so, when?

•

Do you currently have plans to seek new sources for funding? If so, from where? How likely is it that
Voxiva and Zero to Three will be able to obtain this additional funding?

Perceived benefits for promoting Text4baby
b. What are the benefits of the continued involvement of Voxiva and Zero to Three in Text4baby?
Specifically, what are the benefits of being involved in CMCS’s pilot mHealth program? (PROBES:
Additional outreach resources? More media coverage? Opportunity to increase subscription?)
Factors for scale-up of Text4baby
c.

What are the challenges to the continued involvement of Voxiva and Zero to Three in Text4baby? What
are the unique challenges related to involvement in the pilot mHealth program?

Potential of mHealth technology in Medicaid population
d. What factors do you think are needed to implement this pilot or a similar mHealth program in the
Medicaid population on a regional or national scale?
•

What types of infrastructure, stakeholder buy-in, and resources are necessary to expand a mHealth
program to a regional or national scale?

•

What types of advertising, marketing, or incentives do you believe are needed to promote Text4baby
or another mHealth program among Medicaid women on a regional or national scale?

e. What elements of the pilot mHealth program do you feel are replicable with other public health issues in
the Medicaid population?
f.

What elements of the pilot mHealth program still need fine-tuning before they are applied to other public
health issues in the Medicaid population?

g. Do you think that these issues are specific to Text4baby, or to mHealth technology more generally?
•

What issues are specific to Text4baby? What do you think can be done to address these issues?

•

What issues apply to mHealth technology more generally? What do you think can be done to address
these issues?

h. Do you have any final thoughts or comments you’d like to share about Text4baby or the pilot mHealth
program in particular?

B.7

This page left blank for double-sided copying.

APPENDIX B: STATE INTERVIEW

MATHEMATICA POLICY RESEARCH

States: Formative stakeholder interview
I’d like to start by asking you about [STATE’s] role in the coordination and implementation of the pilot mHealth
program.
Sustainability: Text4baby coordination
National and state initiative using Text4baby
a. What was the motivation behind applying for funding to participate in the pilot mHealth program? How
was the decision to participate in the pilot made?
b. How do the goals of the pilot mHealth program align with priorities in the state? How do the goals align
with priorities for your state’s Medicaid program?
c.

What other maternal and child health programs or initiatives are going on in your state right now? Are
any of these programs or initiatives coordinated with the pilot mHealth program? (PROBES: Strong
Start, Maternal and Infant Health Initiative, Healthy Start).

d. Please describe how [STATE] structured the implementation of the pilot mHealth program. (PROBES:
Who leads the effort? How does your state involve CMCS; Voxiva; and Healthy Mothers, Healthy
Babies?)
•
•
•
•
•

Please describe the specific activities you engage in with CMCS; Voxiva; and Healthy Mothers,
Healthy Babies.
What are the formal and informal mechanisms through which you collaborate with these partners?
Do you interact with these partners on a regular basis? How often?
Have you encountered any challenges in working with these partners? What were they?
What has worked well in these partnerships? What hasn’t worked so well? Would you do anything
differently to improve these partnerships?

e. Who are your other partners?
•
•
•
•
f.

Which partners do you work with most?
Which partners are more tangential to the coordination effort?
What challenges, if any, have you encountered in working with these partners?
What has worked well in these partnerships? What hasn’t worked so well? Would you do anything
differently to improve these partnerships?

What partners, if any, should be involved or be more involved in the coordination effort? Why do you
say that? What attempts were made, if any, to reach out to these partners? What was the result of these
attempts?

g. What types of activities do you engage in daily to internally coordinate the implementation of the pilot
mHealth program? (PROBES: Technical activities? Administrative activities?)
Reach: State outreach activities
Next, I’d like to ask you about your state’s outreach efforts for the pilot mHealth program in more detail. Please
include outreach efforts conducted by your Medicaid agency as well as other agencies in the state.
PRA Disclosure Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-TBD
(Expires: TBD). The time required to complete this information collection is estimated to average 60 minutes per response,
including the time to review instructions, search existing data resources, gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form,
please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

B.9

APPENDIX B: STATE INTERVIEW

MATHEMATICA POLICY RESEARCH

Outreach strategies and networks
First, let’s talk about the coordination and support your state provides to outreach partners to promote Text4baby
among Medicaid pregnant women.
a. Which organizations and agencies does your state engage with to conduct outreach to Medicaid women
to sign up for Text4baby?
b. Whom does your state interact with on a regular basis at these organizations or agencies? What are the
formal and informal ways in which your state and outreach partners interact and collaborate?
c.

What types of activities does your state engage in with outreach partners to facilitate subscription to
Text4baby among the Medicaid population?

d. What challenges has your state encountered with working with your outreach partners? Which
partnerships are the most challenging? Why is that?
e. What has worked well in your state’s partnerships that currently exist?
f.

Would your state do anything differently to improve the partnerships? Which partnerships? What would
your state do differently?

Local partnerships to facilitate Text4baby outreach and enrollment
Now I’d like to ask more specifically about the outreach being conducted by your state’s partners.
a. How effective do you think your state’s partners’ outreach efforts have been in getting women enrolled
in Medicaid to sign up for Text4baby?
b. Which strategies do you think are most effective? Why is that?
c.

Which are least effective? Why is that?

d. Which strategies do you think are missing? Are other outreach strategies needed to reach pregnant
women enrolled in Medicaid? What else can be done to increase Text4baby subscription in this
population?
e. Does your state use, or has your state used, community health workers to promote Text4baby to
pregnant Medicaid women?
•

[IF YES]: Have community health workers affected Text4baby subscription in your state? In what
way?

Use of community health workers
f.

What members of the Medicaid population who are pregnant, if any, are not reached as effectively as
they could be? (PROBE: Hispanic/Latina/Spanish-speaking; Native Americans; smokers).
•

[IF YES]: Why do you think they are not reached as effectively as they could be? What are the barriers
to reaching them? What methods or ways of reaching them could be tried?

Education: Text4baby content
Next I’d like to ask you about the ways in which your state customized the Text4baby messages for the mHealth
pilot.
Text4baby content for Medicaid subscribers
a. What messages, if any, did your state develop that covered topics that were not covered in the core set
of Text4baby messages?
•

What process did you use to develop state-specific messages or topics for messages?

•

Who developed the messages that apply to your state?

•

Did any outreach partners provide any input into the content of the messages?

•

What criteria are used to assess the cultural or linguistic appropriateness of the messages?

B.10

APPENDIX B: STATE INTERVIEW

MATHEMATICA POLICY RESEARCH

Evolution of messages
b. Have your state-specific messages evolved over time? If so, how? What has been added or updated?
c.

What content areas, if any, are lacking or could be improved, either in the core set of Text4baby
messages or in the messages that your state developed?

d. What activities has your state conducted, if any, to monitor or evaluate the progress of the pilot mHealth
program? What have you observed?
Sustainability: Institutionalization and dissemination
Costs/resources associated with promoting Text4baby
The next few questions relate to sustaining the pilot mHealth program.
a. What types of resources has your state devoted to the pilot mHealth program? (PROBES: Money?
Staff? Time? Technology support?)
b. Where has funding for these contributions come from?
•

When are these funding sources likely to run out, if ever?

•

What plans, if any, do you currently have to seek new sources of funding? If so, how likely is it that
your state will be able to obtain this additional funding?

•

What other private organizations might be interested in funding this mHealth program? Who do you
think might be interested?

•

Thinking about your current sources of funding and any potential new sources of funding, do you think
this mHealth program is sustainable in your state over the long term? Why do you say that?

Perceived benefits for promoting Text4baby
c.

What are the benefits of your state’s continued involvement in this pilot mHealth program?

d. What are the challenges to your state’s continued involvement in the pilot mHealth program?
Factors for scale-up of Text4baby
e. What factors do you think are needed to implement this pilot or a similar mHealth program in the
Medicaid population on a regional or national scale?

f.

•

What types of infrastructure, stakeholder buy-in, and resources are necessary to expand a mHealth
program to a regional or national scale?

•

What types of advertising, marketing, or incentives do you believe are needed to promote Text4baby
or another mHealth program among Medicaid women on a regional or national scale?

What elements of the pilot mHealth program do you feel are replicable with other public health issues in
the Medicaid population?

Potential of mHealth technology in Medicaid population
g. What elements of the pilot mHealth program need fine-tuning before they are applied to other public
health issues in the Medicaid population?
•

What issues, if any, are specific to Text4baby or mHealth technology more generally?

•

What do you think can be done to address these issues? (specific to Text4baby; specific to mHealth
technology more generally)

h. Do you have any final thoughts or comments you’d like to share about Text4baby or the pilot mHealth
program in particular?

B.11

This page left blank for double-sided copying.

APPENDIX B: OUTREACH PARTNERS INTERVIEW

MATHEMATICA POLICY RESEARCH

Outreach partners: Formative stakeholder interview
I’d like to start by asking you about your organization’s participation in the pilot mHealth program to increase
Text4baby subscription among Medicaid women in your state.
Reach: State outreach activities
Local partnerships to facilitate Text4baby outreach and participation
a. When did your organization become involved with the pilot?
b. How did you organization make the decision to participate in the pilot?
c.

What is your organization’s role in the pilot?

d. Do you promote Text4baby through other projects? If so, what are they?
•

[IF YES]: Do any of these other projects overlap with the pilot mHealth program in any way? If so,
how? (for example, target population)

Administrative, clinical, and technical support for Text4baby
e. Beyond the state Medicaid program, what other partners do you work with to increase Text4baby
subscription among Medicaid women in your state? (PROBE FOR TYPES: Providers? Managed care
organizations? Community health workers?)
•
•
•
•
•
•
•
•
f.

What are the formal and informal mechanisms through which you collaborate with these partners?
What support, information, or resources, if any, does your organization provide to any of these
partners in the effort to increase Text4baby subscription among Medicaid women?
What support, information, or resources, if any, does your organization receive from any of these
partners?
How often do you interact with these partners? (PROBE: Is it frequently, occasionally, rarely, or
never? Is it on a regular basis or only as needed?)
Which partners do you work with most? Why is that?
Which partners are more tangential to the coordination effort? Why is that?
What challenges have you encountered in working with these partners?
What has worked well in these partnerships? Would you do anything differently to improve these
partnerships? What would that be?

Which partners should be involved or more involved in the coordination or outreach effort? Why is that?
What attempts, if any, were made to reach out to these partners? What was the result of these
attempts?

Outreach strategies and networks
Now I’d like to ask about the outreach activities to Medicaid women to promote Text4baby.
a. What are the main ways in which your organization conducts outreach to Medicaid women to promote
subscription to Text4baby?
b. How effective do you think your organization’s outreach efforts have been in getting women enrolled in
Medicaid to sign up for Text4baby?
•

Which strategies do you think are the most effective? Why?

•

Which are the least effective? Why?

PRA Disclosure Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-TBD
(Expires: TBD). The time required to complete this information collection is estimated to average 60 minutes per response,
including the time to review instructions, search existing data resources, gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form,
please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

B.13

APPENDIX B: OUTREACH PARTNERS INTERVIEW

c.

MATHEMATICA POLICY RESEARCH

Which strategies do you think are missing? What other outreach strategies, if any, are needed to reach
pregnant women enrolled in Medicaid? What else can be done to increase Text4baby subscription in
this population?

Use of community health workers
d. Has your organization used community health workers to promote Text4baby to pregnant Medicaid
women?
•

[IF YES]: Do you think that community health workers have increased Text4baby subscription? Why
do you say that?

e. Are there members of the Medicaid population who are pregnant that are not reached as effectively as
they could be? (PROBE: Hispanic/Latina/Spanish-speaking; Native American; smokers).
•

[IF YES]: Why do you think they are not reached as effectively as they could be?

•

[IF YES]: What are the barriers to reaching them? Any others?

Sustainability: Institutionalization and dissemination
The next few questions I will ask are about sustaining the pilot mHealth program.
Costs/resources associated with promoting Text4baby
a. Please tell me about the types of resources that your organization has devoted to the pilot mHealth
program. (PROBES: Money? Staff? Time? Technology support?)
•

Where has funding for these contributions come from?

•

Are these funding sources likely to run out? If so, when?

•

What plan, if any, do you currently have to seek new sources of funding? If so, how likely is it that
your organization will be able to obtain this additional funding?

•

What other organizations might be interested in funding this mHealth program? Who else might be
interested?

Perceived benefits for promoting Text4baby
b. What are the benefits of your organization’s continued involvement in this pilot mHealth program?
c.

What are the challenges to your organization’s continued involvement in the pilot mHealth program?

Factors for scale up of Text4baby
d. What factors do you think are needed to implement this pilot mHealth program in the Medicaid
population on a regional or national scale?
•

What types of infrastructure, stakeholder buy-in, and resources are necessary to expand this pilot
program to a regional or national scale?

•

What types of advertising, marketing, or incentives do you believe are needed to promote Text4baby
among Medicaid women on a regional or national scale?

e. What elements of the pilot mHealth program do you feel are replicable with other public health issues in
the Medicaid population?
f.

What elements of the pilot mHealth program need fine-tuning before they are applied to other public
health issues in the Medicaid population?

Potential of mHealth technology in Medicaid population
g. Do you think that these issues are specific to Text4baby, or to mHealth technology more generally?
•

What issues are specific to Text4baby? What do you think can be done to address these issues?

•

What issues apply to mHealth technology more generally? What do you think can be done to address
these issues?

h. Do you have any final thoughts or comments you’d like to share about Text4baby of the pilot mHealth
program in particular?

B.14

APPENDIX B: PROVIDERS INTERVIEW

MATHEMATICA POLICY RESEARCH

Providers: Formative stakeholder interview
First, I’d like to find out how you learned about Text4baby.
Reach: Awareness of Text4baby
Provider’s awareness and promotion of Text4baby
a. How did you first find out about Text4baby? (PROBES: Did you attend an outreach event? See a media
campaign? Pick up promotional materials? Learn about it through your clinic? From a colleague? From
a patient? Through some other source?)
b. After you first heard out about Text4baby, did you try to learn more about it?
•

c.

[IF YES]: What did you do to learn more about it? (PROBES: Did you visit the Text4baby website or
Facebook page? Did you visit another website that has information about Text4baby? Did you ask a
colleague about it? Did you attend an outreach event? Did you learn more about it in some other
way?)

Are you familiar with the pilot mobile health program your state is participating in that is working toward
increasing Text4baby subscription among women enrolled in Medicaid?
•

[IF YES]: Does your clinic have any specific initiatives related to the pilot? What are they?

Reach: State outreach activities
The next questions are about how your clinic promotes Text4baby. Please focus on general promotion efforts
such as outreach events, media campaigns, or distribution of promotional materials. I will ask you more specifically
about how you promote Text4baby with your own patients on an individual level after.
Outreach strategies and networks
a. Does your clinic promote Text4baby to the pregnant and/or postpartum women that you serve, or to
pregnant and/or postpartum women in your local community?
•

[IF YES]: How does your clinic promote Text4baby? Does your clinic have any specific initiatives
related to Text4baby? What role do you personally play in this effort/these efforts?

•

[IF YES]: How effective do you think your clinic’s efforts have been in getting the pregnant and/or
postpartum women in your community to sign up for Text4baby?
- What has worked well?
- What hasn’t worked so well?
- What are the specific challenges, if any, that your clinic faces in getting women to sign up for
Text4baby?
 [IF YES]: Are these challenges unique to your local community, or are these more general
challenges in getting women to sign up for Text4baby?
- What if anything, would you change about your clinic’s approach to promoting Text4baby?

PRA Disclosure Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-TBD
(Expires: TBD). The time required to complete this information collection is estimated to average 60 minutes per response,
including the time to review instructions, search existing data resources, gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form,
please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

B.15

APPENDIX B: PROVIDERS INTERVIEW

MATHEMATICA POLICY RESEARCH

Administrative, clinical and technical support for Text4baby
•

Is there a dedicated staff person at your clinic who helps patients sign up for Text4baby or helps
Text4baby subscribers troubleshoot issues?

•

What difficulties, if any, has your clinic experienced integrating the promotion of Text4baby into the
work that you do? (PROBES: Time constraints? Financial constraints? Staffing constraints?
Resistance from other staff or the patients that you serve?)

Use of community health workers
b. Has your clinic used community health workers to promote Text4baby in your local area?
•

[IF YES]: What do the community health workers do to promote Text4baby?

•

[IF YES]: Have the community health workers affected Text4baby subscription in your local area? In
what ways? (or Why not)?

Sustainability: Text4baby coordination
Now I’d like to ask about the individual-level communication you have about Text4baby with the patients you serve
in your clinic.
Integration into provider’s work flows
a. Does your clinic ask providers to talk about Text4baby with their pregnant and/or postpartum patients?
•

[IF YES]: What does your clinic encourage you and other providers to say or do? What resources
does your clinic provide you with so that you can talk about Text4baby with your patients?

b. Do you talk about Text4baby with your own pregnant and/or postpartum patients?
•

c.

[IF YES]:
- How do you introduce or bring up Text4baby with your patients? PROBES: How do you start the
conversation? What do you tell them about it?
- When do you bring up Text4baby with your patients? PROBES: Is it during an initial visit with a
patient, or during a subsequent visit? Is it at the beginning of a visit, in the middle, or at the end
of the visit? Does the timing vary depending on whether the client is pregnant or postpartum?
- Do you recommend Text4baby to you patients?
- Do you recommend Text4baby to all patients, or just certain types of patients? (PROBES:
Pregnant women only? Postpartum women only? Low-income women only?)
 [IF CERTAIN TYPES OF PATIENTS]: Why do you recommend Text4baby to [FILL TYPE
OF CLIENT MENTIONED ABOVE] specifically?
- Do you ask your patients in a subsequent visit if they’ve signed up for Text4baby? If a client hasn’t
signed up for Text4baby, do you follow up with her on a subsequent visit to see if she signed up?
 [IF YES]: If a client hasn’t signed up for Text4baby, do you ask her why she hasn’t signed
up?
 [IF YES]: What are some of the reasons your patients give for not signing up?
- Do you record information about Text4baby participation in the client’s medical chart?

Have any of your patients signed upfor Text4baby?
•

IF YES:
- Do any of your patients mention the messages that they receive from Text4baby to you?
- Are there topics or messages that your patients bring up most often?
- Why do patients bring up messages that they receive from Text4baby with you? PROBES: Are
they looking for more information about a topic, or did they just think the information they got was
interesting?
- Do you ever ask your patients about the messages that they receive from Text4baby?
- Have any of your patients told you that they texted STOP to stop receiving Text4baby messages?
 [IF YES]: Did they tell you why they wanted to stop receiving messages? What reasons
did they give?

B.16

APPENDIX B: PROVIDERS INTERVIEW

-

MATHEMATICA POLICY RESEARCH

IF NO OR DON’T KNOW:
 What are some of the reasons why you haven’t initiated a conversation about Text4baby
with your pregnant/and or postpartum patients?
 Have any of your patients ever asked you about Text4baby? What did you tell her about
it? Did you refer her to another place to get more information?
 [IF NO]: Why not? Did you recommend that she sign up?
• [IF NO]: Why not?

Sustainability: Institutionalization and dissemination
Next I’d like to ask you some questions about the resources your clinic has devoted to promoting Text4baby.
Costs/resources associated with promoting Text4baby
a. Does your clinic have sufficient resources for promoting Text4baby?
•

[IF NO]: Which resources are you most in need of? (PROBES: Money? Staff? Time?) If you were able
to get these additional resources, what would you use them for?

Factors for scale-up of Text4baby
b. What are the benefits to your clinic in continuing to promote Text4baby in your local area and/or
individually to the patients that you serve? What are the challenges that you currently face, or that you
might face, in the future?
Perceived benefits of promoting Text4baby
Potential of mHealth in Medicaid population
c.

Based on your experience, do you think text messaging is an effective way to provide health information
to the population that your clinic serves? Why (or why not)? What specific subsets of the population that
your clinic serves would health information provided through text messaging be particularly effective in
reaching and affecting? Why do you say that?
•
•

What subsets of the population would text messaging of health information be particularly ineffective
in reaching and affecting? Why do you say that?
What health information topics to you think the population that your clinic serves would be most
interested in receiving information about?

d. Do you have any final thoughts or comments you’d like to share about Text4baby [or the pilot mHealth
program (if aware of it)] or its use with the population you serve?

B.17

This page left blank for double-sided copying.

APPENDIX B: RECRUITMENT EMAIL

MATHEMATICA POLICY RESEARCH

Recruitment Email to Outreach Partners and Health Care Providers

Subject: Text4baby – Evaluation of a pilot mHealth program
Dear [NAME]:
My name is [NAME] and I am a [POSITION] at Mathematica Policy Research. I am contacting
you because [CONTACT PERSON AT STATE LEVEL] at [ORG] gave me your name as
someone that could help us with an important study for The Centers for Medicare & Medicaid
Services’ Center for Medicaid and CHIP Services (CMCS). I would like to invite you to
participate in an interview as part of the study.
CMCS has contracted with Mathematica Policy Research to conduct an evaluation of a pilot
mobile health program (Text4baby) in the Medicaid population. As part of the evaluation, the
evaluation team is conducting interviews with health care providers and partners who are active
in outreach for the pilot and/or have unique knowledge about the health needs of the Medicaid
population. The purpose of the interview is to gain your insight into the aspects of the mobile
health pilot that are working well, as well as the aspects that could be improved. During the
interview we will ask you how well you think the pilot is reaching Medicaid women, any
observations you have about how Text4baby is impacting the health knowledge and behaviors of
women who receive messages, and the factors affecting continued use and expansion of mobile
health technology in the Medicaid population in general.
[PROVIDERS: The evaluation team will be in your area [DATES].] The interview will be
conducted [PROVIDERS: in-person at your organization][PARTNERS: over the phone] and will
take approximately [PROVIDERS: 30 minutes][PARTNERS: 60 minutes]. If you are interested
and available to participate, please contact me by [DATE] to schedule an interview. I will send
you a brief synopsis of the discussion topics by email one week prior to your interview.
If you have any questions or would like to schedule an interview, please contact me by email at
[email protected] or by phone at xxx-xxx-xxxx. Thank you in advance for your
assistance with this important study!
Sincerely,

B.19

This page left blank for double-sided copying.