Text4baby Evaluation Questions

ATTAChment I

evaluation questions for the text4baby evaluation

This page has been left blank for double-sided copying.

Table I.1 List of text4baby Evaluation Questions and Evaluation Components

Evaluation Question	Safety Net Consumer Survey/Electronic Health Records	Consumer Focus Groups	Key Informant Interviews	Stakeholder Interviews
Individual Level
Reach: What are the characteristics of subscribers? How did subscribers hear about text4baby? What are the barriers to participation? Is the program reaching high-risk women?		
Engagement: In what ways do subscribers use text4baby? How easy is it to use text4baby? What challenges were encountered? How many subscribers STOP using text4baby and what are their reasons? To what extent do subscribers remain engaged throughout pregnancy and through the infant’s first year? What are the barriers to reenrolling after the baby is born?	
Education: To what extent are messages found to be useful and relevant? Are certain messages more useful than others? Are the messages culturally relevant? What changes in knowledge, attitudes, and behavior were made as a result of text4baby health tips? How do these changes vary by population subgroup?			
Connection: What services were accessed as a result of text4baby referrals or resources? How do utilization patterns (including referrals) differ between text4baby subscribers and non-subscribers? What factors may account for these differences? Do text4baby subscribers receive more recommended care than non-subscribers? Among subscribers, do patterns differ according to timing of enrollment or number of messages received?			
Sustainability: How willing would subscribers be to pay for text4baby text messages if the cost was not fully subsidized? Are subscribers satisfied with the level of messaging or would they like to receive more or fewer messages or additional messages on specific topics?		

Table I.1 (continued)

Evaluation Question	Safety Net Consumer Survey/Electronic Health Records	Consumer Focus Groups	Key Informant Interviews	Stakeholder Interviews
System Level
Reach: Who are the partners? What are their roles? Are the partners representative of the target audience? How do the partners interact with one another? What factors account for higher levels of penetration among certain subgroups or geographic areas?				
Engagement: How do partners facilitate outreach, enrollment, or other activities related to text4baby? What are the promising practices for engaging subscribers? What are the challenges?				
Education: How were messages developed and reviewed? How were messages tested for cultural or linguistic competence? How were messages tailored to the due date or date of birth? How did the content evolve over time (e.g., alerts)? Were any effects of the messages observed (e.g., increased calls to hotlines)?				
Connection: How does text4baby complement or substitute for other types of services? What is “value-added” about text4baby?				
Sustainability: What are the costs of text4baby in terms of the technology, outreach/enrollment, message development, and other activities? What resources are required to maintain the initiative among both public and private partners? What are the opportunities and challenges to gaining greater penetration of the initiative among the target population?				