Feeback Survey Grant Recepient Instructions

Family-to-Family Health Information Centers

Feedback Survey Instructions

OMB No: 0906-XXXX

Expiration Date: XX/XX-XXX

Purpose: The U.S. Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau administers the Family-to-Family Health Information Center (F2F HIC) grant program. The goal of the program is to promote optimal health for children and youth with special health care needs (CYSHCN) by facilitating their access to an effective health delivery system and by meeting the health information and support needs of families of CYSHCN and the professionals who serve them. On [XX/XX/18] HRSA received Office of Management and Budget (OMB) approval for F2F HIC grant recipients to conduct feedback surveys with families and professionals served on a one-to-one basis and through training¹. The purpose of the surveys is to obtain data on how effective F2F HICs are in providing information, mentoring, and training to enable families of CYSHCN to make informed health care decisions.

Data from the surveys will support the HHS Secretary's priorities of engagement and performance and will provide mechanisms to capture consistent, performance data from F2F HIC grant recipients. The information collected will help HRSA demonstrate the reach of the F2F HIC program and respond to requests from Congress related to the F2F HIC program and program data. In addition, the data will also allow you to evaluate the effectiveness of your interventions and improve services you provide to families and the professionals who serve CYSHCN.

Reporting Requirement: HRSA requires all F2F HIC grant recipients to submit the feedback survey data annually to the HRSA-funded National Center for Family/Professional Partnerships (NCFPP). The NCFPP will clean, aggregate, and report the data to HRSA. You must submit your survey data to the NCFPP at the end of each project year. Failure to submit your survey data in the required timeframe may adversely affect your grant standing. HRSA, in collaboration with the NCFPP, will provide technical assistance to F2F HICs that request assistance with completing these instructions, including calculating number of surveys, reporting the data, etc.

Instructions: Each F2F HIC will contact randomly selected families and professionals who have received one-to-one service or training from their center and ask them to respond to one of three survey instruments. The three separate surveys (see Appendix A) are for (1) families who receive one-to-one service, (2) professionals who receive one-to-one service, and (3) individuals who attend trainings. Survey responses may be collected in-person or via telephone, e-mail, text message, social media, or other technology-based platform. The information provided in these instructions, will help determine the required number of survey responses you must report each year and the procedures you will use to collect and submit the data.

Each F2F HIC should carry out the steps below in order to collect and report the required survey data each project year:

1. Set up a data collection system. The majority of F2F HICs have a system in place to record and track services provided on a monthly basis. Data collection efforts for the F2F HIC survey should build upon this preexisting system. You are highly encouraged to use electronic systems, such as a Microsoft Access or some other contact database, which allow you to quickly determine all contacts made over time. If you currently use a paper log to record service calls from families each month, you will need to refer to this system when determining which families and professionals to contact for survey data collection.

You will also need a data system to track each person’s response. This can be as simple as an excel spreadsheet (see Appendix B for example). Responses to survey questions range from “strongly agree” to “strongly disagree,” and have a corresponding numeric value. HRSA may use this data for various federal reports, including Government Performance Results and Monitoring Act performance measure reporting. HRSA may report on overall feedback percentages for the F2F HICs or choose to report on scores related to specific survey questions. Any data reported outside of HRSA will be aggregated.

2. Submit to HRSA the number of families and professionals served one-to-one and trained. Each year, F2F HICs are required to submit an unduplicated count of families and professionals served/trained by their center. This information is reported in the HRSA Electronic Handbook as performance measures CSHCN-1 and F2F-1 (OMB Number: 0915-0298).

3. Obtain the required number of survey responses needed from HRSA. Each year, HRSA will determine and provide you the number of survey responses your F2F HIC must submit. HRSA has chosen to use a sample of families and professionals served/trained by all F2F HICs. This sample size is calculated using the total number of families and professionals served/trained for all F2F HICs in the previous reporting year (totals from data you provided in Step 2). The sample size is calculated with a 95% confidence level and 5% margin of error. Below is a sample calculation using 2016-2017 actual data.

Sample calculations based on 2016-2017 data:

Total sample Surveys

Total served by needed to achieve required for

Survey Type All F2F HICs (N) 95% confidence (n) each F2F HIC

Families (one-to-one) 100,484 383 6

Professionals (one-to-one) 35,553 381 6

Training Participants 133,540 383 6

Total Surveys Required for each F2F HIC = 18

NOTE: You will likely send more surveys than required because not every person who receives a survey will respond. For example, if you need to collect 18 total survey responses, your evaluator may need to contact 36 individuals in a year, if, on average, about half of those you contact respond. It would be a good idea to track how many responses you get so you can modify your approach for future efforts.

4. Decide on your process for carrying out the surveys. Complete this step after you know the number of survey responses needed. If you have an evaluation/feedback system in place, you can add these survey questions to it. At a minimum, you must administer 50% of the surveys to families receiving one-to-one services via telephone (versus other methods such as web-based or mail). F2F HICs are peer support projects, and it is good practice to speak with some families directly. Other surveys can be administered in the form of texts, emails, online surveys, or social media.

You are encouraged to ask families and professionals to respond to the survey within a month of their interaction with the F2F HIC to increase their recollection about the interaction. Monthly contacts may also reduce the burden of trying to gather all of your responses once a year.

5. Determine how you will randomize the families and professionals you contact. Each F2F HIC is required to randomize the individuals they ask to complete the surveys. This process will help minimize bias. You should not influence the survey findings by collecting feedback from regular, satisfied customers with whom you often interact. For example, if the F2F HIC has to contact 25 families and professionals during the year, one systematic approach could be to contact every fifth (5^th) person listed in the internal log a month after they received the service. Other options could be to rotate zip code or phone exchange information, or if you collect racial/ethnic background information, collect a certain number of responses from each group. The method should be consistent and you should use the same system every year to identify a random sample of individuals to be surveyed. Write the process down in case you are asked to provide information on the method you use.

Do not collect responses from the same family or professional more than once a year.

It is acceptable to administer same-day surveys to training participants if this is your standard practice. Some organizations send training surveys after the training event, which is also acceptable.

6. Identify an objective “evaluator” to carry out any telephone calls. The role your evaluator has at your F2F HIC depends on the type of survey (e.g., phone versus online) and the number of survey responses needed. Volunteers or student interns can administer surveys, if practical. Some F2F HICs have part-time evaluators. It is up to your organization to select the most appropriate approach.

Objectivity is essential; therefore, any staff serving as the evaluator shall not collect responses from people they have served one-to-one directly. Your assigned evaluator can be responsible for collecting all evaluation data and tallying up the responses; this person must be an organized record keeper. You are encouraged to recruit an evaluator who is familiar with evaluation and familiar with the work of your state’s F2F HIC. If you provide multilingual services, please keep this in mind when selecting your evaluator or survey method.

7. Provide your evaluator the necessary data and be clear on the system for contacting individuals. The evaluator should have access to your paper or electronic records to identify the sample of families and professionals to be contacted. Evaluators must use the random sampling system your F2F HIC developed above in Step #4.

8. Conduct the feedback surveys. Separate surveys have been developed for each type of respondent (i.e., family, professional, or training participant). The first question in each survey is considered a screener question. The survey instruments are provided in Appendix A of these instructions. Answers from the feedback surveys can be logged in a spreadsheet or form similar to the Sample F2F HIC Survey Data Log found in Appendix B.

You are encouraged to use the most cost-effective and practical means for contacting individuals. For training participants, it is appropriate to add survey data questions to an existing post-training evaluation form. When conducting follow-up phone calls, in particular, the evaluator should follow the script provided on the “F2F HIC Feedback Survey Script” (see Appendix C). The evaluator should read the question, and follow up each question with the response options provided (i.e., 1 if you strongly disagree, 2 if you disagree, etc.). If necessary, the evaluator can repeat the question in an attempt to provide clarification to the family or professional. The evaluator should then make note of the person’s response and proceed to the next question.

Do not force anyone to remember the purpose for contacting the F2F HIC. If people do not remember who the F2F HIC is or why they contacted the center, chances are, there is no information to base their opinions on. The screener question in the family and professional surveys include a response that states, “I do not remember contacting you.” Record the fact that the family/professional did not remember on the survey data log. If “do not remember” is chosen, your log should not include any additional responses for that family or professional.

9. Contact families and professionals served/trained, preferably within one month.

You are encouraged to contact families and professionals within one month of the service date. Expanding on the example from above: If an F2F HIC must submit 25 survey responses, this data collection can be spread over the course of a year in order to lessen the burden. This would work out to approximately two (2) survey responses per month (assuming a 100% response rate). In some months, a few less or a few more surveys may be collected; this is acceptable as long as data collection is spread over the year and the required number of surveys is attained. Once the annual required number is met, survey data collection for the year can be concluded.

HRSA understands the survey schedule may vary by F2F HIC. We recommend at least a one-month interval to increase the chances the family/professional remembers contacting your center and to encourage you to avoid completing a year’s worth of surveying at one time.

Of course, there will be individuals who attend both a training and come to an F2F HIC for one-to-one assistance. A screener question is included in the training survey in an effort to avoid duplication. To the extent possible, try not to duplicate those families and professionals who are also training participants. The evaluator should guide the family or professional to answer the questions thinking about one instance of assistance or training only. It is understood that responses will sometimes correspond more to a person’s experience with the F2F HIC overall. This is acceptable.

10. For Training Surveys: In an effort to minimize getting duplicate survey responses from families and professionals who receive one-on-one services as well as participate in training, there is a screener question at the beginning of the training survey. Within your final survey count, do not count responses from those who choose ‘Yes’, but go on to report the response in your survey data log that is submitted to the NCFPP.

11. Submit your survey response logs to the HRSA-funded NCFPP. After compiling all survey responses/data, prepare the data for submission to the NCFPP. Your data must be submitted by the designated due date, typically at the end of each project year.

In conclusion, general questions and answers regarding this process are available in Appendix D of this instruction. If you have additional questions about this instruction or your data collection reporting, do not hesitate to contact your assigned HRSA Project Officer.

Thank you, for your data collection efforts, and thank you for supporting families of CYSHCN and professionals across the nation.

APPENDIX D: Frequently Asked Questions

How will my F2F HIC reach all these individuals for survey data collection?

By keeping up-to-date contact information on individuals served and trained, you should be able to go to your files, whether they are paper or electronic. Gathering contact information from families and professionals served should be routine. As you well know, families often need to be contacted with more information, and you should have a method to reach them, which can also be used to conduct your surveys. When you collect contact information from families and professionals, it may be helpful to (1) ask if it is okay to contact them later with additional information or to gather feedback, and (2) ask if they have a contact preference.

With the exception of the 50% telephone interviews for the one-to-one family surveys, can the evaluator circulate the surveys electronically?

Yes. Many F2F HICs have the capacity to develop an on-line survey (i.e. Survey Monkey, text messaging, etc.) and the capacity to track responses. You are highly encouraged to tap into this expertise and use technology to gather feedback through electronic surveys, email, text messaging, and/or social media platforms.

How accurate will the survey data be?

If all grant recipients follow the protocol and methodology, they will be adhering to the sample size noted in Step # 3. The corresponding sample sizes reflect 95% certainty within a 5% margin of error. Sample sizes were determined using the calculators found at https://www.surveymonkey.com/mp/sample-size-calculator/ and http://www.raosoft.com/samplesize.html.

How often should I survey an individual?

Do not request a response from any one person more than once in a project year.

How does a Family-to-Family Center determine the number of people to get responses from over a year?

HRSA will determine the number of required survey responses, as described in Step #3 of these instructions.

Is the “evaluator” supposed to contact the required number of individuals provided by HRSA or more?

It is highly likely the “evaluator” may need to contact more individuals than the required number of survey responses needed. For example, if you need to collect 25 total survey responses, your evaluator may need to contact 50 individuals in a year, if, on average, about half of those you contact respond.

If time and resources permit multiple attempts to reach a family may be needed. It is recommended that a clear timeframe be set for call backs/follow-up, to avoid having a backlog. Three attempts over a two-week period is the standard for most F2F HICs. If a family has not responded after three attempts, count this as a non-respondent.

What if the evaluator at my F2F HIC calls a family in June (referencing phone services received in April), but the family member references services received six months earlier?

That is okay. It is true that many family members have multiple contacts with an F2F HIC and may not necessarily be able to put a timeframe on their requests for help. Let the family member answer the questions thinking about the impact of your center more generally. The point is you want feedback!

What is the benefit of including totals for each question, within the Survey Data Collection Log?

Comparing total scores or averages of scores for each question may help you identify areas where you can focus your improvement efforts.

Will technical assistance be available?

Yes, of course. F2F HICs vary in number of years in existence, size, scope, and technological sophistication. Technical assistance webinars will be offered through HRSA and the NCFPP. However, if you have specific questions, contact the NCFPP data team. Note: HRSA funds the NCFPP through a cooperative agreement to provide technical assistance and training for F2F HICs, among other activities.

What do I do with the feedback data I am collecting over the year?

Direct your evaluator to keep it up-to-date in a secure data collection system. Your HRSA Project Officer may ask for this information at any time. However, in most cases, feedback survey data should be submitted annually to the NCFPP. Having this data may also be useful when submitting your annual continuation (progress) report.

Does this feedback survey data collection replace other evaluation/quality improvement that our Center is already doing?

No. Other quality improvement activities currently conducted by your organization should continue. This requirement for feedback data collection and submission by no means replaces other efforts you may have to monitor and improve information and support provided to F2F HICs serving CYSHCN, their families, and health professionals.

When is the Survey Data Collection Log due? Survey data is submitted to the NCFPP each year at the time specified (typically after the May 31^st project period end date).

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0906-xxxx. Public reporting burden for this collection of information is estimated to average xx hours per response, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports Clearance Officer, 5600 Fishers Lane, Room 14N136B, Rockville, Maryland, 20857.

1 For the purposes of this survey, workshops and trainings are provided by the F2F HIC staff and include a course/topic description and learning objectives.