FEMA Manual FM-104-FY-22-101

FEMA Manual FM-104-FY-22-101 Qualitative Research Protocol & Moderators Guide Ex.docx

Federal Emergency Management Agency Individual Assistance Customer Satisfaction Surveys

FEMA Manual FM-104-FY-22-101

OMB: 1660-0143

Document [docx]
Download: docx | pdf

CSA QUALITATIVE PROTOCOL

Individual Assistance (IA) Surveys


Disaster survivors’ needs and expectations evolve over time; therefore, the importance of conducting periodic qualitative research is essential to accurately measuring customer satisfaction. Focus groups and one-on-one interviews assists in many areas such as the re-design of the survey questionnaires, provide FEMA with an understanding of how to improve information and correspondence with registrants for disaster assistance, how to improve programs and processes for disaster survivors, as well as how to improve the in-person, on-the-phone and on-line experiences.  


This protocol was developed to provide consistent guidelines within the Customer Survey and Analysis (CSA) Section of FEMA who plan to conduct qualitative studies for issues related to the Individual Assistance (IA) surveys which include: Initial, Contact, and Assessment.


There are three phases to conducting a focus group and these phases are explained below:


PHASE 1: BEFORE THE QUALITATIVE STUDY


  1. Meet with management/leadership to identify overall purpose and outcomes for the focus group/interview.

    • Discuss possible hypotheses.

    • Identify funding sources.


  1. Determine target population and number of responses needed.

  • Multiple focus groups will allow for comparisons and to examine themes that emerge from each discussion.

  • Past results show focus groups average 6% response rate from the initial call to participate. For each of the 4 programs, there is an annual allotment of 960 unique participants. This includes 12 participants per 5 days of separate sessions per 4 geographical location.

  • Focus groups are approximately 2 hours each (plus 1 hour travel).


  1. Identify participants:

  • Determine how many participants are needed in each focus group. Each focus group should have between six and twelve participants. Fewer than six participants may limit the conversation and yield poor data while more than twelve can be unmanageable. We allow for maximum of 12 per session with the anticipation that some may not attend.

  • Develop a list of key attributes to seek in participants based on identified purpose.

  • Secure names and contact information and send invitations.


  1. Generate questions.

  • Based on the purpose and goals of the focus group, identify no more than 5-6 questions.

  • Revisit the questions to make sure they will promote appropriate discussion to address research questions.

  • Order the questions from general to specific.


  • Scope of questions from each stage in the registration to determination process may include topics related to the following survey:


  • Initial

  • Recent changes to registration process.

  • Technology/ Usability with the online registration tool.

  • Application process and resources.

  • Disaster recovery center (DRC) changes/usability/expectations.


  • Contact

  • Recent changes related to correspondences and information.

  • Inspection process: Transparency and understanding.

  • Expectations of inspections.

  • Technology/ Usability with the online/automated process.


  • Assessment

  • Repair quality and cost.

  • Communication and usability.

  • Assistance expectations.

  • Recovery process.



6. Select a facilitator and note-taker

  • The facilitator should not be someone who directly oversees the issue being researched. This may make participants less open to sharing their thoughts or concerns.

  • The facilitator should be able to keep the discussion going, deal tactfully with difficult or outspoken group members, and make sure all participants are heard.

  • The facilitator should ask the questions and probes, but not participate in the dialogue or correct participants.

  • If possible, a program analyst can sit in for specific program questions.


7. Choose the location

  • Choose a location which is comfortable, easily accessible, and where participants can see one another.

  • Choose a setting which does not bias the information gathered. Preferably libraries or other public forums with meeting rooms.


8. Develop script: Moderator’s Guide (See Appendix A)

  • Part one: Welcome participants and make introductions. Provide logistics of building related to emergency exits and restrooms and sign consent forms.

  • Part two: More about FEMA CSA and reasons for the focus groups. Explain that information is confidential and no names will be used. Provide FEMA Helpline number.

  • Part three: Ask for introductions from the group.

  • Part four: Ask prepared questions; remember to use probes and follow up questions to explore the key concepts more deeply.

  • Part five: Close the focus group – thank participants, give them contact information for further follow up if requested, and explain how data will be analyzed and shared.


PHASE 2: CONDUCT THE QUALITATIVE STUDY


1. Bring materials:

  • Notebook/computer

  • Flip chart paper/ no board is available (if available)

  • Focus group list of participants

  • Focus group script

  • Name tags

  • Location signage

  • Watch or clock

  • Consent forms


2. Arrive before the participants to set up room, display signs directing participants to room, etc.


3. Introduce facilitator and the note-taker (if applicable) and carry on the focus group

according to the script (See Appendix A).


4. Conduct the session, being mindful of the following:

  • Set a positive tone while empathizing with their situation as a recent disaster survivor.

  • Make sure everyone is heard; draw out quieter group members.

  • Probe for more complete answers.

  • Monitor questions and the time closely – it is the job of the facilitator job to make sure the group is on track.

  • Don’t argue a point with a participant, even if they are wrong. Address it later if needed.

  • Thank participants and let them know FEMA values their input considering the disruption in their lives when dealing with a disaster.

  • (See Appendix A)



PHASE 3: INTERPRETING AND REPORTING THE RESULTS


1. Summarize each meeting

  • Immediately after the meeting, the facilitator should write up a quick summary of his/her impressions.

  • Transcribe the notes of the focus group. This should be done as soon as possible after the focus group has been conducted.

  • If multiple facilitators have been used for two or more groups, discuss impressions with the other facilitators before reviewing the transcript.


2. Analyze the summaries

  • Read the notes and look for themes/trends. Write down any themes which occur more than once.

  • Context and tone are just as important as words. If comments are phrased negatively or triggered an emotional response, this should be noted in the analysis.

  • Interpret the results

    • What are the major findings?

    • What would be possible recommendations?


3. Write the report

  • The report should include the research purpose, outcomes, process, findings (major themes and minor themes), and recommendations.

  • Submit the report to management.


4. Make adjustments/take action on what was learned (After-action review)

  • Schedule a meeting with management and/or study facilitators to discuss the implications.

    • Highlight the main themes, and/or issues that arose in the study.

      • These themes/issues can be related to logistics of focus group/interviews or program discussion.

    • Prioritize the results and make action plans for the most important priorities.

































Appendix A

Moderator’s Guide Example


IHP LETTERS REVISION

FOCUS GROUP

MODERATOR’S GUIDE

Applicant Feedback for the Revision of the IHP Letters





CUSTOMER SURVEY and ANALYSIS (CSA) SECTION

Example

































IHP LETTERS REVISION FOCUS GROUP

MODERATOR’S GUIDE







4
min







About Us



Reason





Note-taking







Personal Opinion





Ease and Assurance



No case or program info

FEMA #

  • PURPOSE / SESSION DETAILS

Session Details:

      • About us

      • Reason we Invited Participants

      • Note Taking

      • Personal Opinions/Ease/Assurance

      • Ease and Assurance

      • No Case Info

      • FEMA Helpline number

Shape1

Now a little bit about us. We work in various roles in the Recovery Division of FEMA, providing disaster assistance to individuals like you and your families who have been in a disaster. Blair is the Section Chief of the Program Management Section. Anna is an analyst who helps make improvements to the services FEMA provides. And Sheri and I are analysts who collect disaster survivors’ opinions about the services FEMA provides.



The reason we invited you here today is to ask you to review the information letters FEMA has redesigned and then ask you to answer a few questions about the new design. Specifically, we want you to tell us if they are easy to read and easy to understand.



Sheri will be typing your statements as you speak so that we have an accurate record of your responses. With that in mind, we ask that you speak clearly and one at a time so she doesn’t miss anything, and so everyone has an opportunity to speak.



There are no right or wrong answers. This discussion is meant to gather your personal opinions which will help us determine any areas in the letters that may need more attention.



We want you to feel completely at ease to share your true thoughts, so please be assured that your responses today will not in any way affect the outcome of your FEMA case.



I also want to mention that we do not have any information about the decisions of your case. Our purpose here today is simply to gather your opinions about the disaster assistance information that is provided in the new letters.



If you have questions about your case, we encourage you to call the FEMA Helpline at 1-800-621- FEMA (3362).



























































3-6
min

(30 seconds per participant)









3
min













3 Letters

Original Packet





New Intro Letter









Super Letter



  • PARTICIPANT INTRODUCTIONS

Shape2

Okay, now I’d like each of you to introduce yourself to the group.

























  • LETTER DETAILS



      • 3 Letters

      • New 7 page Introduction Letter

      • Shape3

        We will be looking at 3 letters: a new introduction letter, and 2 new approval status letters.





        The first documents you received after you applied with FEMA were in a packet containing a 2 page introduction letter, a 9 page list of referrals to other agencies, and a 33 page booklet titled the Applicant’s Guide to the Individuals & Households Program. [hold up copies of each to show]



        The information in these 44 pages has been shortened to one 5 page introduction letter and one 2 page referral document. [hold up copies of each to show] I will ask you to review the 5 page letter and then ask you to answer some questions about it.


        Next is the approval letters. After you applied you received a letter explaining what assistance you were approved to receive and/or not approved to receive, and how to appeal if you disagreed with the decisions. You also may have received additional letters explaining the results of any appeals you may have requested. We have redesigned these letters and I will ask you to review them and then again ask you to answer some questions.







        New Super Letter and A-Super Letter






10
min











Reading Preferences


  • QUESTIONS - Top of Mind Opening Question


Before we begin with the letter reviews, I will start with a general question. For this first question I will go around the room to give each of you an opportunity to share your opinions one at a time.


1. Which is more important to you when reading information about any type of service being provided to you: do you prefer brief reading materials that you can read quickly in one sitting, or detailed materials that might take you longer to read but fully answer all of your questions?



________, let’s start with you








10
min





Read in Sections





Wanted to Know







Page 1









Two Parts





Easy to Read and Clearly Explain


QUESTIONS – Introduction Letter



We are ready to review the new introduction to FEMA letter now. You will read the letter in sections and I will ask a question after each section. The letter is printed front and back which is the way it will be printed when it is mailed out.


While reading, I would like you to think about what you most wanted to know about disaster assistance after you registered with FEMA.



Read all of page 1, then flip it over and read until you get to the purple flag on page 2.

[allow 5 minutes to read]




2. Tell us if the descriptions of the categories in the Individuals and Households Program were easy to read, and if they clearly explained how FEMA’s disaster assistance works.





Who would like to go first?












10
min











Insurance Inspection SBA Appeal




Thank you everyone. Now you can read the rest of page 2 and continue to page 3, stopping at the green flag.

[allow 5 minutes to read]




3. After reading the details in this section, what is your understanding of what happens after applying for disaster assistance?





Who would like to go first?







10
min





Flood Insurance




Next read the section at the bottom of page 3 under the green flag and stop at the end of the page.

[allow 3 minutes to read]


4. What do you understand about Flood Insurance after reading this section?



Who would like to start?





10
min





Mail

Record Keep

Fraud Discrimination




Now read all of page 4 and stop at the end of the page.

[allow 5 minutes to read]


5. What do you think about the way the topics on this page are explained?





Who would like to start?







10
min





Resources




Now read all of page 5 and stop at the end of the page.

[allow 5 minutes to read]




6. What do you feel about the level of detail on this page?



Who would like to start?










  • Closing



That was the last question! You have all provided excellent feedback. We thank you for your participation and assure you that the feedback you provided today will be used to improve service to disaster assistance victims across the country.

[collect hard copies]












10


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorGuillory, Jessica
File Modified0000-00-00
File Created2023-08-27

© 2024 OMB.report | Privacy Policy