1660-0036 Customer Satisfaction Supporting Statement 2-14-11 PART B

2-9-2011

Supporting Statement for

Paperwork Reduction Act Submissions

OMB Control Number: 1660 - 0036

Title: Federal Emergency Management Agency (FEMA) Individual Assistance Customer Satisfaction Surveys

Form Number(s): Caller Services Registration Survey, FEMA Form 007-0-3 (Formerly titled Registration Intake Survey - - Phone Survey FEMA Form 90-147);

Caller Services Helpline Survey, FEMA Form 007-0-5 (Formerly titled Helpline Survey - - Phone Survey, FEMA Form 90-148); Internet Registration Survey, FEMA Form 007-0-2 (Formerly titled Internet On-Line Registration Phone Survey, FEMA Form 90-150); Internet Inquiry Survey, FEMA Form 007-0-19, (Formerly titled Internet Applicant Inquiry/Update Phone Survey, FEMA Form 90-151); Casework Representative Survey, FEMA Form 007-0-6; Direct Housing Operations Survey, FEMA Form 007-0-4; Disability Access and Functional Needs Representative Survey, FEMA Form 007-0-8; Disaster Recovery Center Survey, FEMA Form 007-0-7; Communication and Process Survey, FEMA Form 007-0-9, Contact Survey, FEMA Form 007-0-10, Correspondence and Process Survey, FEMA Form 007-0-11, E-Communications Satisfaction Survey FEMA Form 007-0-12, Evacuations Plan Satisfaction Survey, FEMA Form 007-0-13, Rapid Temporary Repair Survey FEMA Form 007-0-15, and Site Recertification Survey, FEMA Form 007-0-18.

B. Collections of Information Employing Statistical Methods.



When Item 17 on the Form OMB 83-I is checked “Yes”, the following documentation should be included in the Supporting Statement to the extent it applies to the methods proposed:

1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

All surveys proposed here are Time-Limited.

The target populations of this information collection are individuals who are disaster survivors living in the US territory and registering for federal assistance for each declared disaster (catastrophic disasters may be excluded during weeks when registration activity exceeds 75,000). The sampling frames consist of the names of all the disaster survivors who contact FEMA for disaster assistance. There cannot be a misclassification or eligibility confusion for the sampling frames because they are generated strictly by the definition of the target population stated above. There is neither an exclusion of any element, nor an alternative sample frame. There is no post-stratification procedure included in this IC, but the obtained data are aggregated to estimate the customers’ satisfaction level for the entire primary unit or the target population.

A systematic random sampling method is used to select a group of people to be surveyed from the target population stored in electronic data files, usually in the National Emergency Management Information System (NEMIS), in the Direct Assistance Replacement Assistance Consideration (DARAC), and the Recovery Information Management System (RIMS). The target populations have a specific character for each survey; for example, the target population of Caller Services Registration Survey is all people who contacted FEMA to register for the particular disaster assistance. The surveys have an independent sampling from each other. The entity of the element of all samples is individual, and there is no stratification involved in the sampling. More detailed sampling methods and timeliness are provided below for each survey instrument.

The disaster process covers a span of time and the goal is to measure and then report on those services over that span of time. Participation in the survey is available to all applicants within the targeted population based on a random selection. To achieve these goals and not to over burden the disaster survivors, weekly goals are established which will cumulate into a statistically valid response.

Caller Services Registration Survey, Internet Registration, Caller Services Helpline Survey, Internet Inquiry and Casework Representative Surveys and Casework Representative Surveys

The Caller Services Registration and Internet Registration surveys are conducted in successive eight weeks after the beginning of the registration time frame. Each sampling and survey period has a duration of one week, that is, the sample in each week consists of a random selection from all applicants based on the expected call/internet traffic between days 1-7, days 8-14, says 15-21, etc. The weekly sample is prorated based on the volume of registrations by disaster.

The Caller Services Helpline and Internet Inquiry surveys are conducted in approximately thirteen successive periods over thirteen weeks after the first Helpline contact. Each sampling and survey period have a duration of one week, that is, the sample in consists of a random selection from all contacts based on the expected call/internet traffic between days 1-7, days 8-14, days 15-21, etc. The weekly sample is prorated based on the volume of contacts by disaster.

The Casework Representative sampling and surveys will be conducted in approximately twenty-six successive weeks after the Caseworker contacts the survivor. Each sampling and survey period have a duration of one week, that is, the sample in each week consists of a random selection from all contacts based on the expected call traffic between days 1-7, days 8-14, days 15-21, etc. The weekly sample is prorated based on the volume of contacts by disaster.

Disaster Recovery Center Survey

The Disaster Recovery Center Survey surveys will be conducted in successive daily periods for approximately two weeks after the first visit to the Disaster Recovery Center. Each sampling and survey period have a duration of one day, that is, the sample in each period consists of a random selection from all contacts based on the center traffic between days 1-7, and days 8-14. The daily random sample is prorated based on the volume of visits to the recovery centers by disaster.

Disability Access and Functional Needs Representative Survey

The Disability Access & Functional Needs Representative Survey surveys will be conducted on a per disaster basis after the Disability Representative contact has been made with the disaster survivor’s. The random sample is based on the volume of contacts by disaster.

Direct Housing Operations Survey

The Direct Housing Operations (DHOPs) Survey sampling and surveys will be conducted in successive three waves over a period of approximately twenty-six weeks after the first disaster survivor’s DHOPs contact. Each sampling and survey wave has a unique duration in each disaster, that is, the sample in each wave consists of a random selection from all contacts based on the delivery of the temporary housing unit, then based on the contact for maintenance and finally based on the move out date. The sample is prorated based on the volume of temporary housing units distributed by disaster.

Diagnostic Surveys

The Diagnostic Surveys will be conducted first to measure satisfaction with services received and then to re-measure when events occur that change those services to reflect disaster survivor’s satisfaction with the changes. As an example, the document “Help after a Disaster: Applicant’s Guide to the Individuals and Households Program” may be revised and distributed with a new declared disaster. Re-measuring the customer satisfaction with that form of correspondence will provide information about its ease of use for the readers. Suggestions gathered from the respondents will provide opportunities for improvement based on the voice of the customer. A diagnostic survey will have a varying duration of at least one month during the year, that is, the sample in each period consists of a random selection from all recipients of the designated service based on the timing of the receipt of that service and may be prorated based on the volume of service provided by disaster. The survey may also be repeated during the year.

Example: Based on resources, the aim for most survey types is to complete a statistically valid number of surveys based on approximately 400 per month (or the duration of the survey time frame) over the course of the typical life cycle of the disaster. This provides in effect a minimum 95% confidence level and confidence interval of plus or minus 5% at 50% response distribution. The completion of 400 surveys is generally achieved by finishing 100 surveys for each survey type during each period or each week. For each week, a sufficient sample of applicant data is imported through a systematic random sample extracted from the data source.

Response Rate and Decline Rate – See #3 below: The actual decline rate for the four surveys in FY 2010 was 2.31% of all attempts and completes. Completes to attempts was 27% due to the applicant’s unavailability to complete the survey, bad/wrong phone number, busy signals, declines, no answer, voice mail, privacy manager or the applicant did not remember contacting FEMA or was not familiar with the case. The interviewer phone number is blocked from the respondent which has a negative effect on the response rate. Attempts are made to reach the respondent each time when his/her case systematically returns to the call queue. Up to 4 attempts are made to obtain a survey response if necessary to achieve a valid number of completed surveys. If an applicant is not immediately available, an attempt is made to set up another time within the survey period that would be more convenient for the respondent, and the interviewer explains how important his/her feedback is.

Below shows the data on the size of the universe covered by the collection and the corresponding samples for the universe as a whole. In addition, Table 6A shows the target numbers of completed surveys and the confidence levels and confidence intervals pursued at the worst case of response distribution, p = 0.5.

Table 6. Annual estimates of universe and sample sizes. The entity of the sample elements is ‘individual’ for all surveys.
Phone Survey			Total Annual Universe for 2 Yr Avg All DRs*1			2 Yr Average Number of DRs 2008-2009 *2			Universe 2 Yr Average per DR				Sample per Week prorated by DR based on an average response rate of 27%			FEMA Confidence Level 3
Reported by Week
Caller Services Registration Survey			853,064			30			28,435				400			95%
																[+/- 5%]
Caller Services Helpline Survey			248,931			30			8,298				400			95%
																[+/- 5%]
Casework Representative Survey			116,465			30			3,882				400			95%
																[+/- 5%]
Internet Registration Survey			454,565			30			15,152				400			95%
																[+/- 5%]
Internet Inquiry Survey			136,370			30			4,545				400			95%
																[+/- 5%]
Phone Survey			Total Annual Universe for 2 Yr Avg All DRs*1			2 Yr Average Number of DRs 2008-2009 *2			Universe 2 Yr Average per DR				Sample per Week prorated by DR based on an average response rate of 27% *4			Weekly Confidence Level [Margin of Error]*3
Reported by Disaster
Direct Housing Operations Survey			2,641			7			377				377			95%
																[+/- 5%]
Disability Access and Functional Needs Representative Survey			38,187			30			1,273				400			95%
																[+/- 5%]
Phone Survey			Total Annual Universe for 2 Yr Avg All DRs*1			2 Yr Average Number of DRs 2008-2009 *2			Universe 2 Yr Average per DR				Sample per Day prorated by DR based on an average response rate of 29%			Weekly Confidence Level 3
Reported by Day by Disaster
Disaster Recovery Center Survey			85,306			30			2,844				100			95%
																[+/- 5%]
Phone Survey			Total Annual Universe for 2 Yr Avg All DRs*1			2 Yr Average Number of DRs 2008-2009 *2			Universe 2 Yr Average per DR				Sample per One Month- prorated by DR based on an average response rate of 27%			Confidence Level *3
Reported by Three Months (Quarterly)
Diagnostic Surveys:
Communication and Process			57,394			30			1913				400			95% [+/- 5%]
Contact			248,931			30			8298				400			95% [+/- 5%]
Correspondence and Process			316,870			30			10562				400			95% [+/- 5%]
E-Communications Satisfaction Survey			248,931			30			8298				400			95% [+/- 5%]
Evacuation Plan Satisfaction Survey			12,369			30			412				400			95% [+/- 5%]
Rapid Temporary Repair			12,369			30			412				400			95% [+/- 5%]
Site Recertification			57,394			30			1913				400			95% [+/- 5%]
Notes
*1 : Universe size is estimated based on the average number of disaster survivors in FY 2008-2009 and the Percentage of registrants who use the system
*2 : Number of disasters is based on the average number of disasters occurred during FY 2008-2009
*3 : Confidence Level and Confidence Interval are at 50% response distribution (p = proportion = 0.5) for the samples, which are actually given to the interviewers to make survey calls.
*4 : When the sample size is greater than the universe size, universe survey is conducted.

Table 6A. Number of completed surveys (FY 2009 and proposed), and the confidence level and margin of errors pursued at response distribution, p = 0.5.
Phone Survey	Total Number of Completed Surveys in FY 2009	Number of Completed Surveys per Week during the Entire Period	Annual Number of Completed Surveys for 50 Weeks	Annual Pursued Confidence Level
REPORTED BY WEEK
Caller Services Registration Survey	5,076	100	5,000	95% [+/- 5%]
Caller Services Helpline Survey	3,419	100	5,000	95% [+/- 5%]
Internet Registration Survey	3,394	100	5,000	95% [+/- 5%]
Internet Inquiry Survey	496	100	5,000	95% [+/- 5%]
Casework Representative Survey	Proposed	100	5,000	95% [+/- 5%]
Direct Housing Operations Survey	Proposed	59	1,770	95% [+/- 5%]
Disability Access and Functional Needs Representative Survey	Proposed	100	5,000	95% [+/- 5%]
Phone Survey	Total Number of Completed Surveys in FY 2009	Number of Completed Surveys per Day per Disaster during the Entire Period	Annual Number of Completed Surveys for 50 Weeks	Annual Pursued Confidence Level
REPORTED BY DAY BY DISASTER
Disaster Recovery Center Survey	Proposed	210	6,300	95% [+/- 5%]
		30 DRs
Phone Survey	Total Number of Completed Surveys in FY 2009	Number of Completed Surveys per Three Months during the Entire Period	Annual Number of Completed Surveys	Annual Pursued Confidence Level
REPORTED BY THREE MONTHS (QUARTERLY)
Diagnostic Surveys:
Communication and Process	Proposed	400	400	95% [+/- 5%]
Contact Survey	Proposed	400	400	95% [+/- 5%]
Correspondence and Process	Proposed	400	800	98% [+/- 5%]
E-Communications Satisfaction Survey	Proposed	400	400	95% [+/- 5%]
Evacuation Plan Satisfaction Survey	Proposed	400	400	95% [+/- 5%]
Rapid Temporary Repair	Proposed	400	400	95% [+/- 5%]
Site Recertification	Proposed	400	400	95% [+/- 5%]

2. Describe the procedures for the collection of information including:

Once a sample set is obtained, the interviewer calls the individuals in the sample to conduct an interview until statistical validity is reached. In the case of the proposed information collections, sending a pre-notification letter for the survey is not desirable because of the time constraint (see B. #1). In addition, the items listed in question 3 are employed to the information collection instruments to insure the best balance between maximizing data quality and minimizing respondent time burden, some of which are relevant to response rate improvement.

-Statistical methodology for stratification and sample selection:

A systematic random sample is generated in the entire target population from the electronic database in the National Emergency Management Information System (NEMIS) or a program similar to it that contains the names, phone numbers, addresses, and disaster related information of all such applicants.

Each registered applicant from any given disaster will have the same chance of being chosen by the systematic random sampling method. The phone survey sample is imported into the survey database and is randomly populated onto a computer screen from the pool of names using Microsoft Access software or a similar program for the interviewer.

As an example: For Caller Services Registration or Internet Registration, and Internet Inquiry surveys a weekly sample is imported into the survey database by disaster and by call dates soon after the contact has taken place so that all applicants have the opportunity to participate in the survey.

-Estimation procedure: To estimate the satisfaction level of service provided, the top three positive responses (for example, excellent, good and satisfactory) are averaged based on the completed surveys representing the universe of disaster applicants.

-Degree of accuracy needed for the purpose described in the justification: Although extremely accurate statistical inference is not necessary for this information collection, the goal is to achieve a level of accuracy of the estimated customer satisfaction on a functional basis based on call volume 95% confidence level, for all surveys.

-Unusual problems requiring specialized sampling procedures:

There are no unusual problems requiring specialized sampling procedures.

-Any use of periodic (less frequent than annual) data collection cycles to reduce burden: Usage of any periodic data collection cycle is not applicable to this particular type of information collection since disaster occurrences are not predictable enough to schedule a collection cycle in advance.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

• The opening statement explains, briefly, the purpose of the study and nature of being voluntary, and asks for the Applicant's help in order to improve FEMA's response to future disasters.

• The questions are short and require little time to complete (Historically, for all survey types, it takes less than 6- minutes to complete the questions, on the average 6:14 minutes in FY 2008-2009.)

• The questions are very straightforward and easy to answer.

• An explanation is given that questions in no way affect the outcome of their application with FEMA.

• Information gathered from focus groups will be used to ensure that the survey items included are of interest to Individual Assistance applicants, making respondents more likely to see the survey as relevant.

• Revisions will be made to the survey with attention to correcting low response items.

• Callbacks are made to applicants who state they will be available at a later time when feasible and resources allow.

• When limited sample is available, additional attempts are made to contact the applicant up to 4 attempts.

• Training is provided and more experienced interviewers are retained.

• Interpreters are used to obtain results from applicants with other languages.

• To increase the response rate for the Caller Services Registration and Helpline Surveys, the number of questions has been reduced which reduces the burden on the respondents.

• Time frames for the collection have been restructured to be more focused on specific topic of interest to service providers.

• The diagnostic questions will focus on specific processes that change occasionally to measure the satisfaction with those changes.

These surveys are performed by calling disaster assistance applicants. The decline ratio average is 2.31%of the total attempts and completes. The 2010 response rate for the proposed Individual Customer Satisfaction Surveys under the current OMB inventory is 27% using a response-rate formula recognized by the American Association for Public Opinion Research (AAPOR) as following:

RR = I / {(I+P) + (R+NC+O) + U}

, where

RR = Response rate

I = Complete interview

P = Partial interview

R = Refusal and break-off

NC = Non-contact

O = Other (bad/wrong numbers, technical phone problem, etc.)

U = Unknown eligibility (= 0 in this case, see B. #1.)

The relatively low response rate is compared with a distribution of response rates presented in McCart et al., 2006, a paper concerning phone survey response rates. Figure 1 is a histogram of the response rates using the same RR formula for 205 telephone surveys conducted at the University of Florida Survey Research Center at the Bureau of Economic and Business Research between January 2000 and July 2004, which shows the mode response rate 25%, and the mean about 41.5%. Thus, the response rate of 30% is not in fact as low as it looks without any reference for comparison.

Figure 1. Histogram of the response rates for 205 telephone surveys conducted at the University of Florida Survey Research Center at the Bureau of Economic and Business Research between January 2000 and July 2004 (McCarty et al., 2006, Effort in Phone Survey Response Rates: The Effects of Vendor and Client-Controlled Factors, Field Methods, Vol. 18 No. 2, 172-188).

The target population is disaster survivors. Since this information collection is time constraint, and immediately following a disaster, the survivors have to be interviewed while they are still experiencing disaster trauma. In most of the cases the survivors may be in the worst stage of the disaster trauma when they are called for the surveys. Disaster trauma psychology symptoms may include

[http://www.citizencorps.gov/cert/downloads/training/PM-CERT-Unit7Rev3.doc]:

• Irritability or anger

• Self-blame or the blaming of others

• Isolation and withdrawal

• Fear of recurrence

• Feeling stunned, numb, or overwhelmed

• Feeling helpless

• Mood swings

• Sadness, depression, and grief

• Denial

• Concentration and memory problems

• Relationship conflicts/marital discord

• Loss of appetite

• Headaches or chest pain

• Diarrhea, stomach pain, or nausea

• Hyperactivity

• Increase in alcohol or drug consumption

• Nightmares

• The inability to sleep

• Fatigue or low energy

In addition to disaster trauma, frequent relocations are anticipated for the survivors after a disaster, which attributes to the non-contact portion of the non-response. Often survivors do not have telephone service in their community due to the disaster. Considering even during normal stages of everyday life “time-limited polls often yield very low response rates” (McCarty et al. 2006), we believe that we have achieved a very good response rate if not the best possible for this particular type of target population. Nonetheless, we follow the steps described in B. #2 to maintain the current level of success in the response rate though our respondents may be still in disaster trauma during the survey periods.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Many of the questions in the surveys have been performed for six to ten years and were initially based on comments from past focus groups as well as contractor opinion. FEMA personnel also reviewed questionnaire content and wording to improve readability and clarity. Tests with less than 10 applicants may be performed by FEMA’s customer satisfaction analysis staff when updates are desirable, and all updates to questionnaires will be submitted to OMB for approval

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Maggie Billing

Program Analyst

Customer Satisfaction Analysis Section

Texas National Processing Service Center

940 891-8709

12