1660-0036 Supporting Statement Part B 12192013

December 19, 2013

Supporting Statement for

Paperwork Reduction Act Submissions

OMB Control Number: 1660 - 0036

Title: Federal Emergency Management Agency Individual Assistance Customer Satisfaction Surveys

Form Number(s): FEMA Form 007-0-7, Disaster Recovery Center Survey; FEMA Form 007-0-3, Registration Survey (formerly Caller Services Registration); FEMA Form 007-0-5, Helpline/Contact Survey (formerly Caller Services Helpline Survey); FEMA Form 007-0-6 Casework Survey (formerly Casework Representative Survey); FEMA Form 007-0-2, Internet Registration Phone Survey, (formerly Internet Registration Survey); FEMA Form 007-0-23, Internet Registration Electronic Survey; FEMA Form 007-0-19, Internet Inquiry Phone Survey (formerly Internet Inquiry Survey); FEMA Form 007-0-24, Internet Inquiry Electronic Survey(Presented via Internet); FEMA Form 007-0-4, Direct Housing Operations Survey-Move In (formerly Direct Housing Operations Survey); FEMA Form 007-0-21, Direct Housing Operations Survey-Maintenance; and FEMA Form 007-0-22, Direct Housing Operations Survey-Move Out

B. Collections of Information Employing Statistical Methods.



When Item 17 on the Form OMB 83-I is checked “Yes”, the following documentation should be included in the Supporting Statement to the extent it applies to the methods proposed:

If the collection does not involve statistical methodology please enter “THERE IS NO STATISTICAL METHODOLOGY INVOLVED IN THIS COLLECTION” and delete Q1 through 5.

1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

Part B Question #1: Description of Respondent Universe, Sampling Method, Response Rates
Type of Respond-ent / Entity	Form Name / Form Number	Respond-ent Universe Numerical Estimate Basis	Potential Respond-ent Universe Numerical Estimate	Estimated Number of Disasters Per Year		Sample Size Calculation: Average Number of Respond-ents to Achieve a Completion	Annual Sample Size from FEMA Applicants	Estimated Completions per Week based on 50 Weeks Per Year	Sampling Criteria for Target Popula-tion	FY2012 Completed Surveys and Focus Groups	Actual or Expected Survey Response Rates with Actual FY2012 Avg of 30.08%
		Respondent Universe
Surveys			A		B	C	D = A/B*C
Individuals and Households	Disaster Recovery Center Survey FEMA Form 007-0-7	Based on 2012 Estimated Visits to Disaster Recovery Centers (DRC). May include Multiple Visits by Same Applicant, increasing the Universe.	300,000		25	2.50	30,000	187.5	Daily Sample by Disaster and by DRC during First One-Two Weeks of the New Disaster	6,909	35.82%
Individuals and Households	Registration Survey FEMA Form 007-0-3	Based on 5 Yr Avg of 25 Disasters and Current Avg of 50% Registrations by Phone or In Person.	291,528		25	3.50	40,814	256	Weekly Sample Prorated by Disaster of Phone Registrations	3,593	29.03%
Individuals and Households	Helpline/ Contact Survey FEMA Form 007-0-5	Based on 5 Yr Avg of 25 Disasters and Current Avg of 45% Helpline Contacts.	262,375		25	4.00	41,980	256	Weekly Sample Prorated by Disaster of Helpline Contacts.	3,652	26.85%
Individuals and Households	Casework Survey FEMA Form 007-0-6	Based on Current 2 Yr 7.5 Mons and Current Avg 40% Casework.	71,145		25	4.00	11,383	100	Weekly Sample Prorated by Disaster of Casework Contacts.	New Program	Based on Helpline Response Rate as a Similar Contact 26.85%
Individuals and Households	Internet Registration Survey FEMA Form 007-0-2	Based on 5 Yr Avg of 25 Disasters and Current Avg of 50% Internet Registrations by Internet andt 30% with email prefernce = no.	87,458		25	3.50	12,244	100	Weekly Sample Prorated by Disaster of Internet Registrations.	2,884	25.90%
Individuals and Households	Internet Registration Survey FEMA Form 007-0-2 INT	Based on 5 Yr Avg of 25 Disasters and Current Avg of 50% of Internet Registrations at 70% with email preference = yes.	204,070		25	3.50	28,570	100	Weekly Sample Prorated by Disaster of Internet Registrations with email preference.	New Program	Based on Internet Registration Response Rate as a Similar Contact 25.90%
Individuals and Households	Internet Inquiry Survey FEMA Form 007-0-19	Based on Current 2 Yr 7.5 Mon Avg and Current 12% Internet Interactions and 22% email preference = no.	11,975		25	3.25	1,557	100	Weekly Sample Prorated by Disaster of Internet Interactions.	954	27.96%
Individuals and Households	Internet Inquiry Survey FEMA Form 007-0-19 INT	Based on Current 2 Yr 7.5 Mon Avg and Current 12% Internet Interactions based on 78% email preference = yes.	42,457		25	3.25	5,519	100	Weekly Sample Prorated by Disaster of Internet Interactions with email preference.	New Program	Based on Internet Inquiry Response Rate as a Similar Contact 27.96%
Individuals and Households	Direct Housing Operations Survey, Move In FEMA Form 007-0-4	Based on Current 3 Yr 7.5 Mon Avg of Actual Eligible Temporary Housing Units.	1,158		5	Universe (unless the sample volume on a per disaster basis exceeds	1,158	23	Weekly Sample Prorated by Disaster or When Volume is Minimal, a Universe of Respondents will be Sampled.	0	Based on Helpline Response Rate as a Similar Contact 26.85%
Individuals and Households	Direct Housing Operations Survey, Maintenance FEMA Form 007-0-X	Based on Current 3 Yr 7.5 Mon Avg of Actual Eligible Temporary Housing Units.	1,158		5	Universe	1,158	23	Weekly Sample Prorated by Disaster or When Volume is Minimal, a Universe of Respondents will be Sampled.	0	Based on Helpline Response Rate as a Similar Contact 26.85%
Individuals and Households	Direct Housing Operations Survey, Move Out FEMA Form 007-0-X	Based on Current 3 Yr 7.5 Mon Avg of Actual Eligible Temporary Housing Units.	1,158		5	Universe	1,158	23	Weekly Sample Prorated by Disaster or When Volume is Minimal, a Universe of Respondents will be Sampled.	0	Based on Helpline Response Rate as a Similar Contact 26.85%
Total Survey Sample Size							175,541	1268.5		17,992	30.08% Avg
Focus Groups
Individuals and Households, Partners In Service Staff	Focus Group	Based on 5 Yr Avg of 25 Disasters Total Registrations and Eligible Housing Assistance Applicants.	154,438		25					0
Individuals and Households, Partners In Service Staff	Travel to Focus Group	Travel for Focus Groups	N/A		N/A					N/A
Individuals and Households, Partners In Service Staff	One-on-One Interviews	Based on 5 Yr Avg of 25 Disasters Total Registrations and Eligible Housing Assistance Applicants.	154,438		25					10	2.60%
Individuals and Households, Partners In Service Staff	On-Line Interviews	Based on 5 Yr Avg of 25 Disasters and Current Avg % of Those Using the Internet to Register.	291,528		25					0
Focus Groups Total							0	0		10
Surveys and Focus Groups							175,541	1268.5	0	18,002	30.08% Avg

The table above shows the estimated size of the universe covered by the collection and the corresponding samples for each survey.

2. Describe the procedures for the collection of information including:

-Statistical methodology for stratification and sample selection:

All surveys are time-limited and based on target populations of individuals and households who are disaster survivors seeking federal assistance after a declared disaster. The disaster process covers a span of time and the goals is to measure and then report on those services over the span of time. Weekly or daily goals, based on each survey’s purpose, are established which will cumulate into a statistically valid response.

Each survey has an independent sampling, each sample is individual, and there is no stratification involved in the sampling. The sampling frames consist of the names of all the disaster survivors who have contact with FEMA for disaster assistance for each targeted audience. There cannot be a misclassification or eligibility confusion for the sampling frames because they are generated strictly by the definition of the target populations stated above. There is neither an exclusion of any element, nor an alternative sample frame. There is no post-stratification procedure. The responses are aggregated to estimate the customers’ satisfaction level for the target population.

For the Registration, Helpline/Contact, Casework, Internet Registration Phone, Internet Registration Electronic, Internet Inquiry Phone, Internet Inquiry Electronic, and Disaster Housing Operations (3) Surveys of Move-In, Maintenance and Move-Out, a random sample is generated from the entire target population from the electronic data files in the National Emergency Management Information System (NEMIS) Individual Assistance (IA) Client replicated to the Enterprise Data Warehouse or the Direct Assistance Replacement Assistance Consideration (DARAC) Client, which contain the names, phone numbers, email addresses, and disaster related information of all such survivors. The survey sample is imported into the Customer Satisfaction Analysis System where the survey is stored.

For the Disaster Recovery Center Survey (DRC) the names and phone numbers of all visitors is generated from the Recovery Information Management System (RIMS) or from a reception log.

The target populations have a specific character for each survey. More detailed sampling methods and timelines are provided below for each survey instrument.

Disaster Recovery Center Survey

The Disaster Recovery Center (DRC) Survey surveys are conducted in successive daily periods over approximately one to two weeks after the survivor’s visit to the Disaster Recovery Center (DRC) and after the beginning of the survey process. Each sampling and survey period has a duration of one day, that is, the sample in each period consists of all survivors based on all of those who visited the DRC between days 1, 2, 3, 4, etc. The daily sample is based on the volume of visits to the DRC by disaster.

In a disaster when visitor volume at the DRC is high, the sample in each period consists of a random selection of all survivors based on all of those who visited the DRC between days 1, 2, 3, 4, etc.

Registration Survey, Internet Registration Phone Survey, Helpline/Contact Survey, Internet Inquiry Phone Survey, and Casework Survey:

The Registration and Internet Registration Phone Surveys are conducted by phone for the duration of the registration period in approximately nine successive periods over nine weeks after the beginning of the registration process and the beginning of the survey processes. Each sampling and survey period has a duration of one week, that is, the sample in each period consists of a random selection from all survivors based on all contacts or internet usage and email preferences between days 1-7, days 8-14, days 15-21, etc. The weekly random sample is prorated based on the volume of registrations by disaster.

The Helpline/Contact and Internet Inquiry Phone and Internet Inquiry Electronic Surveys are conducted by phone through the duration of the registration period in approximately nine successive periods over nine weeks after the survivor contacts the FEMA Helpline, is contacted in person, or accesses their case on the Internet and after the beginning of the survey processes based on email preferences. Each sampling and survey period has a duration of one week, that is, the sample in each period consists of a random selection from all survivors based on all contacts or internet usage and email preferences between days 1-7, days 8-14, days 15-21, etc. The weekly random sample is prorated based on the volume of contacts by disaster.

The Casework Surveys are conducted by phone through the duration of the registration period in approximately nine successive periods over nine weeks after the Representative contacts the survivor and after the beginning of the survey process. Each sampling and survey period has a duration of one week, that is, the sample in each period consists of a random selection from all survivors based on all contacts between days 1-7, days 8-14, days 15-21, etc. The weekly random sample is prorated based on the volume of contacts by disaster.

Internet Registration Electronic Survey Presented via the Internet and Internet Inquiry Electronic Survey Presented via the Internet

The Internet Registration Electronic Survey will be conducted via the Internet for the duration of the registration period in approximately nine successive periods over nine weeks after the beginning of the registration process and the beginning of the survey process. Each sampling and survey period has a duration of one week, that is, the sample in each period consists of a random selection from all survivors based on all internet cases with email preferences between days 1-7, days 8-14, days 15-21, etc. The weekly random sample is prorated based on the volume of registrations by disaster.

The Internet Inquiry Electronic Survey will be conducted via the Internet through the duration of the registration period in approximately nine successive periods over nine weeks after the survivor contacts the FEMA Helpline, is contacted in person, or accessing their case on the Internet and after the beginning of the survey processes. Each sampling and survey period has a duration of one week, that is, the sample in each period consists of a random selection from all survivors based on all internet cases with email preferences between days 1-7, days 8-14, days 15-21, etc. The weekly random sample is prorated based on the volume of contacts by disaster.

Direct Housing Operations Survey Move In, Direct Housing Operations Survey Maintenance and Direct Housing Operations Survey Move Out

The Direct Housing Operations (DHOP) Surveys are conducted in three successive/longitudinal periods or in individual periods over a period of approximately twenty-six weeks after each survivor contact. Each sampling and survey period has a unique duration in each disaster, that is, the sample in each period consists of a random selection from all contacts based on the delivery and move into the temporary housing unit (Move In); the next sample is based on the contact for maintenance (Maintenance); and the final is based on the move out date (Move Out). The sample is prorated based on the volume of temporary housing units distributed by disaster.

-Estimation procedure:

For the Registration, Helpline/Contact, Casework, Internet Registration Phone, Internet Registration Electronic, Internet Inquiry Phone, Internet Inquiry Electronic, and Disaster Housing Operations (3) Surveys of Move-In, Maintenance and Move-Out, the survey sample is based on the FY2013 sample size calculation of the average number of respondents required to achieve completion the goal for each survey type.

For the Disaster Recovery Center Survey (DRC), the survey sample is the population of visitors to the DRC in each disaster, except in a disaster when visitor volume at the DRC is high, the survey sample will be based on the FY2013 sample size calculation of the average number of respondents required to achieve completion the goal for the survey type.

-Degree of accuracy needed for the purpose described in the justification:

Although extremely accurate, statistical inference is not necessary for this information collection, the goal is to achieve a level of estimated customer satisfaction based on a response volume at a 95% confidence level, plus or minus 5%, for all surveys, at a 50% response distribution.

Example: The aim for the Registration, Helpline/Contact, Casework, Internet Registration Phone, Internet Registration Electronic, Internet Inquiry Phone, or Internet Inquiry Electronic Surveys is to complete a statistically valid number of surveys based on approximately 400 per month by finishing 100 surveys for each survey type during each period or each week for the duration of the survey time frame over the course of the typical life cycle of the disaster. A sufficient weekly sample of survivor data for the target audience is imported into the survey system extracted from the data source by contact dates, by preference for use of email communication, and by disaster soon after the contact has taken place so that all survivors have the same opportunity to participate in the survey. This provides in effect a 95% confidence level and confidence interval of plus or minus 5% at a 50% response distribution.

-Unusual problems requiring specialized sampling procedures:

There are no unusual problems requiring specialized sampling procedures.

-Any use of periodic (less frequent than annual) data collection cycles to reduce burden:

Usage of any periodic data collection cycle is not applicable to this particular type of information collection since disaster occurrences are not predictable enough to schedule a collection cycle in advance.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Extremely accurate statistical inference is not necessary for the intended use of this Information Collection. Results provide reliable customer satisfaction levels as well as information about areas that need improvement.

The actual average response rate for FY2012 was 30.08%, greater than normally expected ranges for phone surveys. (Research Studies follow below.) Survey efforts to maximize and further increase response rate are listed below.

• The scheduling of the phone surveys covers a span of time between 9 am and 8 pm, typically Monday-Friday in the time zone of the respondent with additional attempts made during a different time frame.

• Callbacks are attempted to survivors who state they will be available at another time within the survey period that would be more convenient for the respondent.

• The interviewer explains how important the feedback is.

• Multiple attempts are made to reach the survivor by phone each time the case systematically returns to the call queue or by email during the survey period.

• The opening statement briefly explains the purpose of the study, the nature of being voluntary, and asks for the survivor's help in order to improve FEMA's quality of service.

• The questions are very straightforward and easy to answer.

• The questions are short and require little time to answer. Historically, the combined average time to complete the questions takes less than 6 minutes (5:58 average), with an average range of 5:07 minutes to 8:01 minutes based on the current survey types used in FY2013 as of 4/30/13.

• An explanation is given that the questions will in no way affect the outcome of the disaster survivor’s application for assistance.

• Information gathered from focus groups will be used to ensure that the survey items included are of interest to disaster survivors, making respondents more likely to see the survey as relevant.

• On-going training is provided to interviewers.

• Interpreters are used to obtain results from survivors more comfortable with other languages.

• Time frames for the survey types are structured to be focused on specific topic of interest to the targeted audience and the service providers.

Note: Sending a pre-notification letter for the survey is not desirable because of the time constraints for each survey type and survey period. (See Part B #2 above.)

The response-rate formula used is recognized by the American Association for Public Opinion Research (AAPOR) as follows:

RR = I / {(I+P) + (R+NC+O) + U}, where

RR = Response rate

I = Complete interview

P = Partial interview

R = Refusal and break-off

NC = Non-contact

O = Other (bad/wrong numbers, technical phone problem, etc.)

U = Unknown eligibility (= 0 in this case, see B #2.)

While the response rate of 30.08% for this customer satisfaction collection is greater than a recent study in 2012 by Pew Research Center (See Research Studies-2 below), factors that contribute to the non-response portion may be due to the disaster; such as, applicants who are survivors often do not have telephone service, cell phone service, nor electrical service in their community due to the disaster. They may be in the clean-up phase of their recovery and unavailable for a survey. Frequent relocations and displacements are anticipated affecting the respondent’s availability to complete the survey. Survivors may not want to use their cell phone minutes to respond to a survey. Disaster trauma may be a factor and the survivor may not remember contacting FEMA or was not familiar with the case. Other factors include bad/wrong phone number, busy signals, no answer, voice mail, and privacy manager.

Research Study 1-Response Rate:

McCarty et al., 2006, a paper concerning phone survey response rates from 205 phone surveys conducted at the University of Florida Survey Research Center at the Bureau of Economic and Business states on Pages 172-173, “…recent research has shown that the effect of nonresponse on data is less critical than previously thought (Curtin, Presser, and Singer 2000; Keeter et al. 2000). This helps put response rates in perspective and reduces the tendency to disregard survey research simply because of low response rates.”

On Page 183, Figure 1 is a histogram of the response rates for 205 telephone surveys, which shows the mode response rate 25% and the mean about 41.5%. (McCarty et al., 2006, Effort in Phone Survey Response Rates: The Effects of Vendor and Client-Controlled Factors, Field Methods, Vol. 18 No. 2, 172-188).

Research Studies 2-Response Rate, Sample Size:

Other research reflects industry-wide increase in sample size just to keep response rates at a current level. Below are bullets from various research articles justifying low response rates including cell phone usage and survey fatigue.

• Increase - Sample sizes are increasing in order to keep response rates at the current level

• Cell phones are becoming primary residence phones

• Unwillingness to answer “Unknown” numbers; recent survey 54.4% would not answer (Buskirk et al., 2008)

• Industry trend in difficulty of reaching respondents (see fig. 2)

• Survey Fatigue (Hader et al., 2012; Kohut et al., 2012)

• Nonresponse Bias

• Is not directly correlated to nonresponse – if response rate is low, does not mean bias (Groves, 2006)

• Increasing response rates by reducing non-contacts can exacerbate disparities among respondents, ex. Income, urbanicity, etc. and create a bias (Braick et al., 1996; Dennis et al, 1999; Groves, 2006)

Figure 2 (below) represents the decline of typical response rates for surveys conducted from 1997-2012. Results were extracted from the PEW Research Center’s 2012 Methodology Study. Response Rates were computed according to the AAPOR standard formula.

Figure 2: Surveys Face Growing Difficulty Reaching, Persuading Potential Respondents:
	1997	2000	2003	2006	2009	2012
	%	%	%	%	%	%
Contact rate
(percent of households in which an adult was reached)	90	77	79	73	72	62
Cooperation rate
(percent of households contacted that yielded an interview)	43	40	34	31	21	14
Response rate
(percent of households sampled that yielded an interview)	36	28	25	21	15	9

Other Sample size research references:

1. Brick, J. M., Allen, B., Cunningham, P., & Maklan, D. (1996). Outcomes of a calling protocol in a telephone survey. Proceedings of the Survey Research Methods Section of the American Statistical Association, Alexandria, VA.

2. Buskirk, T. D., Rao, K., & Kaminska, O. (2008). My cell phone’s ringing, “caller unknown,” now what? Usage behavior patterns among recent landline cord cutters who have become cell phone only users. The American association for (AAPOR) 63rd annual conference, 2008 & AAPOR 61th annual conference.

3. Dennis, J. M., Saulsberry, C., Battaglia, M. P., Roden, A., Hoaglin, D. C., Frankel, M., et al. (1999). Analysis of Call Patterns in a Large Random-Digit-Dialing Survey: The National Immunization Survey. Conference website of the International Conference on Survey Nonresponse 1999: 1-23.

4. Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675.

5. Häder, S., Häder, M., & Kühne, M. (Eds.). (2012). Telephone Surveys in Europe. Springer: London.

6. Kohut, A., Keeter, S., Dohetry, C., Dimock, M., & Christian, L. (2012). Assessing the representativeness of public opinion surveys. The Pew Research Center For the People & The Press.

7. Reimer, B., Roth, V., & Montgomery, R. (2012, July). Optimizing call patterns for landline and cell phone surveys. Presentation delivered at Joint statistical meetings, San Diego, CA.

8. Van Rooy, C., van Steenis, J.C. (1999). Bellen & Gebeld Worden: fabels en feiten. In: Calling and being called: Fables and facts, MOAjaar-boek.

Research Study: Disaster Trauma

Considering even during normal stages of everyday life, “time-limited polls often yield very low response rates” and “survey fatigue,” this collection has achieved a very good response rate, if not the best possible for this particular type of population. The survey efforts described above in Part B #3 are utilized to achieve the success in the response rate even though respondents may be still in disaster trauma during the survey periods. Surveys have to be performed during the disaster response time frames cited in Part B #2 to achieve accurate recall of the interaction and customer satisfaction.

Disaster trauma psychology symptoms may include the following based on the Community Emergency Response Team-Citizen Corps Training for disaster psychology:

http://www.citizencorps.gov/cert/IS317/medops/medops/index03.htm

Irritability or anger	Relationship conflicts/marital discord
Self-blame or the blaming of others	Loss of appetite
Isolation and withdrawal	Headaches or chest pain
Fear of recurrence	Diarrhea, stomach pain, or nausea
Feeling stunned, numb, or overwhelmed	Hyperactivity
Feeling helpless	Increase in alcohol or drug consumption
Mood swings	Nightmares
Sadness, depression, and grief	The inability to sleep
Denial	Fatigue or low energy
Concentration and memory problems

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Most of the questions in the surveys have been performed for ten years and were initially based on comments from past focus groups as well as contractor recommendations. FEMA personnel also reviewed the questionnaire content and wording to improve readability and clarity. Tests with less than 10 survivors may be performed by FEMA’s Customer Satisfaction Analysis staff when updates are desirable.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Maggie Billing

Program Analyst

Customer Satisfaction Analysis Section

Texas National Processing Service Center

940 891-8709

Kyle M. Mills, P.E.

Section Manager

Customer Satisfaction Analysis Section

Texas National Processing Service Center

940 891-8881

15