1660-0107 Supporting Statement Public Assistance PART B

Supporting Statement for

Paperwork Reduction Act Submissions

Nov. 1, 2011

OMB Control Number: 1660 - 0107

Title: FEMA Public Assistance Customer Satisfaction Survey

Form Number(s):

FEMA Form 519-0-1 T, Public Assistance Customer Satisfaction Survey (Telephone)

FEMA Form 519-0-1 INT, Public Assistance Customer Satisfaction Survey (Web)

FEMA Form 519-0-1, Public Assistance Customer Satisfaction Survey (Fill-able)

B. Collections of Information Employing Statistical Methods.



When Item 17 on the Form OMB 83-I is checked “Yes”, the following documentation should be included in the Supporting Statement to the extent it applies to the methods proposed:

1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

The sample is the universe of Public Assistance applicant organizations who received assistance after a presidentially-declared major disaster. The universe is based on a three year average of 10,500 applicants per year and a three year average of 70 disasters declared per year from FY2008 through FY2010. The entities consist of approximately 90% local, state or tribal governments and 10% private non-profit organizations.

The estimated average number of respondents per disaster is based on the three year average of 70 declared disasters per year, an average of 10,500 respondents per year for an average of 150 recipients of assistance per disaster. In FY2010 the actual number of recipients per disaster ranged from a low of 3 to a high of 1749.

The response rate will be calculated using the following formula:

Number of Completed Surveys

(Number of Applicants – Number of Undeliverable Surveys)

This is a fairly new information collection and sufficient data has not been gathered at the time of this filing to present a true historical response rate. The available data to date from FY2011 does indicate an actual response rate of 52.60% based on a universe of 1,135 applicants with 597 completed surveys for 8 disasters surveyed. The objective is to achieve an 80% response rate.

Completed surveys in FY2011 as of date of filing:

Annual Estimates of Universe of Completed Surveys
		Total Estimated Annual Universe Based on 3 Yr Avg All Disasters1		Estimated 3 Yr Average Number of Disasters FY 2008-2010 *2		Estimated Universe 3 Yr Average per Disaster			Total Number of Completed Surveys As of Filing of IC with Actual Response Rate of 52.60%
Private Non-Profit Organizations = Estimated 10% of Universe		1,050		70		15			11
State, Local or Tribal Government = Estimated 90% of Universe		9,450		70		145			586
Total Public Assistance Customer Satisfaction Surveys		10,500		70		150			597
Notes

2. Describe the procedures for the collection of information including:

-Statistical methodology for stratification and sample selection:

The sampling frame is the universe of recipients of Public Assistance funds which are private non-profit organizations, local, state and tribal governments. The sample is generated from data extracted from the Emergency Management Mission Integrated Environment (EMMIE) electronic database or a program similar to it that contains the points of contact names, phone numbers, email addresses, mailing addresses and disaster related information. The survey is sent to all the primary units in the sampling frame for each disaster. There is no stratification in the sampling.

-Estimation procedure:

The method of calculating satisfaction with each performance measure is by combining the positive response choices (i.e., very satisfied, satisfied and slightly satisfied) to a question. Responses that are not included when calculating satisfaction, such as do not know, the various not applicable response choices, and missing responses, are removed from the calculation of responses for that question. For example: the formula for calculating percent satisfaction for 24 of the 26 performance measuring items is as follows:

Very Satisfied + Satisfied + Slightly Satisfied

Total Respondents – (Do Not Know + All Not Applicable Responses +

Not Answered/Bad Response)

For the other two questions, “Was staff turnover a problem?” and “Were site visits performed in a timely manner?” the response choice options differ from that described above. The response choices for staff turnover are “yes” or “no.” In this case, the positive response choice is “no,” staff turnover was not a problem. The response choice options for the measure concerning the timing of Project Worksheet site visits are “too soon after the disaster,” “at the right time,” “too late to be helpful.” The positive response choice for this measure is “at the right time.” In these two cases, the formula for calculating the percent satisfaction for these two measures is:

Positive response choice

Total Responses – (Do Not Know + Not Applicable + Not Answered/Bad Response)

Each survey response is given equal weight when calculating percent customer satisfaction for a given disaster.

-Degree of accuracy needed for the purpose described in the justification:

Surveying the entire universe of Public Assistance fund recipients, including private non-profit organizations, local, state and tribal governments, yields the highest degree of precision, accuracy, representativeness, reproducibility, and completeness of the survey sample. Steps have been taken to reduce sampling error by inviting the entire population of recipients of Public Assistance to participate and by trying to increase the number of responses by follow-up attempts and reminders. Non-sampling error, caused by non-response, is attempted to be reduced. Strategies have been adopted to maximize the response rate so that the results are representative of the entire population of Public Assistance applicant organizations for any given disaster.

-Unusual problems requiring specialized sampling procedures:

There is no anticipation of any unusual problems on hard to reach populations other than those who do not have up-to-date phone numbers, email or mail addresses in the EMMIE database system. The sampling list of phone numbers, email or mail addresses comes from that system. For office locations where assistance has recently been awarded, the contacts are generally accurate in the EMMIE database because the applicant has recently asked for funds and the contact provided a correct phone number or address.

-Any use of periodic (less frequent than annual) data collection cycles to reduce burden:

Usage of any periodic data collection cycle is not applicable to this particular type of information collection since disaster occurrences are not predictable enough to schedule a collection cycle in advance.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

This is a fairly new information collection and sufficient data has not been gathered at the time of this filing to present a true historical response rate. The available FY2011 data does indicate a response rate of 52.60% based on 597 completed surveys out of the universe of 1,135 applicants in 8 disasters that were surveyed. The objective is to achieve an 80% response rate.

There is no financial incentive for the respondents to answer the survey questions. The incentive is being able to express their opinion and evaluate their satisfaction or dissatisfaction with their recent service, which will assist in improving other organizations’ satisfaction in future disasters.

Surveys are expected to be conducted around 180-270 days from the disaster declaration date. Appropriate follow-up activities are expected to be performed to achieve a survey response rate goal of approximately 80%. This time lag allows applicants time to complete all the steps necessary to apply for and obtain funds from FEMA, since the survey asks for feedback on all these processes. Applicants will be phoned initially to volunteer to participate in the survey. Based on the availability of the point of contact, the survey will be conducted at that time or a call back time will be established. If the customer is not available and based on the customer’s preference, an email will be sent with the web link or the fill able form. If the applicant does not have email capabilities, the form will be faxed or mailed. If the web survey or paper/fill able form survey is used, the applicant is allowed a maximum of 60 days to respond to the survey.

Mail and on-line survey tools typically, however, achieve a response rate of between 5% and 30%. Improved timeliness in providing the survey questionnaire, more frequent and timely follow-ups for the survey, staffing the dedicated Public Assistance survey helpline, and using email for distribution will be used to increase the survey response rates.

It is expected that these measures will help to maintain sufficiently high response rates suitable to analysis, but in the event of response rates fall below 80%, a non-response analysis will be performed.

For the surveys conducted at the time of this filing, the non-response bias was studied based on the actual time frame for responding to the survey. In groupings of 0-13 days for completing the survey, 14-27 days, 28 to 41 days, and 42 days or greater, there was a satisfaction variation of 1.39% for the latest responders (assumed to be similar to non-responders) in comparison with the early responders. Therefore, the results of the responders can be assumed to be similar to the non-responders in their ratings of this type of primarily customer service based survey. In the future, further analysis can be provided if needed.

Measures to Maximize Response Rates:

• Ensure a short time lag between the disaster and data collection

• Conduct follow-ups in the form of phone calls, re-emailing, re-faxing or re-mailing the fill able form or web link with a reminder encouraging participation

• Provide a Public Assistance survey helpline to aid in providing return address information and encouragement based on the purpose of the survey to ensure a more frequent likelihood of a returned response

• The opening statement explains, briefly, the purpose of the study and nature of being voluntary, and asks for the Applicant's help in order to improve FEMA's response to future disasters

• The questions are short and require little time to complete

• The questions are very straightforward and easy to answer

• Information gathered from focus groups will be used to ensure that the survey items included are of interest to Public Assistance applicants, making respondents more likely to see the survey as relevant

• Revisions will be made to the survey with attention to correcting low response items

• Current phone numbers and email addresses are requested with the first contact to the applicant

• Callbacks are made to applicants who state they will be available at a later time when feasible and resources allow

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

At the beginning of each collection period, a pilot test may be conducted on up to 10 persons to discover any potential problems with the survey instrument or process. For quality assurance purposes, data from the pilot will be reviewed and improvements made to the survey process as deemed necessary.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Kathy Canaday

Program Analyst

Customer Satisfaction Analysis Section

Texas National Processing Service Center

FEMA

940 891-8856

Or

Maggie Billing

Program Analyst

Customer Satisfaction Analysis Section

Texas National Processing Service Center

FEMA

940 891-8709

6