Summary of Results for 2013 American Housing Survey
Emergency and Disaster Preparedness Module
Aleia Clark and Terry DeMaio
Center for Survey Measurement
November 29, 2012
At the request of the Census Bureau’s Demographic Surveys Division, the Center for Survey Measurement (CSM) conducted cognitive testing of the Emergency and Disaster Preparedness Module that is part of the 2013 American Housing Survey.
The AHS is a longitudinal survey that returns to the same housing units every other year to gather data. The Emergency and Disaster Preparedness Module is designed to collect information about household readiness and planning for potential evacuation in case of emergency.
One round of six cognitive interviews was conducted in September 2012. The interviews were conducted in the Washington DC metropolitan area by Field Representatives who had been trained in cognitive interviewing methods.
Respondents were recruited through advertisements in the Washington Post Express, postings on Craigslist.com, CSM’s database of recruited respondents and personal networks. Target characteristics for respondents included the following: owners and renters; people living in single-unit attached structures, single-unit detached structures, and multi-unit structures; households with more than one adult; households with children under age 18; people who lived within about 10 miles of water; and people with pets.
The table below documents the characteristics of the cognitive interview respondents:
|
Number of Rs |
Gender |
|
Female |
3 |
Male |
3 |
|
|
Race |
|
White |
3 |
Black |
2 |
Asian |
1 |
|
Number of Rs |
Education |
|
High School or GED |
1 |
Some College |
2 |
College Graduate |
2 |
Graduate Education |
1 |
|
|
Age |
|
22-34 |
1 |
35-44 |
N/A |
45-54 |
2 |
55-64 |
3 |
Respondents were instructed to “think aloud” as they were answering the survey questions to facilitate understanding how respondents interpreted the survey questions and how they formulated their responses. A protocol containing pre-scripted probes was used to determine how respondents interpreted specific terms, reference periods, etc., in the questions and unscripted probes were asked when respondents seemed to be having difficulty with the questions or when they made statements that the interviewers deemed worthy of follow-up. Respondents received a stipend of $40 for their participation. Interviews were conducted at the Census Bureau’s cognitive laboratory or at places convenient to respondents such as commercial establishments.
The questionnaire that was used in the cognitive interviews is included as Attachment A.
Results and Recommendations
In this section we provide a question-by-question presentation of the results and recommendations for revising the questionnaire when necessary. A copy of the questionnaire with our final recommendations is included as Attachment B.
INTRO 1: Now I will ask you some questions about your home and the people who live here.
|
|
Each of the six respondents answered that there were two or more people living in their home. One respondent questioned whether her daughter’s boyfriend who she described as “over a lot” should be counted. When probed, this respondent did not count her daughter’s boyfriend and explained that she was only counting family members when deciding who to include in the count. Respondents did not have difficulty with this question.
Recommendation: We do not recommend any changes to this question.
|
Most respondents did not have difficulty with this question. Two respondents answered that their homes were condominiums rather than answering with one of the choices provided.
Recommendation: We do not recommend any changes to this question.
(FR instructions: Read categories until “yes” reply is received)
|
All six respondents answered this question. One respondent answered that the home was “owned by a friend” but that she also contributed by paying rent. Respondents did not seem to have difficulty with this question.
Recommendation: We do not recommend any changes to this question.
INTRO 2: The next series of questions are to determine your preparedness for emergency situations, such as a fire, and for large-scale disaster situations, such as a flood or earthquake.
|
Respondents did not immediately have difficulty with the introduction; however, later in the interview respondents mentioned two issues. One respondent mentioned that the flow of questions in this section was confusing. The questionnaire moves from asking about the building to evacuation planning, to evacuation 50-or-more miles away and then back to the building. Another respondent questioned the type of emergency to which the series of questions referred. The respondent wanted to know if a “terrorist attack” was included in the definition of emergency situations and mentioned that the situation could be more clearly defined.
Recommendation: We recommend that the questions be reordered to reflect a more coherent and logical progression of increasing distance from the home. This reordering is reflected in Attachment B. We also suggest the sponsor consider broadening the wording to include man-made disasters such as terrorism.
Sponsor’s Feedback: The sponsor accepted this recommendation and reordered the questions to improve the flow of the survey. The sponsor also broadened the scope of disasters to be included in the survey. The new Intro reads: “The next series of questions are to determine your preparedness for emergency situations, such as a fire or terrorist hazard, or large-scale disaster situations, such as a flood or earthquake.”
|
Two respondents answered “no” and four respondents answered “yes” to this question. Two respondents who answered yes explained that their house numbers were not reflective but were illuminated at night and thus clearly readable. Respondents did not have difficulty understanding or responding to this question.
Recommendation: We do not recommend any changes to this question.
|
Five of the six respondents answered “no” to this question. There is a potential problem with the term “kit.” One respondent who marked “no” explained that she had emergency supplies but that they were not packed and ready to go. The term “kit” prompted her to answer “no” because she understood the word to mean that her supplies would have to be packed and ready to go to qualify as a kit.
Recommendation: We have no recommendation for changes assuming that the sponsor intends the term “kit” to mean readily-available.
Sponsor’s Feedback: The sponsor accepted the recommendation and revised the question to clarify the term kit. The sponsor also removed the word “necessary.” The new question reads: “[Do you/Does your household] have emergency supplies readily available to take with you if you have to evacuate your home?”
|
Four respondents answered “no” to this question and two respondents answered “yes.”
Respondents assumed that a disaster plan for financial information implied a written version of the information. They viewed this question as outdated due to the capabilities of smartphones and other mobile devices for storing information. There was also evidence of data security concerns. One respondent remarked that it would be “weird” to have a written document saying, “This is how you get into my bank account.” Another said it “…would be insecure to have documents sitting there with everything on it.”
Recommendation: To resolve this issue we recommend that the sponsor update the language in this question to read: “[Would you/Would your household] have access to your vital financial information and contact numbers if you had to evacuate your home, either in paper documents or through a smartphone?”
Sponsor’s Feedback: The sponsor accepted the recommendation in principle and revised the question to read: “[Would you/Would your household] have access to your vital financial information and contact numbers if you had to evacuate your home?”
|
One respondent answered “no” to this question. Five respondents answered “yes.” Respondents did not have difficulty answering this question.
Recommendation: We do not recommend any changes to this question.
|
Five of the six respondents answered “yes” to this question and only one respondent answered “no.” One respondent commented that even though she and her husband take public transportation to work, they both have reliable vehicles that they could use to transport themselves, their pets and supplies in case of an emergency.
Recommendation: We do not recommend any changes to this question.
|
Four respondents answered “no” to this question. One respondent answered “yes” and one respondent said he did not have any pets.
The distinction between the response categories “No” and “No pets” is not clear. When probed, one respondent who answered “no” explained that she has time-release food that she would leave for her fish in case of emergency. Assuming this is not the type of pet the sponsor has in mind, this response would be inaccurate, since it implies that she is included in the universe of pet owners.
Recommendation: To clarify this distinction we have two suggestions for differentiating between the “No” and “No pets” responses. The first suggestion is to add a screener question asking the respondent if there are pets in the household and then only asking Question 9 if the respondent answers “yes” to the screener. The screener could read: “[Do you/Does your household] have any pets living with you?” The specific types of pets intended by the sponsor could be added as a supplementary instruction (for example, if fish are intended to be included by the sponsor).
The second suggestion would be to revise the question to read: “If [you/your household] had to evacuate from your town or city to a safe place at least 50 miles away, would you need assistance in evacuating or sheltering your pets, if you have any?” This second option is not as definitive as the first option although it might induce respondents to respond that they do not have any pets.
Sponsor’s Feedback: The sponsor accepted the first suggestion and changed this question into two. The questions now read: “Pets are also a concern during an emergency. [Do you/Does your household] have any pets?” If the answer is yes, then the next question is read: “If [you/your household] had to evacuate from your town or city to a safe place at least 50 miles away, would you need assistance evacuating or sheltering your pets?”
|
Four respondents answered that they would be most likely to stay “With relatives or friends” in response to this question. One respondent marked “In a public shelter” and one respondent answered “In a room in a hotel or motel.” While respondents were able to choose an appropriate answer, one respondent commented that the list of choices was very long and another respondent asked that the list be reread to her.
Recommendation: We suggest that the sponsor consider shortening the list of response categories.
Sponsor’s Feedback: The sponsor accepted this recommendation and shortened the list of response categories. The response options now read: “With relatives or friends; In a public shelter; In a room in a hotel or motel; In a travel trailer or RV; Somewhere else, specify.”
(FR instruction: IF Q1 IS 1—ONE PERSON, SKIP TO Q.13) |
|
Four respondents answered “no” to this question, one respondent did not know a response and one respondent answered “yes.”
The phrase “agreed-upon meeting point” is not clear in this question. When probed, respondents mentioned several possibilities of what this phrase could mean. One respondent who answered “yes” identified her family’s meeting point as her mother’s house, 30 miles away. She mentioned that she could identify a local meeting point as well, but they had not done that. Another respondent conflated an agreed-upon meeting point with a communication plan. In response to probing, he mentioned the possibilities of a physical meeting place, a phone plan, a contact procedure, or a designated emergency contact. In this case, the respondent anticipated Question 12, suggesting that a sequence change might be helpful.
Recommendation: We recommend that Question 11 and Question 12 be reversed. We also recommend that the sponsor consider broadening the scope of Question 11 to include situations where household members get separated for any reason, not just for evacuation purposes. It is highly likely that their communication plan would be utilized in a wide variety of situations.
Sponsor’s Feedback: The sponsor accepted this recommendation and reversed the order of Questions 11 and 12.
|
Five respondents answered “No” to this question and one respondent answered “Yes.”
The phrase “alternative communication plan” is not clear to respondents. Although they did not have any type of communication plan, respondents often mentioned the use of landlines as the most obvious alternative to cell phones. The respondent who answered “Yes” explained that the plan was for everyone to contact her mother. However, the plan did not include any means of communication. One respondent commented that she did not know what the question was asking, further suggesting comprehension issues.
Recommendation: We suggest that the sponsor define “alternative communication plan” more clearly. One point of clarification should include whether the plan is just having a contact person or a contact person and the means by which to contact the person.
Sponsor’s Feedback: The sponsor accepted the suggestion and changed the question to read: “Do the members of your household have a plan for communicating in the event that cell phone service is disrupted?”
|
Five respondents answered “Yes” and one respondent answered “No.”
Respondents generally understood this question and did not have difficulty answering. Respondents understood the question to be asking about non-perishable foods and mentioned items such as rice, oatmeal, grits, canned vegetables, cereal, beans, tuna, and soup. A potential issue with this question is that some of the items mentioned need to be cooked before being consumed while others would not.
Recommendation: If the sponsor intends to include items that need to be cooked as well as items that do not, we do not recommend any changes to this question.
Sponsor’s Feedback: The sponsor clarified the wording in this question. The new question reads: “[Do you/Does your household] have available non-perishable food to feed [yourself/each family member] for 3 days?”
|
Two respondents answered “Yes” to this question and four respondents answered “No.” Respondents did not have any difficulty understanding or responding to this question.
Recommendation: We do not recommend any changes to this question.
|
Four respondents answered “no” to this question, one said “yes,” and one said “don’t know.”
This question was problematic for respondents who live in multi-unit buildings who were not sure if the question referred to their own personal generators or generators belonging to the building. Two respondents with this type of situation answered the question inconsistently. Both respondents live in multi-unit buildings with generators. The respondent who marked “Yes” explained that the building has a generator that provides limited lighting and HVAC. The respondent who marked “No” replied, “Our household doesn’t but the condo building does.”
Recommendation: We recommend that a fill be added to the question for addresses in multi-unit households. The question would read: “[Not counting generators provided by your building,] [Do you/Does your household] have a generator to provide electricity in case there is a power outage?”
Sponsor’s Feedback: The sponsor added a filter to this question that resolves the problem with multiple unit buildings. This question is only asked of those who live in singe unit attached or single unit detached houses.
(FR instruction: ASK ONLY IF Q3 IS 1—OWNED) |
(FR instruction: Read categories)
|
Three of the respondents who own their homes were asked this question. One respondent could not answer the question because she felt it was too complex. This respondent asked the interviewer to reread the question and explained that it was too much to think about what might be covered by her homeowners insurance due to a natural disaster. This respondent also suggested that the question had too many qualifications and was hard to understand. Another respondent in a multi-unit building felt that the response categories did not apply to her. This could be due to the sequence of the response options that sets the context for a single-family home in the first option.
Recommendation: To resolve these issues, we recommend that the sponsor simplify the question and reverse the first and second response options. The question would read: “If you lost your home in a natural disaster and it was not covered by insurance, what is your most likely long-term housing option?
Purchase a new or existing home
Rebuild home on existing site
Rent housing
Some other option; specify”
Sponsor’s Feedback: The sponsor decided to drop this question.
Attachment A: 2013 American Housing Survey
Emergency and Disaster Preparedness Module
Round One Questionnaire
INTRO 1: Now I will ask you some questions about your home and the people who live here. |
|
|
(FR instructions: Read categories until “yes” reply is received)
|
INTRO 2: The next series of questions are to determine your preparedness for emergency situations, such as a fire, and for large-scale disaster situations, such as a flood or earthquake. |
|
|
|
|
|
|
|
(FR instruction: IF Q1 IS 1—ONE PERSON, SKIP TO Q.13)
|
|
|
|
|
|
(FR instruction: ASK ONLY IF Q3 IS 1—OWNED)
|
(FR instruction: Read categories)
|
Attachment B: 2013 American Housing Survey
Emergency and Disaster Preparedness Module
Final Recommendations
INTRO 1: Now I will ask you some questions about your home and the people who live here. |
|
|
(FR instructions: Read categories until “yes” reply is received)
|
INTRO 2: The next series of questions are to determine your preparedness for emergency situations, such as a fire, and for large-scale disaster situations, such as a flood or earthquake. |
4. Is your building number clearly readable from the street, even at night, such as with reflective numbers posted on the building structure or on the mailbox?
|
5. [Not counting generators provided by your building,][do you | does your household] have a generator to provide electricity in case there is a power outage?
|
6. [Do you | Does your household] have available dry food goods to feed [yourself | each member] for 3 days?
|
7. [Do you | Does your household] have available at least 3 gallons or 24 bottles of water [per person]? (FR instruction: Do not read ‘per person’ if only 1 person in household)
|
8. Do the members of your household have an alternative communication plan, such as a designated emergency contact, in the event that cell phone service is disrupted?
|
(FR instruction: IF Q1 IS 1—ONE PERSON, SKIP TO Q.10) |
9. In some disasters, household members will need to evacuate separately. Does your household have an agreed-upon meeting point if that should happen?
|
10. [Have you | Has your household] prepared any kind of kit with necessary supplies to take with you if you have to evacuate your home?
|
11. [Would you | Would your household] have access to your vital financial information and contact numbers if you had to evacuate your home, either in paper documents or through a smartphone?”
|
12. If [you | your household] had to evacuate from your town or city to a safe place at least 50 miles away, do you have the financial resources, in terms of savings or available credit card balances, to meet expenses of up to $2,000 during an emergency evacuation?
|
13. If [you | your household] had to evacuate from your town or city to a safe place at least 50 miles away,] do you have enough reliable vehicles to carry all of your household members, pets, and a small amount of supplies such as clothes and food?
|
14. If [you | your household] had to evacuate from your town or city to a safe place at least 50 miles away, would you need assistance in evacuating or sheltering your pets, if you have any?
|
15. If [you | your household] had to evacuate from your town or city to a safe place at least 50 miles away for two weeks, where would you be most likely to stay during those two weeks?(FR instruction: Read categories)
|
(FR instruction: ASK ONLY IF Q3 IS 1—OWNED)
|
16. If you lost your home in a natural disaster and it was not covered by insurance, what is your most likely long-term plan? (FR instruction: Read categories)
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | freid002 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |