Supporting Statement

OMB Control #0693-0078

Expiration Date: 07/31/2022

NIST Generic Clearance for Community Resilience Data Collections

HURRICANE MARIA EMERGENCY COMMUNICATIONS INVESTIGATION:

PILOT HOUSEHOLD STUDY

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

The National Institute of Standards and Technology (NIST) has the responsibility to investigate cases of serious failures of the built environment (buildings and infrastructure) under the National Construction Safety Team Act, signed into law in 2002¹. Members of the National Construction Safety Team (NCST) based at NIST were tasked in early 2018 with the investigation of impacts from Hurricane Maria, which hit Puerto Rico on September 20, 2017. NCST duties include:

1. “...establish the likely technical cause or causes of the building failure;

2. evaluate the technical aspects of evacuation and emergency response procedures;

3. recommend, as necessary, improvements to building standards, codes, and practices based on the findings;

4. recommend any research and other appropriate actions needed to improve the structural safety of buildings, and improve evacuation and emergency response procedures, based on the findings of the investigation.”

NIST is conducting multiple projects that represent all of these areas for investigation. However, the survey for which this application is written is a pilot of a larger future survey, and pertains to the second and fourth duties of the NCST listed above. The goal of this specific portion of the investigation is to evaluate the effectiveness of emergency communication as well as behavioral aspects of evacuation response across the population of Puerto Rico. This investigation is not considered research, but is instead a fact-finding mission to establish the role that emergency communication played in effective or ineffective evacuation behavior. The purpose of NCST investigative activities is to make recommendations, based directly on findings, that can help prevent future deaths and injuries across the United States.

To complete this investigation, NIST has partnered with contractors who can utilize trained personnel local to Puerto Rico to complete data collection activities. The contractors, in consultation with NIST scientists and engineers, have used U.S. Census data to develop a sampling strategy for the final survey (with 1,500 respondents planned) whereby data collection will occur in four specific regions of Puerto Rico that facilitate representation of key hurricane-impacted areas, but is still representative of the island’s overall population. This pilot survey (with 20 respondents planned) will be a test of the instrument used for the larger survey, and will also enable interviewers to gather feedback from respondents on the survey questions. The household survey instrument will elicit data on emergency communication systems, messages, factors influencing evacuation behavior, as well as actual evacuation behavior on the island of Puerto Rico for Hurricane Maria. This survey is also a key component among two related, future data collection activities including: interviews with providers of emergency communications messages, as well as interviews with members of the public to assess more qualitative aspects of emergency preparedness and response.

While the final survey will sample households across Puerto Rico, the pilot survey will be conducted with 20 adult colleagues of contractors in Puerto Rico contacted via a snowball sampling methodology. Snowball sampling is a non-probability sampling method, whereby recruiting is conducted by NIST contractors, and then having initial participants recommend other individuals in their acquaintance to participate. In this way, participants who are a suitable fit (having experience in Hurricane Maria, familiarity with Puerto Rico and local emergency procedures, and reflecting relevant diversity of population characteristics) can be efficiently identified utilizing existing social networks. The pilot survey will provide invaluable local expertise and will help to evaluate clarity of language and terminology, and ensure that questions are appropriate, relevant, and effective in meeting their objective. With feedback from respondents and analysis of pilot data from the contractors, NIST will be best positioned to meet its NCST duties with an excellent final survey instrument.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The pilot survey instrument was developed by NIST scientists with backgrounds in sociology, anthropology, communications, and psychology through a series of development and quality control activities. First, a review of previous hurricane literature and survey instruments was conducted to better understand the types of factors that influence public response during these types of disaster events and develop the basis of our survey. When possible, established scales and items were used from previous hurricane and/or evacuation and protective action research.

Next, after a draft questionnaire was developed, NIST investigators elicited informal feedback from numerous content experts in hurricane forecasting, emergency communication, and evacuation behavior from experts at The National Oceanic and Atmospheric Administration (NOAA), as well as professors from the University of Puerto Rico and other academic colleagues across the U.S.. Additional revisions were made that focused on the content of the survey questions (i.e. which concepts were measured and included), and the instrument was condensed to reduce its length.

Then, NIST researchers disseminated the revised questions to a few survey experts for final refinement. These included experts here at NIST, as well as project contractors. A final round of revisions was made focusing on details of the content, including issues with item clarity, question ordering, and consistency in wording. The contents of this application, the deployment of the pilot survey, will then form the last phase of instrument testing and refinement, prior to future revisions, approvals, and full field implementation.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

Professional colleagues will be selected by members of our contract team located in Puerto Rico to complete the pilot survey. These individuals will be selected to best represent the four regions within Puerto Rico that NIST is focusing on for its Hurricane Maria investigation and research activities. At least 5 contractors will gather feedback from these interviewees and then use their recommendations to engage additional participants until 20 responses are collected, with roughly equal distribution across the four geographic areas of focus within Puerto Rico. The contractors anticipate an acceptance rate of 50% (a conservative estimate given that respondents will be known to one another) so each of the 5 contractors will only need to contact a few people in order to reach the desired number of 20 complete responses. Via this snowball sampling data collection approach, we anticipate that we will be able to gather enough feedback to ensure the utility and validity of the survey instrument.

For pilot test feedback, individuals will be contacted via phone by members of the project team residing in Puerto Rico to request participation. If individuals agree to provide us with their feedback, they will conduct the survey at that point, or be called at an agreed-upon time and read the survey aloud by contractor interviewers. At the time of survey completion, participant’s responses will be recorded using the CATI phone recording system. After completing the survey over the phone (see attached paper survey for what will be read by interviewers to respondents) interviewers will ask the respondents for their comments or suggestions, such as identifying items that were unclear, confusing, or overly repetitive. In addition, interviewers will keep records on any problems or issues they had in deploying survey questions. This qualitative data will be collected in addition to the survey responses, and submitted to NIST by the contractor. Responses will be kept anonymous and will not be associated with name or other personally identifiable information. This is not a Privacy Act System of Records, therefore SORN and Privacy Act Statement are not applicable.

Burden hour estimates for the pilot study will be:

Number of Respondents: 20

Minutes/Hours per Response: 45 minutes

Burden: 15 hours

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

There is no intention of using the pilot survey data to make formal conclusions or generalize findings to the entire population of Puerto Rico. Instead, the intent of the pilot survey is to refine the survey instrument to be submitted to PRA for use in the full data collection. So, while results from the pilot will not be generalizable, because we intend the final version of the survey to be so, we are still aiming for results of the pilot to be somewhat representative of different aspects of geographic and demographic diversity across Puerto Rico. NIST interviewers will help guide interviewees in snowball sampling accordingly.

Qualitative feedback on the pilot survey, in addition to response data, will be consolidated by the contractor and delivered to NIST. NIST personnel will then investigate simple descriptive statistics of the data, review contractor and participant feedback, and make final decisions on any modifications necessary to the survey instrument to ensure that it is appropriate for the widest possible Puerto Rican audience. While the content of the survey has already been refined via expert feedback by those at NOAA, NIST, and several universities, it is anticipated that feedback from those living on Puerto Rico will provide valuable information related to language clarity, appropriate usage of terminology for our target population, and provide relevant local context - such as identifying potentially sensitive or taboo topics, or important factors we may have missed. After the pilot test, NIST researchers will finalize the survey instrument and submit a new PRA application, with updated documents, for the full field implementation in Puerto Rico.

The purpose of an NCST investigation is to help prevent future deaths and injuries across the U.S. by recommending actions that can influence codes, standards, and practices. Lessons learned from Puerto Rico regarding the use and effectiveness of emergency communications, and their influence on evacuation behaviors, can be useful not only to better understand the impacts from this particular storm, but also can also be relevant for other hurricane prone regions and in other hazard conditions. For example, lessons learned by NIST’s NCST investigation of the Joplin, M.O. tornado in 2006 have been applied to help standardize siren emergency communications across the U.S.. The entire NIST Hurricane Maria investigation team is hopeful that this kind of impact will also be possible based on the results of our household survey regarding emergency communications in Puerto Rico.

1 https://www.nist.gov/system/files/documents/public_affairs/releases/hr46871.pdf

3