U.S. Department of Commerce
National Institute of Standards and Technology
OMB Control #0693-0078
Expiration Date: 07/ 31/ 2019
NIST Generic Clearance for Community Resilience Data Collections
1. Explain who will be surveyed and why the group is appropriate to survey.
This information collection will be focused on communities that have used or are currently using the NIST Community Resilience Planning Guide for Buildings and Infrastructure Systems (NIST 2016). Respondents are identified through their ongoing relationship with NIST’s Community Resilience Program (CRP) or through communities self-identifying as users of NIST guidance. The CRP works with community representatives to provide support for resilience planning activities and other uses of NIST guidance on community resilience. In the course of this work, NIST has developed strong relationships with communities that are engaged in resilience activities. As a result, the members of this research team have awareness of where the NIST Community Resilience Planning Guide is already being used. This knowledge will allow respondents to be identified and selected based on their role as community champions or conveners of the implementation of the NIST Community Resilience Planning Guide. The existing, primary contact(s) will serve as a starting point to request names and contact information of additional representatives from the collaborative planning team, as needed to reach the appropriate 3-7 representatives per community.
Respondents will be community champions or conveners of the implementation of the NIST Community Resilience Planning Guide. For example, respondents may be mayors, emergency managers, Voluntary Organizations Active in Disaster (VOAD) coordinators, university extension agents, Chamber of Commerce staff, public health workers, and school administrators. This group of respondents are considered early adopters of the NIST Guide and as such, are best positioned to provide important feedback on the practical application of the guidance provided by NIST. The information collected from representatives of these communities is essential for program learning to make improvements for Version 2.0 and ultimately, to provide better support for others seeking to use the Guide.
The NIST PI and Co-PI will be responsible for determining eligibility of respondents during team meetings on the subject. It is necessary that respondents are aware of the NIST Guide and experienced part or all of the NIST Guide process within the community they represent for completion of the survey. By selecting respondents from the collaborative planning team in each community, we strive to ensure these criteria of experience and familiarity with the process are met. In addition to receiving feedback from respondents that follow the guide steps in the manner articulated in the Guide (traditional users), NIST seeks to understand and receive feedback from respondents that have adapted the concepts or processes articulated in the Guide for their own unique applications (non-traditional users). A systematic assessment of NIST Community Resilience Planning Guide implementation will provide a “vital source of information for interpreting results and increasing the power and relevance of an outcome evaluation” (W.K. Kellogg Foundation 2004). A subsequent outcome evaluation can be used to provide NIST information about the impact of the Guide on communities. However, at this point in the NIST Community Resilience Program, understanding why a resilience planning effort is successful is far more important than just knowing its results.
2. Explain how the survey was developed including consultation with interested parties, pretesting, and responses to suggestions for improvement.
The survey was developed in 2017 by NIST research scientists Dillard, Helgeson, and Cauffman to align questions about process to each step of the NIST Guide. The research team at NIST reviewed the survey and refined the questions and then developed an accompanying script. The survey questions have been developed for a formative evaluation of the process phase of the project. In a formative evaluation, the intent is to generate information to guide program improvement (NOAA CSC 2013). Examples of process evaluations for other federal programs were used to inform the development of these survey questions (e.g., CDC 2004). Survey questions align to each of six steps of the NIST Community Resilience Planning Guide as well as a general background module. Phase 1 of the study includes modules through Step 2. Phase 2 will consist of modules for Steps 3-6.
The survey (including script) was then shared with the following sets of reviewers:
Internal NIST colleagues in Community Resilience Program;
External survey researchers in social science disciplines;
External community engagement experts;
And, the NIST Human Subjects Protection Office.
Based upon edits and comments provided by the reviewers, adjustments to question wording, answer options, and question order were made. Additionally, edits to consent language and script were also made.
3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.
The information will be collected by NIST in accordance with the methodology set forth below.
NIST anticipates that no more than 50 respondents per year will be invited to participate in the survey. For each community that has identified itself to NIST, investigators will attempt to recruit between 3-7 representatives. If a respondent elects to take part in this study, he/she will be asked to complete between one and three surveys online at his/her computer, on paper, or over the phone. Participation will require approximately 15 minutes per survey. The surveys will be administered over several months’ time, in relation to the completion of the steps (1-2) of the NIST Guide. The maximum burden per respondent over the next 12 months will be 45 minutes.
Total Burden/12 months (Est): 50 respondents * 45 minutes = 37.5 hours
Communities presently using the Guide have been self-identified through contact with NIST. Respondents are identified through their ongoing relationship with NIST’s Community Resilience Program (CRP). The CRP works with community representatives to provide support for resilience planning activities and other uses of NIST guidance on community resilience. In the course of this work, NIST has developed strong relationships with communities that are engaged in resilience activities. As a result, the members of this research team have awareness of where the NIST Community Resilience Planning Guide is already being used. This knowledge will allow respondents to be identified and selected based on their role as community champions or conveners of the implementation of the NIST Community Resilience Planning Guide. Using the existing primary contacts as a starting point, the NIST PI and Co-PI will request the names and contact information for additional representatives from the collaborative planning team, as needed to reach the appropriate 3-7 representatives per community. Future community representatives will be identified in similar ways. In this manner, a purposive sampling strategy will be employed. Purposive sampling is non-probability sampling where respondents are selected for a study based on specified characteristics based on the study objective(s) (Salant and Dillman 1994).
The information will be collected by NIST in accordance with the methodology set forth below.
Through existing contact with community representatives and/or through the resilience planning process convener or via self-identification, the NIST PI and Co-PI will identify representatives from the collaborative planning team. NIST will invite participation in this study through an email invitation.
If respondent answers affirmatively, a call will be scheduled for NIST researchers to review the consent form with the potential respondent.
Upon consent (i.e., the potential respondent has agreed to participate and has signed and returned the consent form to NIST), NIST researchers will initiate the survey by sending an initial link and a unique identification code that has been generated for the individual respondent. The respondent will be classified according to how the community he/she represents is using the NIST Guide. This classification will determine the survey modules the respondent will be asked to complete.
Non-traditional user – respondents from these communities are using the NIST Guide as a reference to support or assess another resilience effort (e.g., to evaluate existing plans, incorporate resilience into other capital and social plans, or to prioritize investments).
Traditional user - respondents from these communities are using the NIST Guide for resilience planning and intend to work through the six step process presented in the NIST Guide.
Upon successful recruitment, consent, and classification, the respondent will receive an initial, background survey module that asks questions about the respondent and their community.
Non-traditional user – If this describes the way in which the respondent’s community is using the NIST Guide, we will then a brief set of questions to better understand how the community is using the NIST Guide to assist in other planning activities before terminating the survey.
Traditional user - If this describes the way in which the respondent’s community is using the NIST Guide, we will then begin sending survey modules per each completed step of the NIST Guide (e.g., Step 1 Survey Module, Step 2 Survey Module, etc.).
Information on each community classified as a traditional user will be collected at regular intervals during the guide implementation process. NIST researchers will contact respondents quarterly to assess which step(s) from the NIST Guide have been completed in the overall process. These contacts will determine which survey module is to be completed.
Once Step 6 Survey Module has been completed, the survey will be terminated.
For this study, NIST will collect data on background of respondent and community, strengths in resilience planning process, barriers to implementing activities, match of project goals to target population needs, match between available resources and project activities, perceptions of the NIST Guide and related resources for resilience planning, systemic change in the community’s planning process, and experiences with each of the six steps of resilience planning. The source of these data are representatives of the communities who have used or are currently using the NIST Guide. The survey instruments will allow systematic measurement of the same information across representatives and communities.
A range of survey modes are available in this study. The information will be collected by using the most efficient and effective means for the community and its representative(s). The strongest preference from the NIST researchers’ perspective will be toward completion of the survey via web. However, we will make the survey available by the following additional modes: face-to-face and telephone. This multi-modal approach will be used to minimize respondent burden.
NIST researchers will work to maintain the response rate through quarterly email and/or phone contact with respondents. These contacts function as a point at which researchers determine whether any additional NIST Guide steps have been completed. Because consent is an ongoing process, researchers will remind study participants of the consent during quarterly check ins.
4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.
The results of the survey will be entered into a database and standardized to the extent possible. This process may involve coding open ended responses using qualitative techniques. The standardized quantitative responses will then be analyzed using descriptive statistics. Any remaining open ended, qualitative responses will be analyzed by examining the codes to identify common themes that emerge in the data.
Survey results are not expected to be immediately generalizable to a broader population of communities. The level of analysis will not extend beyond basic descriptive statistics until the sample is heterogeneous and large enough to allow for statistical inferences. This will only occur over time with the growth of the NIST Community Resilience Program and expanded use of the NIST Guide by diverse communities. In the short term, generalizable results are not the primary aim. Instead, the aim is to collect in-depth information from the communities using the Guide in order to inform the next generation of guidance on community resilience planning including Version 2.0 of the NIST Guide and accompanying tool development. The results from the sample of communities currently using the Guide were presented in a report. This initial report of the findings was released October 2018 (https://nvlpubs.nist.gov/nistpubs/ir/2018/NIST.IR.8231.pdf) . This data collection is intended to be ongoing for the communities who are presently identified and in the process of NIST Guide implementation. Furthermore, NIST expects to identify new communities for which we would collect data. These data will be analyzed for internal planning processes to inform NIST decisions on future priorities and research projects within the Community Resilience Program, which may include updates to the Community Resilience Planning Guide and implementation.
References
[NIST] National Institute of Standards and Technology. 2016. Community Resilience Planning Guide for Buildings and Infrastructure Systems. 1190-1Congress, 1:114.
W.K. Kellogg Foundation. 2004. Evaluation Handbook. Battle Creek, MI: W.K. Kellogg Foundation.
[NOAA CSC] National Oceanic and Atmospheric Administration Coastal Services Center. 2013. Planning for Meaningful Evaluation. Charleston, SC: NOAA Coastal Services Center.
[CDC] Centers for Disease Control and Prevention. 2009. "Developing Process Evaluation Questions." Pp. 2, Vol. 4, Evaluation Briefs. Atlanta, GA: Department of Health and Human Services/Centers for Disease Control and Prevention.
Salant, Priscilla and Don A. Dillman. 1994. How to Conduct Your Own Survey. New York, NY: John Wiley and Sons, Inc.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Dillard, Maria (Fed) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |