0648-0762 SUPPORTING STATEMENT Part B

SUPPORTING STATEMENT Part B

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

3D Nation Elevation Requirements and Benefits Study

OMB Control No. 0648-0762

SUPPORTING STATEMENT PART B - (Questions and guidance for Responses)

Agencies are instructed to complete Supporting Statement Part B if they are using statistical methods, such as sampling, imputation, or other statistical estimation techniques; most research collections or program evaluations should also complete Part B. If an agency is planning to conduct a sample survey as part of its information collection, Part B of the ICR supporting statement must be completed, and an agency should also complete relevant portions of Part B when conducting a census survey (collections that are sent to the entire universe or population under study). For example, an agency doing a census of a small, well-defined population may not need to describe sampling procedures requested in Part B, but it should address what pretesting has taken place, what its data collection procedures are, how it will maximize response rates, and how it will deal with missing unit and item data.

Agencies conducting qualitative research studies or program evaluations, including case studies or focus groups, should also complete the relevant sections of Part B to provide a more complete description of the use of the information and the methods for collecting the information.

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The sample for this collection consists largely of day-to-day professional users of elevation data (topographic, bathymetric) and activities/industries that rely on elevation data. The survey data will be collected electronically through an online questionnaire. Random sampling techniques are not being used for this study because the study respondents need to have technical knowledge of elevation data acquisition requirements and be familiar with the uses and benefits of elevation data.

Respondent Universe

The respondent population for the online survey consists of a pre-defined list of federal, state, local, and tribal governmental agencies, academia, industry and non-governmental entities known to use elevation data. NOAA and USGS geospatial liaisons will work with their state, local, and tribal counterparts to provide a list of organizations using elevation data. Although we estimate a conservative response rate of 60%, for purposes of this study, up to 1000 people could be selected by their organizations to complete the online survey, (see Table 1 below). Each organization is contacted via email (or other means as needed) and asked to provide a respondent(s) to 1) agree to participate and 2) receive the instructions for completing the survey. In both the 2012 National Enhanced Elevation Assessment (the model for the 3D Nation Study) and the 2018 3D Nation Study, some federal agencies and state governments selected more than one respondent to cover their elevation data requirements, typically by work area, but private sector and non-profit organizations mostly identified one individual. Average number of respondents for Table 1 below were derived from these two earlier studies.

For the 2018 3D Nation study survey, usable responses were received from 824 respondents compared to the target of 1000, resulting in a 82% response rate for the whole collection. The study team found that email worked well as a communication tool. Respondents agreeing to participate were offered a link to the survey as well as an invitation to an optional informational webinar. The team did receive questions that contributed to the Frequently Asked Questions file, and a few suggestions for other potential participants.

Going forward, the study team anticipates at minimum a response rate of at least 60% for the online survey (yielding 600 actual respondents). The response rate from the 2018 collection set a good bar but we do not know yet if this is repeatable until the study is re-administered. We conservatively forecast a lower response rate because some entities who responded to the first questionnaire may have no changes to responses and assume their data is included from the first study. The estimate of a minimum 60% is based on expected responses from federal, state, U.S. territory, tribal government and academic responses, all of which benefit from working through an identified POC who commits to gathering and reporting input. Responses from private sector and non-profit organizations are less of a given.

Table 1. Organizations in respondent universe for Survey and Interviews

Organizations (Population Groups)	Number of Organizations (Population) (a)	Average Number of Respondents Identified by Each Organization (b)	Total Sample Size (c)	Expected Number of Respondents (d)=(c)×60%
Federal agencies	60	5	300	180
U.S. States	50	8	400	240
U.S. Territories	5	2	10	6
Tribal Governments	10	2	20	12
Academia	25	3	75	45
Private Sector/Industry	75	1	75	45
Not-Governmental Organizations	60	2	120	72
Total			1000	600

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

This collection is intended to occur every 5-7 years to reduce burden.

The respondent sample consists of known elevation data users based upon a pre-defined list of organizations invited to participate (n=1000). Representatives from federal, state, local, and tribal government organizations, academia, industry and non-governmental entities will be contacted and asked to participate in this study. The sample is selected from each stratum as described below. Elevation user organizations are known by NOAA and USGS as contacts and through working relationships.

Random sampling techniques are not being used for this study because the study respondents need to have technical knowledge of elevation data acquisition requirements and be familiar with the uses and benefits of elevation data. Study participants are identified as follows.

• Each Federal agency has a Point of Contact (POC) nominated by their agency as someone with broad understanding of agency programs and their use(s) of elevation data. The POCs then identify participants within their agency to respond to the questionnaire and participate in the validation process. They also recommend participants from non-governmental and private organizations. The Federal POCs will also review the agency’s initial responses to the questionnaire and sign off on the final validated agency study input.

• Each state and U.S. territory has a State Champion selected in consultation with regional experts as someone with broad knowledge of elevation activities within their state. Similar to the Federal POCs, the State Champions are identifying study participants within their state to respond to the questionnaire and participate in the validation process. The State Champions will also review the state’s initial responses to the questionnaire and sign off on the final validated statewide study input.

• The 3D Nation Elevation Requirements and Benefits Study has identified tribal needs as an important stakeholder group for improving understanding of 3D elevation data requirements for the nation. State Champions are best positioned to help identify tribes that are active in the elevation data community and have the background needed to assess their tribe’s elevation data needs. Additionally, the 3D Nation Study will have a representative from the Bureau of Indian Affairs (BIA) responding to the study. We will also ask BIA and other agencies’ tribal liaisons to recommend tribal associations and groups that might respond from the collective perspective, such as the National Congress of American Indians and the National Tribal Land Association.

• Private sector/industry, academia, and non-governmental respondents are solicited by sending a request for participation to a broad selection of professional organizations, trade associations, councils, and other groups that represent sectors/industries known to use elevation data. Each organization, association, etc. is provided a link to the questionnaire and requested to invite its members to participate in the study.

The data collected during the online survey study is coded directly into an on-premises computerized database. Data analysis consists of descriptive summary statistics – sum, maximum, minimum, mean, median. Descriptive statistics are used to describe current uses within the programs of the organizations represented within the sample. Because NOAA and USGS are interested in determining ways to improve the availability of elevation data, it is important to gather and update baseline information concerning current requirements and to determine any information gaps or unmet needs.

USGS and NOAA want to understand the elevation data requirements and benefits of the broadest possible spectrum of elevation data users. Responses from all Federal agencies, states and territories are essential to the study, as well as every academic/private sector/non-profit organization willing to respond. If it is determined that certain known sectors/industries that rely on elevation data to conduct their business are not represented in the responses, additional outreach will be conducted to solicit participation in the study. This may take the form of additional email request(s) to respond to the questionnaire or individual interviews. In terms of accuracy of response, the study team reviews responses to ensure they are complete and free of conflict (e.g. basing requirements on one collection standard but stating another collection standard is needed). The team follows up with individual respondents during the validation process to fill any identified gaps or confusing responses.

The monetary benefits being collected in this study are attributed to a given Mission Critical Activity and the geographic Area of Interest and requirements for 3D elevation data needed to accomplish the activity. It should be noted that study participants are free to answer any of the benefits questions as “Unknown” or “Unable to provide.” The benefits questions also include a place to describe the benefits in narrative form as well as how the value was derived. During the validation process, organization stakeholders will be asked to verify that the benefits they provided are reasonable.

In the follow-on analyses of the questionnaire results, the benefits dollars will be apportioned spatially across the Mission Critical Activity Area of interest (e.g. dollars per square mile) and compared to costs per square mile for data collection. If a study participant identifies requirements that will not be met by a program scenario that is being evaluated, the benefits dollars will be reduced to account for the program not meeting all of the requirements for an activity. For example, if a participant identified $1 million in benefits for having Quality Level 2 data updated annually and the program scenario is for Quality Level 2 data delivered every five years, then the benefits would be assumed to be reduced proportionally.

In the study report(s), data aggregation will be done by groupings that are defined in the study questionnaire, such as by Business Use (e.g. Water Supply and Quality), organization type (e.g. Federal, State, etc.), geographic area type (e.g. inland, nearshore, offshore), or data type (e.g. topography, bathymetry). No extrapolation will be made from one type of study participant to a larger group. One organization type’s response (e.g., state, county, Tribe, agency, etc.) will not be assumed to apply to any other similar organization type.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Several steps will be taken to maximize response rate and ensure an accurate and reliable sample. We predict that the response rate of at least 60%, if not higher based on past experience, will be met due to past experience, the highly technical nature of the respondents, support of the NOAA and USGS geospatial liaisons, and contractor follow-up.

We are using this web-based survey as a strategy to decrease costs, increase the speed of data collection, increase response rates by providing additional modes for response, and decrease the amount of non-response error. The Tailored Design Method for mail and internet surveys will be followed to help ensure a high response rate and responses from a broad spectrum of agencies and sectors/industries known to use elevation data. As a part of this process we will:

1. Request participation in advance.

2. Use the questionnaire introduction to share:

   1. the purpose of the survey

   2. the reason for participation

   3. the terms of anonymity and how the results will be used.

3. Allow enough time to complete the survey, with ability to begin the survey and return at a later time if needed.

4. Provide clear survey instructions for each section, explaining how to navigate through and submit the survey.

5. Provide a survey that is easy to follow with clear and direct questions/instructions:

6. Send reminders during the survey period for those that have not completed the survey.

7. Review responses to ensure they are complete.

8. Follow up with individual questions during the validation process to fill any identified gaps or confusing responses.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

For the first 3D Nation Study, NOAA and USGS asked nine federal agency colleagues to complete the questionnaire, in order to troubleshoot issues in advance of broader dissemination. Their feedback was extremely helpful to improve the survey instrument. And as detailed in Part A, Question 8 of this Statement, the questions for this collection were subject to extensive internal NOAA and USGS review, as well as reviews by external partners, including the 3DEP Working Group, the Interagency Working Group on Ocean and Coastal Mapping, representatives from associations such as the American Society for Photogrammetry and Remote Sensing, the National States Geographic Information Council and the Association of American State Geologists. The review and pre-testing of the questionnaire were performed to gather comments concerning the overall structure, sequence and clarity of questions. Individuals were also asked to estimate the time burden of the survey. Comments and suggestions provided by reviewers and pre-test participants were evaluated and used to revise the survey instrument where appropriate. Comments that improved clarity and comprehension of content were also incorporated.

Going forward, the study team will again use a federal test group before initiating the wider survey. Because we have a baseline set of data in hand from the first 3D Nation Study, the team will use feedback and lessons learned from the first 3D Nation Study to clarify any frequently asked questions and streamline the survey to make responding to it even easier. These technical enhancements to streamline the survey will be made when updating the online survey tool.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Ashley Chappell (federal), [email protected], 240.429.0293

Allyson Jason (federal), [email protected], 703.648.4572

Cathleen Yung (contractor), [email protected]