MTMN SS_New 072511 Part B rev

SUPPORTING STATEMENT
MARIANAS TRENCH MARINE NATIONAL MONUMENT KNOWLEDGE AND
ATTITUDES SURVEY
OMB CONTROL NO. 0648-XXXX
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
(If your collection does not employ statistical methods, just say that and delete the following five
questions from the format.)
1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of entities
(e.g. establishments, State and local governmental units, households, or persons) in the
universe and the corresponding sample are to be provided in tabular form. The tabulation
must also include expected response rates for the collection as a whole. If the collection has
been conducted before, provide the actual response rate achieved.
The sampling plan for the Marianas Trench Marine National Monument Knowledge and
Attitudes Survey will be probability-based so that study findings can be used to make statistically
defensible inferences about the entire population of Guam and Mariana Islands
residents. Steps involved in the sample design are described below.
1.1

Target Population. The target population for this survey includes all household residents
of Guam,154,805 (Census 2000, Issued (revised) May 2004) and the Mariana Islands,
69,221 (Census 2000, Issued (revised) May 2003).

1.2
Survey Eligibility. This study will include only those respondents who are at least age 18
and who are the heads of their households.
1.3
Sampling Frame. Respondents will be selected using a dual frame sampling
methodology, one sampling frame of landline phones and a second sampling frame of cell
phones.
1.4 Statistical Methodology for Stratification and Sample Selection. The sample design and
stratification will be based on a dual frame sampling methodology in which respondents will be
selected using random digit dialing (RDD) for each frame and frame stratum (e.g., a random start
and a sampling interval equal to the universe stratum size divided by the desired stratum sample
size). In this way, each householder in each sampling frame and stratum will be given a known,
nonzero probability of selection so that weighted inferences can be made about the entire
population of Guam/Marianas residents. Chromy’s probability minimum replacement selection
methodology will be used for the sample selection process (Chromy, 1979).
The use of separate landline and cell phone samples will ensure that households that do not have
landline telephones are fully covered. Inaccuracies in the national list of phone numbers and/or
recent household transitions not covered in the national list of landline phone numbers are all
addressed using the dual frame, stratified approach.

1

In summary, respondents will be selected using random digit dialing (RDD) selection:
Sampling Frame 1: Landline numbers
1) all Guam/Mariana Islands landline phones (800)
Sampling Frame 2: Cell Phone numbers
2) all Guam/Mariana Islands cell phones (200).
RDD samples will be obtained from Survey Sampling, Inc. Assuming an eighty percent response
rate and a ninety percent eligibility rate, we plan to select approximately 1,389 respondents to
participate in this survey to achieve 1,000 complete responses of Guam/Marianas households. Cell
phone respondents will be asked if they have already participated in the survey via a landline to
prevent duplication.
Weighting
Data from both landline and cell phone samples will be combined and weighted. Household
weights will be weighted by householder age and race/ethnicity. The weighting will be based on
all respondents who were fully screened in the base study, and who did not terminate during the
screening process. The final sample dataset will include screened respondents who did not
participate in the survey, respondents who completed the interview, and respondents who chose
not to participate in the full interview or who only partially completed it. Weighting targets will
come from the Current Population Survey, 2010 Annual Social and Economic Supplement,
conducted by the U.S. Census Bureau and released in January 2011.
Population weights will be based on the subset of fully screened respondents in the base study
who were the initial randomly selected individuals. They will be weighted using a single-stage
weighting procedure by age, sex, and race/ethnicity to the 2010 population estimates from the
Current Population Survey, 2010 Annual Social and Economic Supplement, conducted by the
U.S. Census Bureau and released in January 2011. The household weight will be used to weight
the base study and to analyze the substantive results from the full interviews. Separately, a
population weight will be derived to estimate the Guam/Mariana Islands population in order to
compare the sample of the population in our survey to that of the population estimates provided
by the Census Bueau.
1.5
Response Rates. Our goal is to achieve an overall response rate of eighty percent of all
screened respondents. Our response rate estimates are based on recent experience with other
general population surveys. The surveys that we considered when determining a likely response
rate for this survey included those using the same data collection methodology (telephone survey
with follow-up of non-respondents) as well as similar subject matter and respondent burden
(questionnaire length).

2

Assuming an expected response rate of eighty percent, we would need to select approximately
1,389 sample cases to reach this goal (assuming a ninety percent eligibility rate). The proposed
survey will be using CATI with non-response telephone follow-up and will be used to maximize
the chance that the response rate goal is achieved.

1.6
Reliability of Estimates. With an overall respondent sample of 1,000, the 95-percent
confidence interval (CI) is expected to be less than ± 3.2 percent for generated percentages. For
example, assuming that the percentage of respondents who experience a significant burden is 50
percent, then using a 95 percent confidence interval, in 95 out of 100 samples like the one
selected, the results should be no more than 3.2 percentage points above or below this figure.
Another important research objective is to be able to make estimates for different types of
respondents, for example, Chamorro vs. other residents. Although the sample sizes will be
sufficient to make estimates for many national measures, multivariate models may be used for
detailed subgroup comparisons.
1.6
Estimation Procedures. Following data collection, sample weights will be prepared
based on the initial probability of selection and adjusted to 1) compensate for nonresponse, and
2) remove multiple selection opportunities. The end product will be final analysis weights
suitable for use in analysis. This weighting scheme inflates the respondents' data to represent the
entire universe of Guam and Mariana Islands households.
Data management and statistical analyses will be conducted using SAS. Various descriptive
statistical techniques will be used to analyze the data, including crosstabs and frequency
distribution, t-tests, chi-square tests, but multivariate methods may also be used. Direct variance
estimates that reflect the sample design will be computed for each analysis variable, and will be
used in all analytic comparisons of final results.
2. Describe the procedures for the collection, including: the statistical methodology for
stratification and sample selection; the estimation procedure; the degree of accuracy
needed for the purpose described in the justification; any unusual problems requiring
specialized sampling procedures; and any use of periodic (less frequent than annual) data
collection cycles to reduce burden.
The proposed survey is designed as a telephone survey using Computer Assisted Telephone
Interviewing (CATI) with non-response telephone follow-up. The use of CATI is advantageous
for several reasons; 1) it simplifies questionnaires by offering many logic options that dictate
how the interviewer proceeds through the questionnaire (e.g., the survey can skip irrelevant
questions, perform calculations, and modify questions based on the answers to earlier questions);
2) it can check the logical consistency of answers; and 3) it can present questions or answers
choices in a random order (thus limiting the potential bias of ordering effects, or the first
presented item having an impact on the outcome of following items). In addition, telephone is
the most efficient method of data collection due to the greater number of call attempts and thus a
greater likelihood of reaching someone and achieving a completed interview. CATI often yields
better information than mail, as we can determine whether we have an eligible respondent and/or

3

whether we need to make more calls. The maximum number of calls to one house would be
three.
We believe that, overall, CATI would yield the biggest response rate for this study, yielding a
potential response rate of over 80 percent for all eligible respondents. In addition, CATI will
improve the quality of the data by ensuring that the most knowledgeable respondent is located to
respond to the survey.
The data collection methodology will include:
•

Extensive validation of all phone numbers used in the sampling frame that will be used to
draw the sample;

•

A comprehensive review of phone numbers and potential phone number look-ups will
determine the quality of the contact information;

•

Tracing efforts using commercial locating databases and directory assistance will be done to
obtain either new addresses or updated phone numbers to locate non-respondents, resulting in
an update of any new information obtained from searches;

•

Development and programming of CATI codes for refusal, ineligible, and unlocatable cases;

•

Programming a survey management system to track completed cases, partially completed
cases, call history, and locating history;

•

Developing training programs and training interviewers;

•

The use of screening questions to locate the most knowledgeable respondent for the survey;

•

Monitoring response rates, analyzing completed cases by time of day and days of the week to
optimize calling times and performing refusal conversion calls.

The survey is designed as a one-time survey. No future contacts are planned after a completed
questionnaire is returned and/or the interview is completed by phone.
3. Describe the methods used to maximize response rates and to deal with nonresponse.
The accuracy and reliability of the information collected must be shown to be adequate for
the intended uses. For collections based on sampling, a special justification must be
provided if they will not yield "reliable" data that can be generalized to the universe
studied.
The methods described above have been shown in methodological research to be likely to yield
response rates of eighty percent when the survey is of reasonable length and on a salient topic.
Nonresponse will be addressed using standard best practices to maximize response rates. These
practices will include making sure to call back for numbers where the head of household is not
available at the time of the first call. There will also be an analysis to determine if there is a bias in

4

nonresponse of the population and methods will be changed accordingly if a bias is determined.
Research suggests that the following procedures will also encourage respondent participation:
•

Number of follow-up contacts;

•

Interviewer training to emphasize the difficulties in communicating with participants and ways
to overcome these difficulties;

•

A toll-free number at American Directions, Group Inc., and a Web link to answer
respondents’ questions. Calls to the toll-free number will be returned to address respondents’
concerns.

The most important of these factors is the number of contacts, including telephone follow-up calls.
Stating the sponsorship of the survey helps to engage recipients by providing immediate
assurance that the survey is legitimate and not an attempt to sell them something. The likelihood
of acceptance is greatly increased when respondents are told early why the survey is being
conducted and why their responses are important. A set of screener questions has been added to
the beginning of the questionnaire to identify the person as the most appropriate respondent.
Our data collection approach for this survey will employ all of the features described above.
4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as
effective means to refine collections, but if ten or more test respondents are involved OMB
must give prior approval.
We have tested the survey with a small (n=9) sample of diverse respondents in the Honolulu area
to ensure that the flow of questions and survey length are appropriate. The telephone data
collection procedures themselves have been fully tested and interviewers have been well-trained
in survey interviewing on numerous American Directions Group surveys. The interviewers
selected and trained to conduct interviews will be selected from the most experienced American
Directions Group interviewers.
5. Provide the name and telephone number of individuals consulted on the statistical
aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other
person(s) who will actually collect and/or analyze the information for the agency.
Statistical Consultant
Harley Heimovitz, PhD
Washington Analytics, LLC
704 Lanark Way
Silver Spring, MD 20901
301-593-6032 (Voice)
240-604-3695 (Mobile)
[email protected]

5

Contractor who will collect the data
Neal Cohen
American Directions Group
1350 Connecticut Ave., NW
Washington, DC 20036
(202) 223-4747 ext. 18
[email protected]

NOAA/NMFS Project Officer
Stewart Allen, PhD
NOAA Fisheries Pacific Islands Fisheries Science Center
1601 Kapiolani Blvd. Suite 1000
Honolulu, HI 96814
(808) 944-2186
[email protected]

6