A-11 Fast-Track Clearance Request Template_COVID Vaccine

Request for Approval under the “Generic Clearance for Improving Customer
Experience (OMB Circular A-11, Section 280 Implementation)” (OMB
Control Number: 2900-0876)
TITLE OF INFORMATION COLLECTION: COVID-19 Vaccine Survey
PURPOSE

The COVID-19 Vaccine Survey is designed to measure the customer experience
during the recruitment into the COVID-19 clinical trial and other VA Researches. VA
understands that COVID-19 has significantly impacted the medium, frequency, and
overall experience in which Veterans receive their healthcare. These surveys are
created to understand how Veterans, family members, friends and other volunteers are
recruited into clinical trial via various modalities such as newsletters, posters, social
media, targeted mails and emails, referral from a doctor or from a Veterans Service
Officer (VSO) etc. Participants’ experience data will be collected using an online
transactional survey disseminated via an invitation email sent to selected beneficiary or
via the VA.gov page questionnaires (for a Landing Page Survey). Surveys invites will be
sent twice per week. After the survey has been distributed, recipients have two weeks to
complete the survey. Invitees will receive a reminder email after one week.
There are four surveys targeting the recruitment for Covid-19 Vaccine trial and
other research.: (1) the first survey aims to understand the modality that lead the
participant to the COVID-19 research landing page; (2) the second survey aims to
measure the experience of participants from their interaction during the recruitment
phase; (3) the third survey aims to measure the experience of participants in scheduling
their appointment; (4) the fourth survey pertains to the Study Team who enrolls
volunteers at specific study sites.
The survey questionnaire is brief and contains general Likert-scale (a scale of 15 from Strongly Disagree to Strongly Agree) questions, checkbox and radio button
questions, to assess customer experience. A qualitative study was conducted to
analyze the question content and record reactions on a focus group consisting of
Veterans. The sample will be distributed across the participants population defined
further below.

DESCRIPTION OF RESPONDENTS:
Survey

How

When

Response

Population

Survey
1

Landing page
register volunteer
list (assumption survey will only be
sent to people who
registers on
va.gov)

Va.gov

Immediately
after
interaction

Not
Applicable

Everyone who registers on
VA.gov

Survey
2

VA interaction–
sign up into a
research/don’t sign
up into a research

EMDEmail

3 days after
interaction

2 weeks (
one reminder
after 7 days)

After volounteers register for
the study, they are contacted
with further information.
People that do not want to
move forward after being
called will receive Survey 2
(Exclude – ineligible and
cohort that move forward)

Survey
3

Appointment

EMDEmail

3 days after
interaction

2 weeks (
one reminder
after 7 days)

Anyone who made an
appointment

Survey
4

Study Team
Survey

EMDEmail

3 days after
interaction

2 weeks (
one reminder
after 7 days)

Study staff that interacted
with the participants for the
first Appointment.

TYPE OF COLLECTION: (Check one)
[ ] Customer Comment Card/Complaint Form
[ ] Usability Testing (e.g., Website or Software
[ ] Focus Group

[X] Customer Satisfaction Survey
[ ] Small Discussion Group
[ ] Other: ______________________

CERTIFICATION:
I certify the following to be true:
1. The collection is voluntary.
2. The collection is low-burden for respondents and low-cost for the Federal Government.
3. The collection is non-controversial and does not raise issues of concern to other federal
agencies.
2

4. Personally identifiable information (PII) is collected only to the extent necessary and is not
retained.
5. Information gathered is intended to be used for general service improvement and program
management purposes.
6. The collection is targeted to the solicitation of opinions from respondents who have
experience with the program or may have experience with the program in the future.
7. All or a subset of information may be released as part of A-11, Section 280 requirements on
performance.gov. Additionally, summaries of the data may be released to the public in
communications to Congress, the media and other releases disseminated by VEO, consistent
with the Information Quality Act.
Name: Dan Ostrow, Implementation Lead, Veterans Experience Office [email protected]
(414) 690-8587

To assist review, please provide answers to the following question:
Personally Identifiable Information:
1. Will this survey use individualized links, through which VA can identify particular
respondents even if they do not provide their name or other personally identifiable
information on the survey? [ X ] Yes [] No
2. Is personally identifiable information (PII) collected? [ ] Yes [X] No
3. If Yes, will any information that is collected be included in records that are subject to the
Privacy Act of 1974? [ ] Yes [ ] No [N/A]
4. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No
[N/A]
Gifts or Payments:
Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to
participants? [ ] Yes [ X] No

BURDEN HOURS
Category of Respondent
Landing Page Survey
VA interaction Survey
Appointment Survey
Study Team Survey
Totals

No. of
Respondents
4,200 Annual
4,200 Annual
4,200 Annual
2,400 Annual
15,000 Annual

Participation
Time
2 Minutes
1 Minute
1 Minute
1 Minute
5 Minutes

Burden
140 hours
70 hours
70 hours
40 hours
1250 hours

Please answer the following questions.

3

1. Are you conducting a focus group, a survey that does not employ random sampling,
user testing or any data collection method that does not employ statistical methods?
Yes ___
No _X_

If Yes, please answer questions 1a-1c, 2 and 3.
If No, please answer or attach supporting documentation that answers questions 2-8.
a. Please provide a description of how you plan to identify your potential group of
respondents and how you will select them.

b. How will you collect the information? (Check all that apply)
[ ] Web-based or other forms of Social Media
[X] Telephone
[ ] In-person
[ ] Mail
[ ] Other- E-mail-based surveys
c. Will interviewers or facilitators be used? [ ] Yes [ X ] No
2. Please provide an estimated annual cost to the Federal government to conduct this data
collection: __$13,000______
3. Please make sure that all instruments, instructions, and scripts are submitted with the request.
This includes questionnaires, interviewer manuals (if using interviewers or facilitators), all
response options for questions that require respondents to select a response from a group of
options, invitations given to potential respondents, instructions for completing the data
collection or additional follow-up requests for the data collection.
-Done
4. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection methods to be used. Data on the number of entities
(e.g., establishments, State and local government units, households, or persons) in the
universe covered by the collection and in the corresponding sample are to be provided in
tabular form for the universe as a whole and for each of the strata in the proposed sample.
Indicate expected response rates for the collection as a whole. If the collection had been
conducted previously, include the actual response rate achieved during the last collection.
- Please see Statistical Sample Plan in the Appendix.
Category of Respondent
Landing Page Survey

No. of Respondents
4,200 Annual
4

VA interaction Survey
Appointment Survey
Study Team Survey
Totals

4,200 Annual
4,200 Annual
2,400 Annual
794,535 Annual

5. Describe the procedures for the collection of information, including:
a. Statistical methodology for stratification and sample selection.
b. Estimation procedure.
c. Degree of accuracy needed for the purpose described in the justification.
d. Unusual problems requiring specialized sampling procedures.
e. Any use of periodic (less frequent than annual) data collection cycles to reduce
burden.
- Please see Statistical Sample Plan in the Appendix.
6. Describe methods to maximize response rates and to deal with issues of nonresponse. The
accuracy and reliability of information collected must be shown to be adequate for intended
uses. For collections based on sampling, a special justification must be provided for any
collection that will not yield "reliable" data that can be generalized to the universe studied.
- Please see Statistical Sample Plan in the Appendix.
7. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an
effective means of refining collections of information to minimize burden and improve
utility. Tests must be approved if they call for answers to identical questions from 10 or more
respondents. A proposed test or set of tests may be submitted for approval separately or in
combination with the main collection of information.
- Please see Statistical Sample Plan in the Appendix.
8. Provide the name and telephone number of individuals consulted on statistical aspects of the
design and the name of the agency unit, contractors, grantees, or other person(s) who will
actually collect or analyze the information for the agency.
Statistical Aspects:
Yao Djilan, Statistician, Veterans Experience Office, VA. (206) 240-1304
Collection and Analysis:
Evan Albert, Dir. of Measurement and Data Analytics, Veterans Experience Office, VA
(202) 875-9478
Dan Ostrow, Implementation Lead, Veterans Experience Office, VA (414) 690-8587

5

APPENDIX

COVID-19 Vaccine Survey
Sampling Methodology Report
Prepared by
Veteran Experience Office
Version 1.4
October 2020

6

Contents
Executive Summary ................................................................................................................................... 8
Part I – Introduction .................................................................................................................................... 9
A. Background ........................................................................................................................................ 9
B. Basic Definitions .............................................................................................................................. 10
C. Application to Veterans Affairs ...................................................................................................... 10
Part II – Methodology............................................................................................................................... 11
A. Target Population and Frame ....................................................................................................... 11
B. Sample Size Determination ........................................................................................................ 12
C. Data Collection Methods ............................................................................................................. 15
D. Reporting ....................................................................................................................................... 15
E. Quality Control .............................................................................................................................. 16
F. Quarantine Rules ......................................................................................................................... 16
Part III – Assumptions and Limitations.................................................................................................. 17
Coverage Bias, and Non-Response Bias ......................................................................................... 17
Appendix 1. List of Data Extraction Variables ............................................................................ 18
Appendix 2. Survey Questions ....................................................................................................... 18
Appendix 3. References ................................................................................................................... 19

7

Executive Summary
The COVID-19 Vaccine Survey is designed to measure the customer experience
during the recruitement into the COVID-19 clinical trial and other VA Researches. VA
understands that COVID-19 has significantly impacted the medium, frequency, and
overall experience in which Veterans receive their healthcare. This surveys is created to
understand how Veterans, ,family members, friends and other volunteers are recruited
into clinical trial via various modalities such as newletters, posters, social media,
targeted mails and emails, referral from a doctor or from a Veterans Service Officer
(VSO) etc. Participants’ experience data will be collected using an online transactional
survey disseminated via an invitation email sent to selected beneficiary or via the
VA.gov page questionaires (for a Landing Page Survey). Surveys invites will be sent
twice per week. After the survey has been distributed, recipients have two weeks to
complete the survey. Invitees will receive a reminder email after one week.
There are four surveys targeting the recruitment for Covid-19 Vaccine trial and
other research.: (1) the first survey aims to understand the modality that lead the
participant to the COVID-19 research landing page; (2) the second survey aims to
measure the experience of participants from their interaction during the recruitment
phase; (3) the third survey aims to measure the experience of participants in scheduling
their appointment; (4) the fourth survey pertaints to the Study Team who enrolls
volunteers at specific study sites.
The survey questionnaire is brief and contains general Likert-scale (a scale of 15 from Strongly Disagree to Strongly Agree) questions, checkbox and radio button
questions, to assess customer experience. A qualitative study was conducted to
analyze the question content and record reactions on a focus group consisting of
Veterans. The sample will be distributed across the participants population defined
further below.
This report describes the methodology used to conduct the COVID-19 Vaccine
Survey. Information about quality assurance protocols, as well as limitations of the
survey methodology, is also included in this report.

8

Part I – Introduction
A. Background
The Enterprise Measurement and Design team (EMD) is part of the Veterans
Experience Office (VEO). The EMD team is tasked with conducting transactional
surveys of the Veteran and Beneficiary population to measure their satisfaction with the
Department of Veterans Affairs (VA) numerous benefit services. Thus, their mission is
to empower Veterans by rapidly and discreetly collecting feedback on their interactions
with such VA entities as NCA, VHA, and VBA. VEO surveys generally entail probability
samples which only contact minimal numbers of beneficiaries necessary to obtain
reliable estimates. This information is subsequently used by internal stakeholders to
monitor, evaluate, and improve beneficiary processes. Beneficiaries are always able to
decline participation and have the ability to opt out of future invitations. A quarantine
protocol is maintained to limit the number of times a beneficiary may be contacted, in
order to prevent survey fatigue, across all VEO surveys.
In order to continue to provide quality service to Veterans, VEO has been
commissioned to measure the satisfaction Veterans and Non Veterans, as it relates to
their experience for enrolling into the the COVID-19 Vaccine research registry.

9

B. Basic Definitions
Coverage

The percentage of the population of interest that is
included in the sampling frame.
Measurement Error The difference between the response coded and the
true value of the characteristic being studied for a
respondent.
Non-Response
Failure of some respondents in the sample to provide
responses in the survey.
Transaction
A transaction refers to the specific time a beneficiary
interacts with the VA that impacts the beneficiary’s
journey and their perception of VA’s effectiveness in
caring for beneficiaries.
Response Rate
The ratio of participating persons to the number of
contacted persons. This is one of the basic indicators of
survey quality.
Sample
In statistics, a data sample is a set of data collected
and/or selected from a statistical population by a defined
procedure.
Sampling Error
Error due to taking a particular sample instead of
measuring every unit in the population.
Sampling Frame
A list of units in the population from which a sample may
be selected.
Reliability
The consistency or dependability of a measure. Also
referred to as standard error.
C. Application to Veterans Affairs
Customer experience and satisfaction are usually measured at three levels to:
1) provide enterprises the ability to track, monitor, and incentivize service quality;
2) provide service level monitoring and insights; and
3) give direct point-of-service feedback.
This measurement may bring insights and value to all stakeholders at VA. Frontline VA leaders can resolve individual feedback from beneficiaries and take steps to
improve the customer experience; meanwhile VA executives can receive real-time
updates on systematic trends that allow them to make changes.

10

Part II – Methodology
A. Target Population and Frame
The target population of the COVID-19 Vaccine Survey is the participants who
register to the volunteer list on the VA COVID-19 Research landing page for the first 3
surveys, subsequentely differentiated by other criteria; while the research admins
constitute the population for the survey 4 (Study Team Survey). Survey 1 is targeting all
individuals who volunteerly register themselves on the Va.gov Covid Research website.
Survey 2 is targeting the individuals who have declined to join a study after interacting
with a research study team coordinator staff. Individuals who were found ineligible for a
survey will be excluded from the is Survey 2 cohort. Survey 3 is targeting all the
participants of the study who have scheduled an appointment. However, the survey 4 is
targeting the research study team, which comprises the staff and the administrator
within 3 Days of interacting with a study participant during their first appointment. See
table 1 below.

Table 1. Surveys Population

Survey

How

When

Response

Population

Survey
1

Landing page
register volunteer
list (assumption survey will only be
sent to people who
registers on
va.gov)

Va.gov

Immediately
after
interaction

Not
Applicable

Everyone who registers on
VA.gov

Survey
2

VA interaction–
sign up into a
research/don’t sign
up into a research

EMDEmail

3 days after
interaction

2 weeks (
one reminder
after 7 days)

After volounteers register for
the study, they are contacted
with further information.
People that do not want to
move forward after being
called will receive Survey 2
(Exclude – ineligible and
cohort that move forward)

11

Survey
3

Appointment

EMDEmail

3 days after
interaction

2 weeks (
one reminder
after 7 days)

Anyone who made an
appointment

Survey
4

Study Team
Survey

EMDEmail

3 days after
interaction

2 weeks (
one reminder
after 7 days)

Study staff that interacted
with the participants for the
first Appointment.

A random sample will be drawn from the target population. The participants are
the primary sampling unit and they are randomly selected from the population. Table 2
below depicts the recruitment mode and methods for each surveys.

Table 2. Survey Mode
Surveys

Recruitment Method

Time After
Transaction

Recruitment
Period

Landing
Page Survey

Email / Social Media /
Newsletter / Referral
Recruitment

Within 3 days
after registration

14 Days

VA
interaction

Email

Within 3 days
after interaction

14 Days

Within 3 days
after interaction

14 Days

Within 3 days
after interaction

14 Days

Appointment Email

Study Team
Survey

Email – TBD

(Reminder after
7 Days)
(Reminder after
7 Days)
(Reminder after
7 Days)
(Reminder after
7 Days)

B. Sample Size Determination
This survey aims to collect enough responses per month to begin to understand
the Veterans or Beneficiaries and their families Experience during their recruitment into
the COVID-19 Clincal Trial.
12

Since we do not currently have enough information on the target population, a
sequential sampling technique can be implemented to attain the statistical significance.
However, this will end up being more costly because if the required analysis between
the sampling collection phases. A generic simple randomization sampling will be be
adopted instead. To achieve a certain level of reliability, the sample size for a given
level of reliability is calculated below (Lohr, 1999):
For a population that is large, the equation below is used to yield a representative
sample for proportions:
2
𝑍𝛼/2
𝑝𝑞
𝑛0 =
2
𝑒
where
•
•

•

𝒁𝜶/𝟐 = is the critical Z score which is 1.96 under the normal distribution when using
a 95% confidence level (α = 0.05).
p = the estimated proportion of an attribute that is present in the population, with
q=1-p.
o Note that pq attains its maximum when value p=0.5 or 50%. This is what is
typically reported in surveys where multiple measures are of interest. When
examining measures closer to 100% or 0% less sample is needed to
achieve the same margin of error.
e = the desired level of precision or margin of error. For example, for the ECC
survey the targeted margin of error is e = 0.03, or +/-3%.

For a population that is relatively small, the finite population correction is used to
yield a representative sample for proportions:
𝑛0
𝑛=
𝑛
1+ 0
𝑁
Where
•
•

𝒏𝟎 = Representative sample for proportions when the population is large.
N = Population size.

The margin of error surrounding the baseline proportion is calculated as:
𝑀𝑎𝑟𝑔𝑖𝑛 𝑜𝑓 𝐸𝑟𝑟𝑜𝑟 = 𝑧𝛼/2 √

𝑁 − 𝑛 𝑝(1 − 𝑝)
√
𝑁−1
𝑛

Where
•
𝒁𝜶/𝟐 = 1.96, which is the critical Z score value under the normal distribution when
using a 95% confidence level (α = 0.05).
•
N = Population size.
•
n = Representative sample.
•
p = the estimated proportion of an attribute that is present in the population, with
q=1-p.

13

VEO’s traditional approach of developing a probabilistic survey with a 90-95%
confidence interval with a 3-5% MOE will be applied to this survey. Since we do not
have enough information on the target population yet, we will be more lenient and
proceed at 95% confidence interval with a 5% MOE.
Table 3 below shows the sampling numbers proposed; where the estimated
response rate is suggested from past surveys and the voluntary enrolment into the trial.
Survey 2 is expected to have a lower response rate because it is targeting the cohort
that are declining to join the trial. We are making these recommendation because of the
lack of specific information on the target population.

14

Table 3. Samples Recommendation:
Survey

Survey 1

Estimated
Number in
the
Population
>3,000

Estimated
Monthly
Response
Needed
350

Survey 2

>3000

Estimated Monthly
Response Invitation
Rate
Sample
Needed
~40%
875
~20%

350
Survey 3

>3,000

438
1750

~40%
350

Survey 4

<500

219
875

7%
200

Weekly
Invitation
Sample
Needed
219

2,858 or All
Participating
Staff

715 or All
Participating
Staff

C. Data Collection Methods
Recruitment occurs twice per week for all Surveys. Participants will have two weeks to
complete the survey. A reminder email is sent after one week to non-respondents, to remind them
that the survey is available for another week. Once the participants complete the survey, their
response data is immediately available within Veterans Signals (VSignals).

D. Reporting
Researchers will be able to use the Veteran Signals (VSignals) system for interactive
reporting and data visualization. VA employees with a PIV card and appropriate permissions may
access the system at https://va.voice.va.gov/sso/va/pages/. The scores may be viewed by Age
Group and Gender in various charts for different perspective. They are also depicted within time
series plots to investigate trends. Finally, filter options are available to assess scores at varying
time periods and within the context of other collected variable information.
The consumption and dissemination of this data will be restricted to Authorized Users with
the need to know; this will also be addressed in training and in the communication plan.
Recruitment is continuous but the results should be combined into a monthly data file for
more precise estimates, at the call center level. Short interval estimates are less reliable for small
domains, and should only be considered for aggregated populations. Monthly estimates will have
larger sample sizes, and therefore higher reliability. Estimates over longer periods are the most
precise but will take the greatest amount of time to obtain and are less dynamic in that trends and
short-term fluctuation in service delivery may be missed. Users examining subpopulation should
be particularly diligent in assuring that insights stem from analysis with sufficient sample in the
subpopulations being examined or compared.
15

E. Quality Control
To ensure the prevention of errors and inconsistencies in the data and the analysis, quality
control procedures will be instituted in several steps of the survey process. Records will undergo a
cleaning during the population file creation. The quality control steps are as follows.
1. Records will be reviewed for missing sampling and weighting variable data. When records
with missing data are discovered, they will be either excluded from the population file or put
into separate strata upon discussion with subject matter experts.
2. Any duplicate records will be removed from the population file to both maintain the
probabilities of selection and prevent the double sampling of the same Veteran.
3. Invalid emails will be removed.
The survey sample loading and administration processes will have quality control
measures built into them.
1. The survey load process will be rigorously tested prior to the induction of the survey to
ensure that sampled customers are not inadvertently dropped or sent multiple emails.
2. The email delivery process is monitored to ensure that bounce-back records will not hold
up the email delivery process.

F. Quarantine Rules
VEO seeks to limit contact with Veterans as much as possible, and only as needed to
achieve measurement goals. These rules are enacted to prevent excessive recruitment attempts
upon Veterans. VEO also monitors Veteran participation within other surveys, to ensure Veterans
do not experience survey fatigue. All VEO surveys offer options for respondents to opt out, and
ensure they are no longer contacted for a specific survey.
Table 5. Proposed Quarantine Protocol
Quarantine Rule Description
Repeated
Sampling for the
COVID-19
Vaccine Survey
Other Surveys

Opt Outs

Elapsed
Time

Number of days between receiving one invite and
receiving another for the COVID-19 Vaccione
Surveys.
Veterans engaged that have recently completed
other VEO surveys will not be selected for 30
days.
Persons indicating their wish to opt out of either
phone or online survey will no longer be
contacted.

16

30 Days

30 Days

Indefinite

Part III – Assumptions and Limitations
Coverage Bias, and Non-Response Bias
Nonresponse is defined as failure of selected persons in the sample to provide responses.
This is observed virtually in all surveys, in that some groups are more or less prone to complete
the survey. The nonresponse issue may cause some groups to be over- or under-represented.
Coverage bias is another common survey problem in which certain groups of interest in the
population are not included in the sampling frame. The reason that these beneficiaries cannot
participate is because they cannot be contacted (no email address available). In both cases, the
exclusion of these portions of beneficiaries from the survey contributes to the measurement error.
The extent that the final survey estimates are skewed depends on the nature of the data collection
processes within an individual line of business and the potential alignment between beneficiary
sentiment and their likelihood to respond.
Survey practitioners recommend the use of sample weighting to improve inference on the
population so that the final respondent sample more closely resembles the true population. It is
likely that differential response rates may be observed across different age and gender groups.
Weighting can help adjust for the demographic representation by assigning larger weights to
underrepresented group and smaller weights to over-represented group. Stratification can also be
used to adjust for nonresponse by oversampling the subgroups with lower response rates. In both
ways of adjustments, weighting may result in substantial correction in the final survey estimates
when compared to direct estimates in the presence of non-negligible sample error.

17

Appendix 1. List of Data Extraction Variables

Variables
VE_MTM_KEY (provided by EMD)
SURVEY_TYPE
FIRST_NM
LAST_NM
Interaction with the VA
DATEandTime
DOB
Age
GENDER
RACE/ ETHNICITY
Phone
EMAIL
Zip
State
Appointment_Date (if
Applicable)
Flu shot (Yes/No)
Flue Shot Date
Covid (yes/no)
Veteran (Y/N)
SmokeorVape (Y/N)
Employement Status
Transportation (Y/N)
Household size (with you
included)
CreateDateTime
LatestTimeDate (if create date
is blank, use this)
receivedOtherC19Vaccine (Y/N)
(Exclusion Criteria??)
studyTeamEligibilityOutcome
studyTeamEnrollmentStatus
studyTeamName
Other fields of interest

Format
Character
Character
Character
Character
DateTime
Date
Numeric
Character
Character
Numeric
Character
Numeric
Character
DateTime
Character
Character
Character
Character
Character
Character
Character
Numeric
DateTime
DateTime
Character
Character
Character
Character
TBD

Appendix 2. Survey Questions
See insert

18

COVID-19 VACCINE COVID-19 VACCINE COVID-19 VACCINE COVID-19 VACCINE
TRIAL_VA.GOV_V5.A_102920.pdf
TRIAL_V5.A_StudyTeam_110320.pdf
TRIAL_EMD 2_V5.A_102920.pdf
TRIAL_EMD 1_V5.A_102920.pdf

Appendix 3. References
Choi, N.G. & Dinitto, D.M. (2013). Internet Use Among Older Adults: Association with Health
Needs, Psychological Capital, and Social Capital. Journal of Medical Internet Research,
15(5), e97
Kish, L. (1992). Weighting for unequal P. Journal of Official Statistics, 8(2), 183-200.
Lohr, S. (1999). Sampling: Design and Analysis (Ed.). Boston, MA: Cengage Learning.
Liu, J., Iannacchione, V., & Byron, M. (2002). Decomposing design effects for stratified
sampling. Proceedings of the American Statistical Association’s Section on Survey
Research Methods.

19