Veteran Education Sample Size

VETERAN EDUCATION TRANSACTIONAL SURVEY
METHODOLOGY BRIEF: SAMPLE DESIGN
The Veteran Benefits Administration oversees several programs that support Veterans furthering their
education or receiving approved training. This includes such programs as the Montgomery GI Bill (Active Duty and
Reserves), the Reserve Educational Assistance Program (REAP), and the Post 9/11 GI Bill. Veterans that participated in
these four initiatives were identified for measurement regarding their experience in the benefits process. Recent
recipients of the Education benefits process will be contacted via email and invited to complete a brief online survey.
The goal of this document is to describe the number of veterans that will be contacted for the survey.
Target Population: Veterans that applied for an education program OR had their school enrollment certified OR received
a tuition/stipend payment in the past calendar month. These will represent the 3 major reporting groups.
Over a million veterans took advantage of these services in 2015. The chart below shows the population figures from
that year. VEO will conduct a biweekly random sample from persons with an email address provided to the VA. In order
to reduce respondent burden, only a portion of participating veterans are contacted. This reduces the number of
veterans that will be contacted multiple times throughout the year to maintain quality estimates in successive months.
We also stipulate that persons will not be contacted repeatedly for consecutive months. The total amount of veterans
contacted will be determined to ensure a certain level of precision for the resulting monthly survey estimates1.
Education Benefit

Chapter
Type

FY 2015 Annual
Beneficiaries1

FY 2015
Monthly
Beneficiaries
65,873
5,117

Proposed Monthly
Respondent Size

Post 9/11 GI Bill
33
790,480
Montgomery GI Bill
30
61,403
(Active Duty)
Montgomery GI Bill
1606
63,030
5,252
(Selective Reserve)
Reserve Educational
1607
9,965
830
Assistance Program
1. Source: FY 2015 Annual Benefits Report (see footnote for web address).

3,300
1,100

Monthly
Recruitment
Sizes
16,500
<5,500

1,100

<5,500

1,100

<5,500

Sample Size Determination
Sample size will be based on a 95% Confidence Level and a 3% Margin of Error. This is a standard level of
precision used by survey administrators (Lohr 1999). It requires a minimum sample size of 1,100, which will be applied
separately to each of the for benefit groups (Post -9/11, MGIB-AD, MGIB-SR, and REAP). Within the largest beneficiary
group, Post-9/11, VEO intends to obtain precision estimates for three major reporting groups: Applications, Enrollments,
and Payments. Therefore, the minimum sample size across groups would be 6,600. It is advisable to increase upon this
to improve estimates for subpopulations of interest (e.g., Age Group, Gender, etc.). For these reasons, the monthly
sample target is set at 6,600 (1,100 per group). Presuming a 20% response rate, obtaining 6,600 respondents would
require contacting 33,000 veterans per month. This represents the maximum number of persons that may be contacted
on a monthly basis. In practice, these numbers may be reduced based on the population sizes for a particular month.
Due to seasonality, the amounts of applications, enrollment certifications, and payments will fluctuate each month.
1. https://benefits.va.gov/REPORTS/abr/ABR-Education-FY15-02032016.pdf
2. Lohr, Sharon Sampling Design and Analysis 1999

Sampling Design
The Veteran’s Experience Office will conduct a probability sample on the monthly target population. A frame is
prepared by extracting population information directly from governmental database resources: the Veteran Business
Administration Enterprise Data-Warehouse (EDW). These extracts will also be used to obtain universe figures for the
sample weighting process.
Beneficiaries will be randomly selected from the biweekly population according to a stratified design. The object
is to obtain a representative sample, with respect to the stratification variables. There should also be ample precision for
all major reporting variables. This may require oversampling at the Chapter level (sample weighting ) Levels 1 and 2 will
be defined explicitly and will have allocation targets. The allocation targets will fluctuate with monthly changes in the
population. However, the demographic strata in level 3 will require implicit selection via sequential sampling, and will
not have explicit targets for each grouping.




Level 1 – Encounter Type: Applications, Enrollments, and Payments (explicit)
Level 2 – Chapter Type (30, 33, 1606, 1607) (explicit)
Level 3 – Demographics: Age Group, Gender, Location (Implicit)

The exact allocation will be dependent on the biweekly beneficiary populations. At the middle and end of every
calendar month, a database query will be enacted, and the sample will be selected from those persons with a valid email
address. Selected persons will be invited to join the survey and provide information on their experience with the
education benefits process from the past month. Their answers will be collected for a two-week period, at which point
the survey will close. Beneficiaries will never be contacted in consecutive months, and respondents will not be contacted
again for a 3-month period. Once all the responses are collected, the sample will be weighted to reflect the entire
population (email and non-email).
Many survey practitioners recommend the use of weighted survey estimates improve inference to the
population. The method attempts to correct for under-coverage and non-response biases. The sample weighting
methodology will be applied separately to each encounter type: Applications, Enrollments, and Payments. Age Group
and Gender will be included as the primary weighting variables. VEO will consider adding Chapter Type as a secondary
weighting variable, providing that its inclusion does not add substantial variation to the final weights.
Summary
Sampling Design Feature
Survey Reporting
Data Collection
Stratification (Explicit)
Stratification (Implicit)
Quarantine
Weighting Classes
Weighting Variables

Description
Monthly, Quarterly
Biweekly
Encounter Type, Chapter Type
Age Group, Gender, Location
3-months for Respondents, 1 month for Non-participants
Encounter Type (Application, Enrollment, Payments)
Age Group, Gender , Chapter Type

1. https://benefits.va.gov/REPORTS/abr/ABR-Education-FY15-02032016.pdf
2. Lohr, Sharon Sampling Design and Analysis 1999