Justification_B_SOE 2018 OMB Clearance

SUPPORTING STATEMENT FOR OMB CONTROL NUMBER 2900-0609, CONTINUED

STATISTICAL METHODS

VHA adjusts its oversampling and sampling distribution strategies each year to meet the needs of a changing Veteran population. The 2017 Survey Methodology report survey is included provide a complete overview of statistical methods used in the Survey of Enrollees. Below is an overview of key statistical methods.

B-1: Respondent Universe and Respondent Selection Method

The Survey of Enrollee Universe is the population of Veterans enrolled in the VA health care system as of the end of the fiscal year preceding the survey administration. Enrollee records without a valid address or that are missing one of the stratification variables are not included. In addition, Veterans living outside the U.S. or Puerto Rico are also excluded from the sample. In 2017, the sampling frame included 8,180,466 records from which 499,743 enrolled Veterans were selected.

B-2: Stratification Procedure

Traditionally, stratification variables have included VISNs (Veterans Integrated Service Network), whether the Veteran was grandfathered into the system during the 1999 enrollment reform (pre-enrollee) or enrolled after the reform (post-enrollee), and Veteran’s enrollment priority group collapsed into three bands of Priority 1-3, Priority 4-6, and Priority 7-8. In 2015, VHA began stratifying the sample by each of the health care systems 96 markets in order to provide more locally relevant detail for health care planning.

The current stratification targets are to guarantee an effective sample size of at least 315 completed interviews in each market (96), 597 completed interviews for each of the three priority groupings in each of the 18 VISNs, and a minimum of 30 percent of all completed interviews representing pre-enrollees (those using the VA health care system before 1999 enrollment reform). In total, this represents 576 strata.

For a sample size of approximately 42,000, we expect survey estimates based on the total sample to have error margins of approximately +/-0.5 percentage points at the 95 percent confidence level.  For each priority level combining pre and post enrollees within VISN, or each market combining pre and post enrollees within market, with a sample size of approximately 350, we expect survey estimates to have error margins in the range of approximately +/-5 percentage points at the 95 percent confidence level.  For a sample size of approximately 12,600 pre enrollees (30% of total), the margin of error should be approximately +/- 0.9 percentage points. For a sample size of approximately 29,400 post enrollees (70% of total), the margin of error should be approximately +/- 0.6 percentage points. Confidence interval projections are based on measuring a population percentage equal to 50 percent.  These projections do not account for sample design effects, which may increase the actual error margins for the survey estimates.  VA will provide the contractor a list of enrollees from which to draw.

B-3 & B-4: Response Rate Maximization and Methodological Tests

VHA has systematically conducted tests to refine the Survey of Enrollees in general and, specifically, the survey’s response rate. Key changes resulting from these tests include:

• A propensity score weighting adjustment to correct for differential non-response by health utilization and demographic information

• The introduction of a multi-mode survey instrument to increase response and cooperation rates and reduce response bias by reaching more enrollees who may have a different response pattern than those who reply to telephone surveys.

B-5: Consultants on statistical design

Gerard Benson, Director, Strategic Analysis Service (202-461-7084)

Office of Strategic Planning and Analysis

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Laura Bowman, Survey Project Manager/Program Analyst, Strategic Analysis Service (Tel. 202-461-7108)

Office of Strategic Planning and Analysis

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Mike Schwaber, Program Analyst, Strategic Analysis Service (Tel. 202-461-7108)

Office of Strategic Planning and Analysis

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Contractor:

Advance Survey Design

1193 10^th Street, Suite A

Monterey, CA 93940

Page 2