Supporting Statement B_Survey of Enrollees' Health & Use of Health Care 2022_updated Aug 2021

SUPPORTING STATEMENT FOR SURVEY OF VETERAN ENROLLEES’ HEALTH AND

USE OF HEALTH CARE

VA FORM 10-21034g

OMB CONTROL NUMBER 2900-0609

STATISTICAL METHODS

The 2020 Survey Methodology report survey is included to provide a complete overview of statistical methods used in the Survey of Enrollees. Below is an overview of key statistical methods.

B-1: Respondent Universe and Respondent Selection Method

The Survey of Enrollee Universe is the population of Veterans enrolled in the VA health care system as of the end of the fiscal year preceding the survey administration. Enrollee records without a valid address or that are missing one of the stratification variables are not included. In addition, Veterans living outside the U.S. or Puerto Rico are also excluded from the sample. In 2020, the sampling frame included 8,725,547 records from which 134,176 enrolled Veterans were randomly selected to receive an invitation to participate.

B-2: Stratification Procedure

Traditionally, stratification variables have included VISNs (Veterans Integrated Service Network), whether the Veteran was grandfathered into the system during the 1999 enrollment reform (pre-enrollee) or enrolled after the reform (post-enrollee), and Veteran’s enrollment priority group collapsed into three bands of Priority 1-3, Priority 4-6, and Priority 7-8. In 2015, VHA began stratifying the sample by each of the health care systems 96 markets in order to provide more locally relevant detail for health care planning.

The current stratification targets are to guarantee an effective sample size of at least 350 completed interviews in each market (96), 597 completed interviews for each of the three priority groupings in each of the 18 VISNs, and a minimum of 20 percent of all completed interviews representing pre-enrollees (those using the VA health care system before 1999 enrollment reform). In total, this represents 576 strata. Note: Pre-enrollees tend to have more co-morbidities and are therefore more expensive to treat. For this reason, the Enrollee Health Care Projection Model actuary historically requested oversampling of this population. However, as pre-enrollees represent a smaller portion of the total enrollee population, the survey methodology will begin to phase this oversampling out by five percentage points each year.

For a sample size of approximately 42,000, we expect survey estimates based on the total sample to have error margins of approximately +/-0.5 percentage points at the 95 percent confidence level.  For each priority level combining pre and post enrollees within VISN, or each market combining pre and post enrollees within market, with a sample size of approximately 350, we expect survey estimates to have error margins in the range of approximately +/-5 percentage points at the 95 percent confidence level. Confidence interval projections are based on measuring a population percentage equal to 50 percent. These projections do not account for sample design effects, which may increase the actual error margins for the survey estimates. VA will provide the contractor a list of enrollees from which to draw.

B-3 & B-4: Response Rate Maximization and Statistical Analysis

Methodological experiments early in the development of the survey taught key lessons about improving both response rates and reducing bias. The current survey methodology adheres to that developed between 2012 and 2015. Key changes included:

• A propensity score weighting adjustment to correct for differential non-response by health utilization and demographic information

• The introduction of a multi-mode survey instrument to increase response and cooperation rates and reduce response bias by providing expanded access to the survey.

An in-depth discussion of sample development, stratification and weighting can be found in the 2020 Methodology report.

B-5: Consultants on statistical design

Laura Bowman, Survey Project Manager/Program Analyst, Strategic Analysis Service (Tel. 202-615-3309)

Office of Strategic Planning and Analysis

VHA Chief Strategy Office (formerly VHA Office of Policy and Planning)

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Milliman Inc. (Contractor)

Ed Jhu, Principal and Consulting Actuary, (Tel. 206-504-5828)

Rob Bachelor, Consulting Actuary, Contractor (Tel. 206-504-XXXX)

Contractor:

Advance Survey Design

1193 10^th Street, Suite A

Monterey, CA 93940

Mike Larson, PhD

Lead Statistician

Professor, Department of Statistics, George Washington University

Professor, Department of Mathematics, St. Michael’s College (Vermont)

Previous Advisors/Contractors

Westat (Contractor)

1600 Research Boulevard

Rockville, MD 20850

Cathy Tomczak (retired)

Lead Statistician

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Jim Schaefer (Currently, Director of Surveys, Office of Performance Measures)

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue NW

Washington, DC 20420

Mike Schwaber, Program Analyst, Strategic Analysis Service (Tel. 202-461-7108)

Office of Strategic Planning and Analysis

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

IDF (Contractor)

126 College Street

Burlington Vermont 05401

Burlington, VT, Contractor for the survey

Vicki A. Freedman, Ph.D.

Research Professor

Institute for Social Research

University of Michigan

426 Thompson Street

Ann Arbor, MI 48106

Judith Kasper, Ph.D.

Professor

The Johns Hopkins University Bloomberg School of Public Health

Rm 641, 624 N. Broadway

Baltimore MD 21205

Page 1