Supporting Statement B_2900-0609_Survey of Veteran Enrollees' Health & Use of Health Care_updated Sep 2024

SUPPORTING STATEMENT B

SURVEY OF VETERAN ENROLLEES’ HEALTH AND USE OF HEALTH CARE

VA FORM 10-400

OMB CONTROL NUMBER 2900-0609

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each strata. Indicate expected response rates. If this has been conducted previously include actual response rates achieved

STATISTICAL METHODS

The 2023 Survey Methodology report for the Survey of Enrollees is included to provide a complete overview of statistical methods used in the survey. Below is an overview of key statistical methods.

B-1: Respondent Universe and Respondent Selection Method

The Survey of Enrollee Universe is the population of Veterans enrolled in the VA health care system and living within the U.S, Puerto Rico, or U.S. Virgin Islands as of the end of the fiscal year preceding the survey administration. Enrollee records without a valid address or that are missing one of the stratification variables are not included. In 2023, the sampling frame included 8,228,035 records from which 162,489 enrolled Veterans were randomly selected to receive an invitation to participate (2024 data is not available as of this writing).

2. Describe the procedures for the collection of information, including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed

• Unusual problems requiring specialized sampling procedures

• Any use of less frequent than annual data collection to reduce burden

B-2: Stratification Procedure

Traditionally, stratification variables have included VISNs (Veterans Integrated Service Network), whether the Veteran was grandfathered into the system during the 1999 enrollment reform (pre-enrollee) or enrolled after the reform (post-enrollee), and Veteran’s enrollment priority group collapsed into three bands of Priority 1-3, Priority 4-6, and Priority 7-8. In 2015, VHA began stratifying the sample by each of the health care systems 96 markets to provide more locally relevant detail for health care planning. However, data use at the market level has not been high and stratifying at this level of detail has increasingly become costly as the sample size needs to be larger. Therefore, in 2025 VHA will return to stratifying by the 18 VISNs to achieve the needed geographic distribution in the Survey of Enrollees. Due to rising survey costs we are also reducing our overall response target from 42,000 to 40,000, which will still allow for with an error margin of +/- 5%.

Stratification targets for 2025 and beyond will be to guarantee an effective sample size of at least 2000 completed surveys in each VISN (18) and 600 completed interviews for each of the three priority groupings in each 18 VISNs. The sample is also classified in three selection groups to avoid asking Veterans who responded receiving an invitation to take the survey again. Selection group one represents those Veterans who responded to the survey in the preceding year, selection group two represents enrolled Veterans who did not respond (either were not in the random sample or who were asked to respond, but did not), and selection group 3 represents recent enrolled Veterans who were not in the previous year’s survey universe. Selection group 2 is oversampled so that they can represent both selection group one and two. Finally, the sample is constructed to assure that the representation of female Veterans mirrors the percentage of female Veterans in the survey universe and that the representation of pre-enrollees mirrors the percentage of pre-enrollees in the survey universe. Note that the survey will no longer oversample pre-enrollees as this population has decreased and oversampling would place a burden on this elderly population.

After making these allocations, the final strata count for the survey is represented by VISN (18) x priority group (3) x pre/post enrollee (2) x selection group (3) * male/female (2) for a total of 648 strata.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

B-3: Response Rate Maximization and Statistical Analysis

Methodological experiments early in the development of the survey taught key lessons about improving both response rates and reducing bias. The current survey methodology adheres to that developed between 2012 and 2015. Key changes included

• A propensity score weighting adjustment to correct for differential non-response by health utilization and demographic information

• The introduction of a multi-mode survey instrument to increase response and cooperation rates and reduce response bias by providing expanded access to the survey.

An in-depth discussion of sample development, stratification and weighting can be found in the 2023 Methodology report.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

B-4: Response Rate Maximization and Statistical Analysis

Methodological experiments early in the development of the survey taught key lessons about improving both response rates and reducing bias. The current survey methodology adheres to that developed between 2012 and 2015. Key changes included

• A propensity score weighting adjustment to correct for differential non-response by health utilization and demographic information

• The introduction of a multi-mode survey instrument to increase response and cooperation rates and reduce response bias by providing expanded access to the survey.

An in-depth discussion of sample development, stratification and weighting can be found in the 2023 Methodology report.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

B-5: Consultants on statistical design

Laura Bowman, Survey Project Manager/Program Analyst, Strategic Analysis Service (Tel. 202-615-3309)

Office of Strategic Planning and Analysis

VHA Chief Strategy Office (formerly VHA Office of Policy and Planning)

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Milliman Inc. (Contractor)

Ed Jhu, Principal and Consulting Actuary, (Tel. 206-504-5828)

Rob Bachelor, Consulting Actuary, Contractor (Tel. 206-504-XXXX)

Contractor:

Trilogy Federal, LLC

1100 Wilson Blvd, Suite 1100

Arlington, VA 22209

Westat

1600 Research Boulevard

Rockville, MD 20850

Mike Kwanisai, PhD

Senior Statistician, Westat

Previous Advisors/Contractors

Cathy Tomczak (retired)

Lead Statistician

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

Jim Schaefer (Currently, Director of Surveys, Office of Performance Measures)

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue NW

Washington, DC 20420

Mike Schwaber, Program Analyst, Strategic Analysis Service (Tel. 202-461-7108)

Office of Strategic Planning and Analysis

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue, NW

Washington, DC 20420

IDF (Contractor)

126 College Street

Burlington Vermont 05401

Burlington, VT, Contractor for the survey

Vicki A. Freedman, Ph.D.

Research Professor

Institute for Social Research

University of Michigan

426 Thompson Street

Ann Arbor, MI 48106

Judith Kasper, Ph.D.

Professor

The Johns Hopkins University Bloomberg School of Public Health

Rm 641, 624 N. Broadway

Baltimore MD 21205

Page 1