Justification Part B

SUPPORTING STATEMENT FOR OMB CONTROL NUMBER 2900-0609, CONTINUED

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each stratum. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

The Health and Reliance Survey universe to be sampled is the enrollees specified at some point in time. For example, the approximately 6.7 million living enrollees as of December 2004 was the population of interest for the 2005 survey. A random sample of approximately 100 in each of approximately 10 priority categories, for the pre and post enrollees in each of 21 healthcare networks is expected to yield an optimally stratified sample of approximately 42,000. The attached 2005 Health and Reliance Stratification show the potential priority group definitions and stratifications. The projected universe and sample size by strata, based on the 2005 enrollee survey, are included in these attachments. The pre- and post-enrollee strata have been combined in the 2005 enrollee survey strata attachment.

2. Describe the procedures for the collection of information, including: Statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose in the proposed justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

a. SURVEY:

(1) For the Health and Reliance Survey, VA and the contractor have arrayed the VA enrolled population into strata by VISN, priority level, and pre and post enrollee status. The sampling methodology is to generate an optimally stratified random sample of approximately 100 per strata cells, and call the veterans up to a maximum of 6 times according to a defined protocol in order to complete an interview. If, after the protocol is completed and 100 are not reached, additional sample in replicates of 10 are released for calling with the same protocols. Sample is released until the quotas are met, and all sample is brought to protocol. For a sample size of approximately 42,000, we expect survey estimates based on the total sample to have error margins of approximately +/-0.5 percentage points at the 95 percent confidence level. For each priority level combining pre and post enrollees within VISN, with a sample size of approximately 200, we expect survey estimates to have error margins in the range of approximately +/-7 percentage points at the 95 percent confidence level. Confidence interval projections are based on measuring a population percentage equal to 50 percent. These projections do not account for sample design effects, which may increase the actual error margins for the survey estimates. VA will provide the contractor a list of enrollees from which to draw. This survey will be repeated on an approximately annual basis.

(2) The 95% confidence intervals for data from the enrollee survey are a function of the number of cells in the tabulations. In the 2005, the data collected for this survey are from a survey sample. Inherent in a sample is sampling error. Since sampling error can be estimated, it is important to consider standard errors when comparing subpopulations, such as among VISNs. To assist such comparisons, standard errors are available upon request.

(3) Over the years that the survey has been conducted, the questionnaire has evolved to meet VHA’s increasingly complex data needs. In particular, some of the questions have changed little over time, as they probe for basic socioeconomic, demographic, and health-related data. However, other questions have been added, deleted, or expanded upon to address more and different sorts of issues of timely and topical interest to VHA. When questions change to reflect the Departmental emphasis and VHA’s specific, forward-looking data needs and interests, OMB will be asked to approve the new survey.

b. FOCUS GROUP PROGRAM:

(1) Key Informant and Stakeholders to be interviewed (Phase I) would probably be selected by a VHA enrollee survey committee on the basis of the potential participants' knowledge and experience of veterans and with veterans issues. For example, a committee might identify certain people here at VACO who it feels can contribute to the survey effort, and/or the committee might decide to send a global e-mail out to Network Planners in the field requesting recommendations of names of potential participants and then just go down the list that we get back till we have enough people who agree to participate in Phase I.

(2) On the other hand, veteran participants in individual In-Depth Interviews or in Focus Groups (Phases II and III/IV) ) would be selected by random sampling from the VHA enrollment file, based on stratification by the basic characteristics of interest - which will vary from one survey year to the next. In order to include non-enrolled veterans, as well, we would probably use Random Digit Dialing techniques to locate potential participants. The number of potential respondents will be increased until a sufficient number of individuals have agreed to participate.

(3) However, as Focus Group activities are "qualitative" and not "quantitative" research, none of these participant selection methods are statistical samples in the true sense; there are no weights or statistical errors to compute. However, the methods outlined above do allow for covering fairly broad cross-sections of the enrollee and veteran populations, as might be necessary or desirable in conducting focus groups.

3. Describe methods used to maximize the response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Veterans generally identify with the need to provide VA information to improve the system’s ability to provide for their care. The response rates for the Health and Reliance Survey of approximately 41 percent is of concern. However, VA is working diligently to improve that rate, which is primarily due to a lack of adequate phone numbers. Nevertheless, once veterans are contracted, their cooperation rate is an impressive 73 percent. This compares to response rates obtained in other national telephone surveys. Terms of clearance state that OMB continues to encourage VA to work with their contractor to enhance the coding of case level outcomes to better understand reasons for nonresponse and conduct research and test strategies for identifying and handling veterans in institutions.

Given the wide variety of covariates that are related to the likelihood of an enrollee responding, we recommend instituting a weighting adjustment that corrects for this differential. The weighting adjustment should include health utilization and demographic information. Stratum level disproportionate sampling should be adjusted for with base weights (or design weights) prior to the nonresponse adjustments. We recommend a propensity score weighting adjustment, in which individual propensities to respond are measured with a probability model (such as the logistic regression modeling presented herein). The estimated probabilities are then used to group the enrollees in to classes with similar probabilities. Typically five classes (or quintiles) are formed. Within each class, the respondents are weighted up to account for the nonrespondents. These weighting adjustments reduce potential bias to the extent that the nonrespondents and respondents with similar response probabilities are also similar with respect to the survey statistics of interest

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

Earlier (1999, 2000, 2002, 2003, and 2005) survey questions on the Health and Reliance Survey have been clarified based upon veteran or interviewer questions and input. Any proposed survey questions will be pre-tested on fewer than 10 veterans in order to work out any problems with wording or veterans’ comprehension of questions, etc., before full implementation of the CATI surveys. The attached 2008 VHA enrollee survey instrument includes all necessary modifications to date in the survey instrument.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Randall Remmel, Ph.D., (Tel. 813.645.5614),

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

810 Vermont Avenue NW

Washington, DC 20420

Chip Harvey (Tel. 919.460.5313), VSSC,

VHA Office of the Assistant Deputy Under Secretary for Health for Operations and Management, Department of Veterans Affairs

810 Vermont Avenue NW

Washington, DC 20420

Donald Stockford (Tel 703.683.0535

Retired September 2006 from the VHA Office of the Assistant Deputy Under Secretary

for Health for Policy and Planning,

3303-D Commonwealth Avenue

Alexandria VA 22305

Marybeth Matthews (Tel. 414-384-2000 ext 42359),

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

5000 West National Avenue

Milwaukee WI 53295

Dee Ramsel, Ph.D., (Tel 414.384.2000 ext 42353)

VHA Office of the Assistant Deputy Under Secretary for Health for Policy and Planning,

Department of Veterans Affairs

5000 West National Avenue

Milwaukee WI 53295

Kathi Patterson, FSA, MAAA, (Tel. 206.504.5539),

Principal and Consulting Actuary, Contractor

Macro, Inc. (Tel. 802-863-9600),

Leslyn Hall

Randall ZuWallack

126 College Street

Burlington Vermont 05401

Burlington, VT, Contractor for the survey

Page 3