VA Response to Prior Terms

Department of
Veterans Affairs
Date:

Memorandum

April 12, 2012

From: Aaron Schneiderman, Ph.D., Principal Investigator, and Acting Director, Epidemiology
Program, Office of Public Health, VA
Subj: “National Health Study for a New Generation of U.S. Veterans” (OMB No. 2900-0722) –
Response to Office of Management and Budget (OMB) Request for Non-Response Bias
Analysis
To:

OMB; Cynthia Harvey-Prior, Management Analyst (OMB Liaison), VA; Denise McLamb,
Program Analyst, VA; Regina Grant, Director, VHA Forms & Publications Office, VA

Please see the VA response to the Office of Management and Budget request for a
robust non-response bias analysis for the first wave of the “National Health Study for a
New Generation of U.S. Veterans.” This includes a discussion that places our study in
context with other similar health studies of military and Veteran populations.

“National Health Study for a New Generation of U.S. Veterans” (OMB No. 2900-0722)
Response to Office of Management and Budget Request for Non-Response Bias Analysis
From: Epidemiology Program, Office of Public Health, Department of Veterans Affairs
April 12, 2012

The National Health Study for a New Generation of U.S. Veterans is a longitudinal
health study comparing the health status of Operation Enduring Freedom/Operation Iraqi
Freedom (OEF/OIF) Veterans and non-deployed Veterans serving during the same time period
by comparing chronic medical conditions, post-traumatic stress disorder and other psychological
conditions, general health perceptions, functional status, mortality, health care utilization, and
VA disability compensation between these two Veteran populations. In order to accomplish this,
a permanent panel of 30,000 OEF/OIF Veterans and 30,000 non-deployed OEF /OIF era
Veterans will be followed for 10 years. The permanent panel is comprised of a populationbased sample of troops representing each branch of service, unit component (Active duty,
Reserve, National Guard) and gender. Women were oversampled to ensure adequate
representation in the study and comprise 20 percent of the sample.
A combination of mail surveys, online surveys, and Computer Assisted Telephone
Interviews (CATI) were used to collect data from Veterans. Data collection began in April 2009
for the pilot study, and August 2009 for the main study; data collection ended in January of
2011. Overall, the response rate was 34.3% (n=20,563). Of those who responded, 49%
responded on the web, 45% on the paper questionnaire, and 6% on the CATI. Table I shows
responder status by sampling frame characteristics. Respondents were more likely to be
female, have been deployed, and be older than non-respondents, which is consistent with other
studies of military and Veteran populations.
Since there are limited analyses that can be performed on a baseline cross sectional
data collection, the mode of data collection (paper and web vs. CATI) was examined as a
possible surrogate for non-response. The CATI calls were initiated as a mechanism to reach

non-responders after several repeat mailing attempts. Thirteen hundred and eighty six
participants responded to the CATI; these are subjects who would likely have been nonresponders if this final attempt had not been made. Table II shows a comparison of early (paper
or web) vs. late (CATI) responder status by sampling frame variables and by self-reported
health status. Late responders were more likely to have been deployed, and be younger; they
also more often endorsed that their health was excellent.
The response rate in this study is similar to other recent large cohort studies of military
populations. Smith et al (2011) reported a response rate of 34% for the baseline survey of the
Millennium Cohort Study (MCS), a population based longitudinal study consisting of 150,000
active duty service members. This 34% response rate includes participants from three panel
waves between 2001 and 2008. Exactly half of the participants in MCS have served in support
of the wars in Iraq and Afghanistan, which is the population of interest for the National Health
Study for a New Generation of U.S. Veterans.
Littman et al (2010) reported the results of a non-response analysis performed on a
subset of the MCS cohort (members enrolled between 2001 and 2003) at the first follow up
survey in 2004. Logistic regression models were used to calculate propensity scores and
propensity weights to examine the effect of non-response on measures of association at the
follow up survey. Results from this analysis indicated that nonresponse did not substantially
affect odds ratios of health outcomes related to deployment and other risk factors. The factors
that were found to be associated with response to the follow up survey were the same as those
found to be associated with response to the baseline survey (and greater response to health
surveys in the general population), such as female gender, being married, older age, and
higher educational attainment. This is the type of analysis that our office proposes to do when
follow up data is available for the study cohort.
Kang et al (2009) reported the results of the second follow up of a similar study of Gulf
War and Gulf War Era Veterans. The response rate for this follow up assessment was 34%

(9,970 total Veterans). Non-respondents were more likely to be younger, non-white, single,
Reserve/National Guard, male and enlisted rank during their service at the time of the 1991 Gulf
War. In order to determine whether or not these factors were significantly different from nonresponders with respect to health status, the self-reported health status (ranging from excellent
to poor) at baseline (1995) was compared between those who participated and those who did
not participate in the 2005 follow up. No differences were observed, and self reported health
status in 1995 was not a good predictor of response in 2005.
These published data suggest that the response rate achieved in the current study is
similar to the experience of other large population based health surveys of active duty military
and Veterans. The consistency of these findings may not be related to the study design, but a
function of the unique characteristics of the population under study and the trend toward
decreased participation rates in U.S. epidemiological studies in recent decades. Since we only
have baseline data and are limited on the types of analysis we can perform, we propose to
explore in greater depth non-response when data from the follow up survey is available. Failure
to continue this unique study with a follow up survey would result in a loss of valuable data to
VA on the health experiences and concerns of this specific group of recent Veterans over a
significantly long time frame; the health care needs of both VA and non-VA users; and
information to help VA effectively allocate health care resources and plan quality care.

Table I: Comparison of sampling frame characteristics between respondents and non-respondents to
the New Generation Study, 2009-2011.
Characteristic

Respondents
(N=20,563)

Non-Respondents
N= (39,437)

Deployment Status
Deployed
11,337 (55.13)
Non-deployed
9,226 (44.87)

18,663 (47.32)
20,774 (52.68)

Branch of Service
Air Force
Army
Marine
Navy

4,339 (21.10)
11,165 (54.30)
1,969 (9.58)
3,090 (15.03)

7,309 (18.53)
21,589 (54.74)
4,825 (12.23)
5,714 (14.49)

Unit Component
Active
Guard
Reserve

7,860 (38.22)
5,614 (27.31)
7,089 (34.47)

16,140 (40.93)
10,386 (26.34)
12,911 (32.74)

Sex
Males
Females

16,217 (78.86)
4,346 (21.14)

31,783 (80.59)
7,654 (19.41)

Age
24-29
30-39
40-49
50-59
60+

4,673 (22.73)
6,647 (32.33)
5,691 (27.68)
2,876 (13.99)
676 (3.29)

15,168 (38.46)
14,474 (36.70)
6,995 (17.74)
2,462 (6.25)
338 (0.85)

Table II: Comparison of sampling frame characteristics and self reported health among Early and Late
Responders
Characteristic

Early Responder
(Web and Paper)
(N=19,177)

Late Responder
(CATI)
(N=1,386)

Deployment Status
Deployed
Non-deployed

10,598 (55.26)
8,579 (44.74)

739 (53.32)
647 (46.68)

Branch of Service
Air Force
Army
Marine
Navy

4,082 (21.29)
10,386 (54.16)
1,831 (9.55)
2,878 (15.0)

257 (18.54)
779 (56.20)
138 (9.96)
212 (15.30)

Unit Component
Active
Guard
Reserve

7,330 (38.23)
5,211 (27.17)
6,636 (34.60)

530 (38.24)
403 (29.08)
453 (32.68)

Sex
Males
Females

15,099 (78.73)
4,078 (21.27)

1,118 (80.66)
268 (19.34)

Age
24-29
30-39
40-49
50-59
60+

4,250 (22.16)
6,158 (32.11)
5,359 (27.94)
2,755 (14.37)
655 (3.42)

423 (30.52)
489 (35.28)
332 (23.95)
121 (8.73)
21 (1.52)

Self-Reported Health*
Excellent
Very Good
Good
Fair
Poor

2,372 (12.42)
6,525 (34.16)
6,962 (36.44)
2,804 (14.68)
441 (2.30)

246 (17.81)
468 (33.86)
433 (31.33)
196 (14.18)
39 (2.82)

*Total denominator is 20,486 for this variable, to accommodate missing values