Cognitive Testing Results

Cognitive Tests of Veterans Supplement Questions

Summary Report

1. Purpose

The Current Population Survey (CPS) is a nationally-representative monthly sample survey of households that provides information on labor force activity in the United States. The CPS is the source of the monthly national unemployment rate, as well as rich demographic details, including age, sex, race, Hispanic origin, educational attainment, marital status and family attributes, foreign-born status, veteran status, and other demographic characteristics.

The Veterans Supplement is conducted every two years and is asked at the end of the CPS. August 2005 was the last time this supplement was fielded. This supplement is sponsored by the Department of Veterans Affairs (VA) and the Veterans Employment and Training Service (VETS) at the Department of Labor.

Recently the Veterans Supplement was updated to reflect the growing interest in learning about the labor market situation of veterans who are or were in the Reserve and National Guard. Of particular interest are those Reserve or National Guard veterans who served in combat during the Gulf War era (August 1990 to present). Five new questions were added to the supplement to capture this information. Before these new questions could be fielded, they needed to undergo cognitive testing by OSMR.

The purpose of the cognitive interview of the new Veterans Supplement questions is to test how well the questions capture the information that is sought. Are the questions clear? Do they use any terminology that is unfamiliar or confusing to the respondents? We will test the new questions for clarity, comprehension, length, and any sensitivity among questions.

2. Procedure

Interviews were conducted by an OSMR researcher with one participant at a time. Each session lasted approximately 30 minutes. The majority of the interviews were conducted in-person (N=8) but some were completed on the telephone (N=4). The OSMR researcher administered the interviews and recorded all responses on paper. The sessions were also audio-taped to verify and supplement the notes kept by the researcher. The cognitive interviews were conducted retrospectively. First, participants were administered a paper version of the Current Population Survey and a paper version of the Veterans’ Supplement. They were instructed to answer each question as if they were being interviewed for the survey. Then the participants were asked retrospective probes about question meaning and they were asked to comment about some of the major concepts studied in the survey.

3. Participants and Burden Hours

Participants were recruited from the general population, with special care to recruit those who served in the Armed Forces. Approximately 12 hours were used. Participants who were interviewed in person were compensated $40.00 for their time and effort. Participants who were interviewed via telephone did not receive compensation.

Twelve cognitive interviews were conducted: 7 interviews with veterans and 5 interviews with spouses or other household members of veterans (also called proxies).

The characteristics of the participants are shown below in Table 1.

Table 1. Participant Characteristics

	Participant No.	Branch of Armed Forces	Gulf War	Gender1
Veteran	1	Army National Guard	Yes	Male
	2	Marine Corps Reserve	Yes	Male
	3	Marine Corps Reserve	Yes	Male
	4	Army Reserve	Yes	Male
	5	Air Force	Yes	Male
	6	Air Force	Yes	Male
	7	Army Reserve	Yes	Female
Proxy	1	Army Reserve	Yes	Female
	2	Army	No	Female
	3	Navy Reserve	Yes	Male
	4	Air Force	Yes	Female
	5	Marine Corps Reserve	Yes	Female

4. Findings

Overall, the new questions for the Veteran’s Supplement tested very well with very few comprehension problems observed. Each of the new questions are listed below with a detailed summary of the findings.

S1b. Earlier it was reported that you served on active duty in the U.S. Armed Forces. Was any of your active service the result of a call-up from the Reserve or National Guard?

Veterans were able to paraphrase the question well, indicating that they comprehended the question. When paraphrasing, participants often said “activation” or “was activated” in lieu of “call-up from the Reserve.” They understood what was meant by call-up (involuntary deployment; mobilizing a unit), but activation might be more natural language.

Proxies’ answers were similar to the ones given by the Veteran’s group. They said this question was asking (in the current environment) about being deployed for a tour of duty (i.e., 6 months or 1 year) in Iraq or Afghanistan.

S1c . Have/has (you/name) ever been a member of the Reserve or National Guard?

No problems were observed with this question. All of the participants (who received this question) found this question to be very straightforward.

S1d. Are you currently a member of the Reserve or National Guard?

Again, no problems were observed with this question.

S10a From which branch of the Armed Forces was/were (you/name) last released from active duty?

Branch of the Armed Forces

• Participants (both Veterans and Proxies) said that when they think of branches of the armed forces, they think of the Army, Navy, Air Force and the Marine. They do not often think of the Coast Guard, the Reserves or the National Guard. Most said that they did not think of the Coast Guard as the military, although they acknowledge that technically they are part of the armed forces. Only one participant mentioned the Coast Guard, unprompted. One participant (Veteran) said that the Coast Guards do not fall under the Department of Defense, like the other branches (Army, Navy, Marine, Air Force), and so he did not initially think of them as a branch of the armed forces.

• The participants who were in the Reserve or National Guard (i.e., Marine Corps Reserve) gave the main branch (i.e., Marine Corps) as their answer and did not mention the Reserve. Most of the participants said that they think of the reserves or national guard as part of the main branches of the Marine Corps, Army, Navy or Air Force, since once they are on active duty, they are no longer considered Reserves. Once they are activated, they are considered the same as the main branch.

Released from Active Duty

• Most of the participants understood that the question was referring to the last time they were on active duty (when they were discharged or when there was a break in service).

• Another comment had to do with the language “released from active duty.” One veteran commented that he liked this language, because it is inclusive of all reasons why someone would separate from service or active duty (including discharges, retirement, etc).

• Proxies said that this question referred to when they were retired or separated from the military, for various reasons, e.g., completion of initial enlistment, retirement, or voluntary leave.

Other comments

• At least one of the proxies thought that this question would be better if the multiple choice answers were read aloud, since he was not sure the level of specificity that was required. For example, he thought that the question might be asking for a specific battalion.

Recommendation

A potential problem with this question is that the reserve or national guard information would be omitted. The level of specificity prompted with this question is at the level of the main branches of the armed forces, i.e., Army, Navy, Marine or Air Force. If more specificity is wanted, i.e., Army Reserve, then there should be a follow-up question to obtain this information.

S11a. Did (you/name) EVER serve in a combat or war zone?

Overall, most of the participants answered this question without difficulty or hesitation. However, during the cognitive interview, some ambiguity was observed with this question. There was some disagreement about whether serving in a combat or war zone involved having to be outside of the base and in harms way, or serving in an area that is declared a war zone, e.g., Iraq or Afghanistan.

One participant thought that this was a touchy subject. He felt that what he considers combat or war zone might be different from what others think. He felt that combat meant being outside of safety or outside of the base. Even for those who are deployed in Iraq, he thought that some never step outside of the base and so they would not be considered being in combat. Combat meant a military capable mission or engaging with the enemy. Likewise, he did not consider Iraq to be a war zone.

In contrast, some of the other participants thought that being in active duty status in a declared combat or war zone qualified, whether or not they were necessarily actively engaged in the conflict. For them, combat or war zone had to be “declared.”

The female veteran answered that she had served in combat or war zone. Women in the armed forces do not technically engage in combat, but she reported that she served in Afghanistan and had received combat pay.

The ambiguities about what is a combat or war zone were easily cleared up when the statement about “Persons serving in a combat or war zone usually receive combat zone tax exclusion…” was given. Three participants (veterans) used this in their explanation of what is combat or war zone, unprompted. One of the proxies thought a more detailed definition would be helpful, since he was not sure about the type of pay his housemate would have received.

Proxies tended to think of combat or war zone as some place (e.g., Iraq or Afghanistan) where there is a war going on and where US Armed Forces are deployed. Veterans had more specific notions about this area, having to do with special pay. But veterans said that these are usually one and the same.

Recommendation:

Check with the sponsor to see if receiving combat pay is a sufficient definition for this question.

1 Gender refers to the participant who was interviewed.