DADT_Supporting Statement B

Supporting Statement B For:

DoD Comprehensive Review Working Group (CRWG)

on the Impact of Repealing the

“Don’t Ask, Don’t Tell” Policy

Spouse Mail Survey

3 August 2010
Revised

SPOUSE SURVEY

Table of Contents

B. Collections of information employing statistical methods 1

B.1 Respondent Universe and Sampling Methods 1

B.1.1 Mail Survey 1

B.2 Procedures for the Collection of Information 2

B.2.1 Statistical Methodology for Stratification and Sample Selection 2

B.2.2 Problems Requiring Special Sampling Procedures 8

B.2.3 Periodic Data Collection to Reduce Burden 9

B.2.4 Data Collection Procedures—Mail Survey 9

B.2.5 Estimation 11

B.3 Methods to Maximize Response Rates and Deal with Non-response 11

B.3.1 Mailing protocol 11

B.3.2 Statistical weighting 12

B.3.3 Nonresponse-bias study 12

B.4 Test of Procedures or Methods to be Undertaken 13

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 14

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1 Respondent Universe and Sampling Methods

Different approaches will be used to select respondents for different data collections. This section provides a description of the respondent universes and sampling methods for the mail survey data collection.

B.1.1 Mail Survey

Two populations of spouses will be surveyed: spouses of active duty members and spouses of Reserve and National Guard members. The target population of active duty spouses will be spouses of active duty members of the Army, Navy, Marine, Air Force and Coast Guard, up to and including pay grade O06 (captain in the Navy and Coast Guard, or colonel in the other services) with at least 6 months service as of June 15, 2010. The target population of Reserve and National Guard member spouses will be spouses of members of the Army National Guard, the Army Reserve, the Naval Reserve, the Marine Corps Reserve, the Coast Guard Reserve, the Air Force Reserve, or the Air National Guard, up to and including pay grade O06 with at least 6 months service as of June 15, 2010.

Spouses of members of the Reserve or National Guard who have been activated under authority of Title 10 or Title 32 will be included in the population of Reserve and National Guard spouses, not the population of active duty spouses. Both spouse populations will exclude spouses in dual-military marriages--that is, a marriage in which each spouse is an active duty member, Reserve, or National Guard member.

The population size for the active duty spouses is approximately 660,000 and that for spouse of Reserve and National Guard members is approximately 390,000. Data from the Defense Enrollment Eligibility Reporting System (DEERS) will be used to construct a sampling frame of the married service members corresponding to each target population. A stratified sample will be selected from each sampling frame, and then DEERS will be used a second time to obtain contact information for the spouses of the sampled service members. The expected overall response rate for the survey is 35 percent.

B.2 Procedures for the Collection of Information

This section describes procedures that will be performed before, during, and after data collection. The discussion includes stratification, sample selection, protocols for data collection, and estimation.

B.2.1 Statistical Methodology for Stratification and Sample Selection

DEERS data will be used to create a sampling frame of the married service members corresponding to each target population. DEERS contains a large number of data variables that can be used to stratify the constructed sampling frames. To determine which variables should be used to create strata, members of the DOD’s Comprehensive Review Working Group (CRWG) were consulted to determine the estimation domains of interest specified in Table B-1. DEERS data were then used to identify low-prevalence domains, and the response rates from other DOD surveys of military spouses were used to identify low-response domains. This resulted in selection of the following variables to be used to stratify the frame of married active duty members, so that low-prevalence and low-response domains can be oversampled:

• Service (5 levels: Army, Navy, Air Force, Marine Corps, and Coast Guard)

• Pay grade (5 levels to be crossed with other variables; warrant officers put in separate stratum )

• Geographic location of service member (2 levels)

• Gender of service member (2 levels)

Similarly, the following variables were selected to stratify the frame of married Reserve and National Guard members:

• Reserve component (7 levels)

• Pay grade (5 levels to be crossed with other variables; warrant officers put in separate stratum )

• Gender of service member (2 levels)

• Reserve program (3 levels to be crossed with other variables; Individual Mobilization Augmentees (IMA’s) put in separate stratum )

Table B-1. Estimation domains for spouse surveys

Domain variable	Levels of domain variable	Active duty spouses	Reserve component spouses
Member demographics Service Pay grade Service by pay grade Deployment status Location Program	Army, Navy, Marine Corps, Air Force, Coast Guard for active duty; ARNG, USAR, USNR, USMCR, ANG, USAFR, USCGR for National Guard and reserves E1-E4, E5-E9, W1-W5, O1-03, O4-O6 Same as above, except warrant officers not disaggregated by service Not deployed past 36 months, Deployed past 36 months, Not Deployed past 12 months, Deployed past 12 months U.S., Overseas (Europe, Asia and Pacific), On base, Off base Reserve unit, Reserve unit technician, Full-time support personnel, IMA	X X X X X	X X X X X
Spouse demographics Gender Race/Ethnicity Age Family status	Male, Female Non-Hispanic white, Non-Hispanic Black, Hispanic 25 years old or younger, 26-30, 31-35, 36-40, 40+ With child(ren), Without child(ren)	X X X X	X X X X

For the survey of active duty spouses there will be 73 strata and for the survey of reserve component spouses there will be 120 strata. Stratum allocations will be determined by using the Sample Design Tool developed by Research Triangle Institute (Kavee and Mason, 1997) for the Defense Manpower Data Center, based on the multivariate allocation algorithm described by Chromy (1987). A simple random sample of married service members will be selected from each stratum, yielding corresponding fielded spouse samples of approximately 70,000 active duty spouses and approximately 80,000 spouses of Reserve and National Guard members. Table B-2 displays the values of the stratification variables, population sizes, fielded sample sizes, and expected number of completes for each stratum in the survey of active-duty spouses. Table B-3 displays similar information for the survey of Reserve spouses.

Table B-2: Strata for Active Duty Spouse Survey

Stratum	Values of stratification variables	Population size	Expected completes	Fielded sample size
1	Army_E1-E4_US_Male	71,805	883	4,947
2	Army_E1-E4_US_Female	8,697	85	835
3	Army_E1-E4_Oversea_Male	9,116	113	634
4	Army_E1-E4_Oversea_Female	999	10	98
5	Army_E5-E6_US_Male	80,900	1,109	5,026
6	Army_E5-E6_US_Female	4,465	54	344
7	Army_E5-E6_Oversea_Male	10,702	150	691
8	Army_E5-E6_Oversea_Female	555	7	46
9	Army_E7-E9_US_Male	36,482	646	2,115
10	Army_E7-E9_US_Female	1,613	26	97
11	Army_E7-E9_Oversea_Male	4,329	78	259
12	Army_E7-E9_Oversea_Female	224	4	14
13	Army_O1-O3_US_Male	17,401	593	1,547
14	Army_O1-O3_US_Female	2,153	69	209
15	Army_O1-O3_Oversea_Male	1,883	65	168
16	Army_O1-O3_Oversea_Female	240	8	23
17	Army_O4-O6_US_Male	20,342	612	1,401
18	Army_O4-O6_US_Female	1,488	47	104
19	Army_O4-O6_Oversea_All	2,726	83	190
20	Navy_E1-E4_US_Male	27,777	657	3,857
21	Navy_E1-E4_US_Female	3,735	66	742
22	Navy_E1-E4_Oversea_All	1,182	28	169
23	Navy_E5-E6_US_Male	56,332	807	3,567
24	Navy_E5-E6_US_Female	3,532	45	293
25	Navy_E5-E6_Oversea_Male	4,929	70	323
26	Navy_E5-E6_Oversea_Female	292	4	25
27	Navy_E7-E9_US_Male	21,305	659	2,080
28	Navy_E7-E9_US_Female	664	20	78
29	Navy_E7-E9_Oversea_All	2,072	66	202
30	Navy_O1-O3_US_Male	12,666	619	1,604
31	Navy_O1-O3_US_Female	1,139	53	161
32	Navy_O1-O3_Oversea_All	1,074	52	139
33	Navy_O4-O6_US_Male	13,974	618	1,517
34	Navy_O4-O6_US_Female	1,136	50	128
35	Navy_O4-O6_Oversea_All	1,426	64	158
36	Marine Corps_E1-E4_US_Male	29,565	695	3,591
37	Marine Corps_E1-E4_US_Female	789	15	141
38	Marine Corps_E1-E4_Oversea_All	1,449	34	187
39	Marine Corps_E5-E6_US_Male	26,924	673	3,076
40	Marine Corps_E5-E6_US_Female	505	12	80
41	Marine Corps_E5-E6_Oversea_All	2,170	55	260
42	Marine Corps_E7-E9_US_All	10,206	636	2,116
43	Marine Corps_E7-E9_Oversea_All	1,298	81	274
44	Marine Corps_O1-O3_US_All	5,697	623	1,930
45	Marine Corps_O1-O3_Oversea_All	483	53	166
46	Marine Corps_O4-O6_US_All	4,846	599	1,697
47	Marine Corps_O4-O6_Oversea_All	560	70	199
48	Air Force_E1-E4_US_Male	22,845	566	2,425
49	Air Force_E1-E4_US_Female	3,271	69	431
50	Air Force_E1-E4_Oversea_Male	4,056	103	442
51	Air Force_E1-E4_Oversea_Female	372	8	50
52	Air Force_E5-E6_US_Male	44,057	538	2,027
53	Air Force_E5-E6_US_Female	4,129	53	230
54	Air Force_E5-E6_Oversea_Male	10,673	139	526
55	Air Force_E5-E6_Oversea_Female	791	11	46
56	Air Force_E7-E9_US_Male	17,768	564	1,782
57	Air Force_E7-E9_US_Female	1,272	42	129
58	Air Force_E7-E9_Oversea_All	4,315	139	437
59	Air Force_O1-O3_US_Male	14,275	580	1,447
60	Air Force_O1-O3_US_Female	1,741	71	185
61	Air Force_O1-O3_Oversea_Male	1,700	70	175
62	Air Force_O1-O3_Oversea_Female	204	9	24
63	Air Force_O4-O6_US_Male	17,443	592	1,508
64	Air Force_O4-O6_US_Female	1,600	58	134
65	Air Force_O4-O6_Oversea_Male	2,346	80	205
66	Air Force_O4-O6_Oversea_Female	234	9	21
67	Coast Guard_E1-E4_US_Male	3,850	159	694
68	Coast Guard_E1-E4_US_Female	371	12	95
69	Coast Guard_E5-E6_All_All	9,114	400	1,434
70	Coast Guard_E7-E9_All_All	3,442	156	500
71	Coast Guard_O1-O3_All_All	1,889	473	1,260
72	Coast Guard_O4-O6_All_All	2,161	551	1,420
73	Warrant Officers	15,820	1,387	4,851

Table B-3: Strata for Reserve Spouse Survey

Stratum	Values of stratification variables	Population size	Expected completes	Fielded sample size
1	ANG__E1-E4_Male_Only___TPU_Only____	35,425	781	4,568
2	ANG__E1-E4_Male_Only___AGR_Only____	217	5	35
3	ANG__E1-E4_Male_Only___MilTech_Only	1,140	26	151
4	ANG__E1-E4_Female_Only_TPU_Only____	3,953	77	589
5	ANG__E1-E4_Female_Only_AGR-Mil_Only	272	6	40
6	ANG__E5-E6_Male_Only___TPU_Only____	42,450	1,107	4,626
7	ANG__E5-E6_Male_Only___AGR_Only____	6,068	149	753
8	ANG__E5-E6_Male_Only___MilTech_Only	5,178	137	564
9	ANG__E5-E6_Female_Only_TPU_Only____	2,291	56	274
10	ANG__E5-E6_Female_Only_AGR_Only____	653	16	87
11	ANG__E5-E6_Female_Only_MilTech_Only	451	11	54
12	ANG__E7-E9_Male_Only___TPU_Only____	12,213	377	1,131
13	ANG__E7-E9_Male_Only___AGR_Only____	7,098	212	745
14	ANG__E7-E9_Male_Only___MilTech_Only	3,303	103	305
15	ANG__E7-E9_Female_Only_TPU_Only____	384	12	37
16	ANG__E7-E9_Female_Only_AGR_Only____	533	15	59
17	ANG__E7-E9_Female_Only_MilTech_Only	241	8	22
18	ANG__O1-O3_Male_Only___TPU_Only____	8,670	375	1,196
19	ANG__O1-O3_Male_Only___AGR_Only____	999	41	159
20	ANG__O1-O3_Male_Only___MilTech_Only	573	25	79
21	ANG__O1-O3_Female_Only_NonIMA_	862	35	128
22	ANG__O4-O6_Male_Only___TPU_Only____	5,307	282	739
23	ANG__O4-O6_Male_Only___AGR_Only____	2,410	122	380
24	ANG__O4-O6_Male_Only___MilTech_Only	953	51	132
25	ANG__O4-O6_Female_Only_NonIMA_	492	25	75
26	AR___E1-E4_Male_Only___TPU_Only____	13,910	381	2,434
27	AR___E1-E4_Male_Only___AGR-Mil_Only	360	10	71
28	AR___E1-E4_Female_Only_NonIMA_	4,148	108	771
29	AR___E5-E6_Male_Only___TPU_Only____	19,268	493	2,233
30	AR___E5-E6_Male_Only___AGR_Only____	2,282	55	325
31	AR___E5-E6_Male_Only___MilTech_Only	1,473	38	171
32	AR___E5-E6_Female_Only_TPU_Only____	2,717	66	338
33	AR___E5-E6_Female_Only_AGR_Only____	365	8	57
34	AR___E5-E6_Female_Only_MilTech_Only	411	11	55
35	AR___E7-E9_Male_Only___TPU_Only____	8,757	292	959
36	AR___E7-E9_Male_Only___AGR_Only____	3,942	129	524
37	AR___E7-E9_Male_Only___MilTech_Only	1,268	43	143
38	AR___E7-E9_Female_Only_TPU_Only____	933	31	105
39	AR___E7-E9_Female_Only_AGR_Only____	593	19	86
40	AR___E7-E9_Female_Only_MilTech_Only	278	9	32
41	AR___O1-O3_Male_Only___TPU_Only____	5,572	329	1,122
42	AR___O1-O3_Male_Only___AGR-Mil_Only	698	39	163
43	AR___O1-O3_Female_Only_NonIMA_	1,660	92	360
44	AR___O4-O6_Male_Only___TPU_Only____	7,579	285	787
45	AR___O4-O6_Male_Only___AGR_Only____	1,728	63	213
46	AR___O4-O6_Male_Only___MilTech_Only	262	10	28
47	AR___O4-O6_Female_Only_TPU_Only____	1,218	44	134
48	AR___O4-O6_Female_Only_AGR-Mil_Only	211	8	29
49	NR___E1-E4_Male_Only___TPU_Only____	3,314	291	2,014
50	NR___E1-E4_Male_Only___AGR_Only____	653	42	653
51	NR___E1-E4_Female_Only_TPU-AGR_Only	922	77	607
52	NR___E5-E6_Male_Only___TPU_Only____	12,021	354	1,616
53	NR___E5-E6_Male_Only___AGR_Only____	2,518	46	693
54	NR___E5-E6_Female_Only_TPU_Only____	1,578	45	218
55	NR___E5-E6_Female_Only_AGR_Only____	430	12	85
56	NR___E7-E9_Male_Only___TPU_Only____	2,560	314	963
57	NR___E7-E9_Male_Only___AGR_Only____	906	80	610
58	NR___E7-E9_Female_Only_TPU-AGR_Only	369	44	151
59	NR___O1-O3_Male_Only___TPU_Only____	2,064	302	1,089
60	NR___O1-O3_Male_Only___AGR_Only____	190	25	137
61	NR___O1-O3_Female_Only_TPU-AGR_Only	292	41	162
62	NR___O4-O6_Male_Only___TPU_Only____	5,280	326	906
63	NR___O4-O6_Male_Only___AGR_Only____	853	42	231
64	NR___O4-O6_Female_Only_TPU-AGR_Only	698	42	126
65	MCR__E1-E4_Male_Only___TPU-AGR_Only	4,199	357	4,199
66	MCR__E1-E4_Female_Only_TPU-AGR_Only	237	16	237
67	MCR__E5-E6_AllGen_TPU-AGR_Only	2,963	317	2,963
68	MCR__E7-E9_AllGen_TPU-AGR_Only	1,129	288	1,129
69	MCR__O1-O3_AllGen_TPU-AGR_Only	275	75	275
70	MCR__O4-O6_AllGen_TPU-AGR_Only	903	424	903
71	AFNG_E1-E4_Male_Only___TPU_Only____	3,835	409	2,264
72	AFNG_E1-E4_Male_Only___AGR-Mil_Only	337	37	201
73	AFNG_E1-E4_Female_Only_NonIMA_	835	85	526
74	AFNG_E5-E6_Male_Only___TPU_Only____	13,373	361	1,411
75	AFNG_E5-E6_Male_Only___AGR_Only____	2,546	65	310
76	AFNG_E5-E6_Male_Only___MilTech_Only	5,053	139	526
77	AFNG_E5-E6_Female_Only_TPU_Only____	1,508	40	169
78	AFNG_E5-E6_Female_Only_AGR_Only____	319	8	40
79	AFNG_E5-E6_Female_Only_MilTech_Only	332	9	38
80	AFNG_E7-E9_Male_Only___TPU_Only____	5,216	192	557
81	AFNG_E7-E9_Male_Only___AGR_Only____	3,341	118	400
82	AFNG_E7-E9_Male_Only___MilTech_Only	6,043	225	635
83	AFNG_E7-E9_Female_Only_TPU_Only____	480	18	56
84	AFNG_E7-E9_Female_Only_AGR_Only____	654	22	85
85	AFNG_E7-E9_Female_Only_MilTech_Only	385	14	43
86	AFNG_O1-O3_Male_Only___TPU_Only____	1,938	325	1,087
87	AFNG_O1-O3_Male_Only___AGR_Only____	292	46	184
88	AFNG_O1-O3_Male_Only___MilTech_Only	392	67	220
89	AFNG_O1-O3_Female_Only_NonIMA_	377	60	224
90	AFNG_O4-O6_Male_Only___TPU_Only____	3,295	278	762
91	AFNG_O4-O6_Male_Only___AGR_Only____	1,502	120	389
92	AFNG_O4-O6_Male_Only___MilTech_Only	1,344	114	312
93	AFNG_O4-O6_Female_Only_NonIMA_	507	41	124
94	AFR__E1-E4_Male_Only___NonIMA_	2,650	335	2,198
95	AFR__E1-E4_Female_Only_NonIMA_	840	100	733
96	AFR__E5-E6_Male_Only___TPU_Only____	8,050	297	1,362
97	AFR__E5-E6_Male_Only___AGR_Only____	358	13	76
98	AFR__E5-E6_Male_Only___MilTech_Only	1,938	73	324
99	AFR__E5-E6_Female_Only_TPU_Only____	1,185	42	209
100	AFR__E5-E6_Female_Only_AGR-Mil_Only	192	7	34
101	AFR__E7-E9_Male_Only___TPU_Only____	4,372	235	756
102	AFR__E7-E9_Male_Only___AGR_Only____	513	28	107
103	AFR__E7-E9_Male_Only___MilTech_Only	2,171	117	376
104	AFR__E7-E9_Female_Only_TPU_Only____	586	31	108
105	AFR__E7-E9_Female_Only_AGR-Mil_Only	342	18	65
106	AFR__O1-O3_Male_Only___NonIMA_	1,234	264	900
107	AFR__O1-O3_Female_Only_NonIMA_	243	50	185
108	AFR__O4-O6_Male_Only___TPU_Only____	2,823	142	389
109	AFR__O4-O6_Male_Only___AGR_Only____	562	28	90
110	AFR__O4-O6_Male_Only___MilTech_Only	783	39	111
111	AFR__O4-O6_Female_Only_NonIMA_	539	26	78
112	CGR__E1-E4_AllGen_TPU_Only____	765	61	391
113	CGR__E5-E6_Male_Only___TPU_Only____	1,419	134	616
114	CGR__E5-E6_Female_Only_TPU_Only____	220	20	103
115	CGR__E7-E9_AllGen_TPU_Only____	759	86	282
116	CGR__O1-O3_AllGen_TPU_Only____	464	213	464
117	CGR__O4-O6_AllGen_TPU_Only____	410	216	410
118	Only W1-W5 excluding IMAs	7,720	732	2,275
119	Only IMAs excluding W1-W5	8,944	2,585	8,944
120	Only IMAs AND W1-W5______	141	4	16

The size of the selected samples will be such that the expected maximum margin of errors (i.e., half widths of 95% confidence intervals) for proportions estimated for the domains specified in Table B‑1 are less than or equal to 5%. Stratum-specific response rates observed in 2008 surveys of military spouses will be used to determine the stratum allocations. The overall response rate for the 2008 Active Duty Spouses Survey was 28% and that for the 2008 Reserve Component Spouse Survey was 30%. Due to the subject matter, the response rates for the proposed surveys are expected to be higher than those for the 2008 surveys. The burden estimates in A-1 are based on overall response rates of 35% for each of the proposed spouse surveys.

B.2.2 Problems Requiring Special Sampling Procedures

Some of the domains of interest have very low prevalence. For example, the proportion of married reservists who are Individual Mobilization Augmentees (IMA’s) is less than 3% of all married reservists. Warrant officers are also a low-prevalence domain of interest. To satisfy the precision goal of constraining the margins of error for domains to be less than or equal to 5%, IMA’s and warrant officers will be assigned to their own strata and then oversampled.

B.2.3 Periodic Data Collection to Reduce Burden

This request is for a one-time data collection.

B.2.4 Data Collection Procedures Mail Survey

The data collection protocol for the Spouse survey is summarized below. This study will use a number of methods to maximize the rate of response and data quality. First, the survey will make multiple contacts with the Active Duty and Reserve Component spouse, following a Total Design Method recommended by Dillman, et al. (2008). This protocol has been designed to achieve the desired response rate of 35% through the five separate mailing activities described below.

Pre-notification Letter. An 8.5 x 11 inch pre-notification letter will be mailed to all sampled Spouses (using first-class postage) that announces the study, describes the information to be collected, and when the data collection will begin. A toll-free telephone number (and/or an email address) will be provided so respondents can make inquiries about the study. We will use letterhead and signatures that respondents will recognize and feel are important. Logos, emblems, other relevant artwork, and the use of up to two colors will be incorporated to increase awareness and distinguish our correspondence from other mail. Each letter will be personalized and inserted into a windowed number 10 business envelope.

Mail-based, self-administered questionnaire packet. This survey packet, mailed to all sample members (about one week after the pre-notification letter was mailed) using first-class postage, will include a 16-page survey booklet, an integrated cover letter, and a postage-paid business reply envelope. The sample member’s name and address information printed on the survey booklet will show through a window on the outer/carrier envelope. Surveys and their outer envelopes, should take advantage of the same logos, emblems, artwork and colors used for the pre-notification letter. An identification number (e.g. barcode) will be printed on the survey booklet for tracking respondents allowing for follow-up activities to non-respondents.

First Thank You/Reminder postcard. A standard (4x6 inch) sized postcard will be mailed to all sample members approximately 14 days after mailing the survey packet. Content of the postcard will not only thank those who have already participated in the survey, but remind those who have not yet responded to do so. Postcards will be mailed using first-class postage.

Second self-administered questionnaire packet. Identical in format and in content (with the exception of the language used in the cover letter) to the first questionnaire packet, we will mail a second questionnaire to all survey non-respondents about 1 month following the initial survey mailing. We suggest using stronger language in the cover letter that accompanies this second questionnaire to encourage participation. We anticipate mailing this packet to approximately 85% of the sample. Some completed surveys will be in-transit at the time of this mailing, so some sample members will receive a second survey despite having completed and returned their first survey. This circumstance is unavoidable given the intended data collection schedule.

Final Thank You/Reminder postcard. A second thank you/reminder postcard will be mailed only to non-respondents about two weeks after the second survey packet is mailed. This postcard, much like the cover letter used for the second survey packet cover letter, will use stronger language than previous correspondence to encourage participation aimed at increasing overall response rates. Again, the card will thank those who have already participated in the survey, but remind those who have not yet responded to do so immediately. Postcards will be mailed using first-class postage.

Survey Receipt and Processing

Sample management and receipt control systems are vital components of effective management and monitoring of the data collection process for this survey. Maintaining current and accurate sample management information prevents costly missteps and minimizes errors. A complete record of all data transactions and updates to ensure that the data are accurate and available during all phases of the survey administration will be maintained.

Returned mail surveys will be logged into the receipt control system on a continuous basis and then be prepared for data capture via TeleForm. Our receipt control and sample management systems provide real-time progress reports and are generated through a web-based interface.

After receipting returned surveys, the booklets are scanned and processed with TeleForm, they will be exported to a SQL Server database. Returned surveys will be stored securely in locked file cabinets within the enclave workspace. Electronic form images from each scanned individual paper survey page will be stored in Alchemy, an image database and retrieval system. This central digital archive will be used throughout data collection to retrieve and view images of the paper surveys if needed.

B.2.5 Estimation

Though there is some interest is using the survey data to calculate population estimates, but there is more interest in using the survey data to calculate sub-population estimates and differences among sub-population estimates. Section B.3.2 describes the procedure that will be used to calculate sampling weights, which will be used to compute weighted totals and proportions and will also be used in the multivariate analyses described in Section A.16.1. For the sub-populations specified in Table B-1, the half widths of the 95% confidence intervals for estimated proportions will be less than or equal to 5% and for differences among sub-population proportion estimates, the half width of the 95% confidence intervals will be less than or equal to 7%.

B.3 Methods to Maximize Response Rates and Deal with Non-Response

For the mail survey, the expected overall response rate is expected to be 35% for both spouses of active duty service members and spouse of reserve component service members. The mail survey data collection protocol will employ techniques to maximize response rates and address non-response. Following data collection, statistical weighting will be used to decrease non-response bias and a nonresponse bias study will be conducted.

B.3.1 Mail Protocol

To maximize the effectiveness of mail surveys the following design and process elements will be utilized throughout the field period:

• Survey booklets and outer envelopes will be printed in two-color ink and include visual elements (logo’s, artwork developed for the Don’t Ask, Don’t Tell survey, etc.) that “link” printed pieces together.

• A machine- and human-readable code will be printed on survey booklets for tracking and quality control purposes.

• Letters will be personalized and include the critical information about the study.

• The assembly process will be simplified and quality control improved by using windowed envelopes (obviating the need to match personalized letter with printed address label).

• Mailings will be sent via USPS using first-class postage ensuring timely and accurate delivery and prompt return of undeliverable mailing pieces.

• Users can get survey support by contacting staff using our toll-free telephone line and by email. This information will be provided in all mailings.

.

Steps to minimize nonresponse are also built into the mail study protocol. These include the following:

• Survey Advance Letters. Advance materials will be sent to all households. The advance letters will describe the study’s goals and objectives and will give assurances of confidentiality. Letters will be sent to households approximately 1 week before the household is mailed the survey.

• Multiple Followup for the Mail Survey. If a survey is not received from a designated household 2 weeks after they are sent, a postcard reminder will be sent. If a survey has not been received 2 weeks after the postcard, a final remailing of the surveys will be sent.

B.3.2 Statistical Weighting

Sample weights will be calculated for each completed questionnaire to allow for unbiased estimates of population and sub-population proportions. The sample weights are products of the base weight, non-response adjustments, and a post-stratification adjustment. The base weight is the reciprocal of the probability of selection of each married service member. The non-response adjustments are designed to reduce the potential bias caused by differences between the responding and non-responding population and are equal to the reciprocals of weighted response rates within carefully selected response cells. The non-response adjustment cells will be constructed by starting with the sampling strata described in Tables B-2 and B-3. Some of these strata are expected to have a small number of respondents. When this occurs, strata having similar response rates will be combined to create nonresponse adjustment cells such that the resulting cells contains no less than 30 respondents.

The post-stratification adjustment modifies the non-response-adjusted base weights so that they aggregate to demographic totals computed from DEERS data by the Defense Manpower Data Center. This adjustment has the effect of reducing variance. Post-stratification adjustments will be performed by raking adjusted weights so that they add to control totals for each level of variables associated with the domains listed in Table B-1. For the Active Duty Spouse Survey, the raking dimensions will be service by pay grade, current-deployment status, location, gender, race and ethnicity, age category, and family status. All of these variables are associated with the military member. (Table B-1 indicates that he domains of interest for age and for race and ethnicity are the spouse’s age and the spouse’s race and ethnicity, not the military member’s, but control totals are not available for spouse variables.) The same raking dimensions will be used for the Reserve Spouse Survey, except instead of location the variable for Reserve program will be used.

B.3.3 Nonresponse-Bias Study

When unadjusted sample weights are used to estimate the population mean for some item, the non-response bias present in the resulting estimator is equal to the product of the nonresponse rate and the difference between the average value of the item for respondents and the average value of the item for nonrespondents. Hence, nonresponse rates are one measure of potential nonresponse bias. After the completion of data collection and the calculation of the sampling weights, base-weighted response rates will be calculated for each sample survey and for the domains specified in Table B-1. AAPOR Response Rate Formula RR3 will be used to calculate response rates.

To estimate the difference between the average value of an item for respondents and the average value of the item for nonrespondents, it is necessary to have data for the item for all the units in a sample, not just the responding units. Thus, administrative data present on the sample file, along with the base weights, will be used to estimate the difference in population means of respondents and nonrespondents for the available administrative-data variables. This will allow the estimation of non-response biases present in estimates of proportions calculated with unadjusted weights. This type of analysis will be performed on the following variables:

1. Variables used in stratification (and the creation of nonresponse adjustment cells),

2. Variables used in raking,

3. Additional variables:

• Whether military member is living on base,

• Military member’s occupation area,

• Military member’s career deployments

• Whether military member deployed in last 12 months,

• Whether military member deployed in last 24 months

• Whether military member deployed in last 36 months

Another way to estimate nonresponse biases is to compute population means from all the data on the sampling frame and then subtract this from weighted means calculated from the unadjusted weights and the administrative data associated with the respondents. The reductions in nonresponse bias resulting from adjusting the survey weights will be determined by using the adjusted weights to repeat the estimation of the nonresponse biases present in the above administrative data variables.

B.4 Test of Procedures or Methods to be Undertaken

Testing of survey questions to understand respondent comprehension, recall methods and judgment and estimation processes will be done with active military and reserve personnel prior to the finalization of the instrument. There are no planned cognitive tests with active duty or reserve spouses.

B.5 Individuals Consulted on Statistical Aspects and/or Analyzing Data

The individuals consulted on technical and statistical issues related to the data collection are listed below.

DoD has consulted with the following staff at Westat regarding this information collection:

Shelley Perry, Ph.D.

Associated Director, Westat

Phone: (301) 251-4209 Email: [email protected]

Kimya Lee, Ph.D.

Senior Study Director, Westat

Phone: (301) 610-5522 Email: [email protected]

Susan Berkowitz, Ph.D.

Senior Study Director, Westat

Phone: (301) 294-3936 Email: [email protected]

Wayne Hintze, MA.

Senior Study Director, Westat

Phone: (301) 517-4022 Email: [email protected]

Richard Sigman, M.A.

Senior Statistician, Westat

Phone: (240) 453-2783 Email: [email protected]

On DoD’s side, those consulted in the development process included:

David E. McGrath

Chief, Personnel Survey Branch

Department of Defense Manpower Data Center (DMDC)

Phone: (703)-696-2675 Email: [email protected]

Fawzi Al Nassir

Department of Defense Manpower Data Center (DMDC)

Phone: 703-696-5825 Email: [email protected]

References:

Chromy, James R, (1987). Design Optimization with Multiple Objectives, Proceedings of the Section on Survey Research Methods, American Statistical Association, Alexandria, VA, pp. 194-199.

Kavee, Jill D. and Mason, Robert E (1997). DMDC Sample Planning tools User’s Manual (Version 1.2), Defense Manpower Data Center, Arlington, VA.

i