2021ADSS_Supporting_Statement_Part_B

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

If the collection of information employs statistical methods, it should be indicated in Item 17 of OMB Form 83-I, and the following information should be provided in this Supporting Statement:

1.  Description of the Activity

Describe the potential respondent universe and any sampling or other method used to select respondents.  Data on the number of entities covered in the collection should be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.  Indicate the expected response rates for the collection as a whole, as well as the actual response rates achieved during the last collection, if previously conducted.

The population of interest for the 2021 ADSS consists of spouses of active duty members from the Army, Navy, Marine Corps, and Air Force who are below flag rank. In addition, for the spouse to remain eligible for the survey, they must indicate they are currently married to an active duty member at the time of the survey. OPA uses a sampling tool developed by the Research Triangle Institute (RTI) to determine the sample size needed to achieve 95% confidence and an associated precision of 5% or less on each reporting domain. We select a single-stage, non-proportional stratified random sample to ensure statistically adequate expected number of responses for the reporting categories (i.e., domains). For the active duty spouse survey, OPA uses Service, paygrade, gender, and family status to define the initial strata. We collapse these strata when there are fewer than 200 individuals in the stratum and there are 80 final strata. Attachment A contains a table with the number of individuals in the population and sample by strata. The expected weighted response rate for this survey is about 16%; the weighted response rate for this survey in 2019 was 16.3%.

2.  Procedures for the Collection of Information

Describe any of the following if they are used in the collection of information:

a.  Statistical methodologies for stratification and sample selection;

As described above, OPA uses a sampling tool developed by the Research Triangle Institute (RTI) to determine the sample size needed to achieve 95% confidence and an associated precision of 5% or less on each reporting category domain. We select a single-stage, non-proportional stratified random sample to ensure statistically adequate expected number of responses for the reporting domains. For the active duty spouse survey, OPA uses Service, paygrade, gender, and family status to define the initial strata. We collapse these strata when there are fewer than 200 individuals in the stratum. Once OPA determines the stratum-level sample sizes, a random number is assigned to every member of the population and the population is sorted by stratum and random number prior to sampling, which results in a randomly-ordered population within each stratum. We then select the appropriate number of spouses from each stratum.

b.  Estimation procedures;

OPA weights the eligible respondents in order to make inferences about the entire population of active duty spouses. The weighting methodology utilizes standard weighting processes. First, we assign a base weight to the sampled member based on the reciprocal of the selection probability. Second, OPA uses 20-30 administrative variables in the XGBoost application of Generalized Boosted Model (GBM) to predict survey eligibility and completion. OPA’s accurate and detailed administrative data on both survey respondents and nonrespondents provides confidence in our survey estimates. We adjust the sampling weights and then all prior-stage weights by the inverse of these model-predicted probabilities to adjust for nonresponse. Finally, we rake these adjusted weights to known population totals to further reduce the variance and bias of the estimates.

c.  Degree of accuracy needed for the Purpose discussed in the justification;

OPA creates variance strata so precision measures can be associated with each estimate. We produce precision measures for reporting categories using 95% confidence intervals with the goal of achieving a precision of 5% or less (e.g, 80% (+/- 5%) of spouses of Army E1-E4 are satisfied with their job).

d.  Unusual problems requiring specialized sampling procedures; and

OPA recognizes the response rates vary for certain domains of interest such as Service and paygrade. To account for this, we average the response rates for the previous three surveys at the stratum level and these response rates are utilized by the sampling tool to adjust the sample and compute expected sample sizes.

e.  Use of periodic or cyclical data collections to reduce respondent burden.

OPA conducts the ADSS survey every other year to reduce respondent burden.

3.  Maximization of Response Rates, Non-response, and Reliability

Discuss methods used to maximize response rates and to deal with instances of non-response.  Describe any techniques used to ensure the accuracy and reliability of responses is adequate for intended purposes.  Additionally, if the collection is based on sampling, ensure that the data can be generalized to the universe under study.  If not, provide special justification.

To maximize response rates, OPA offers the survey via the Web as well as a paper survey option. Reminder letters, emails, and phone calls to nonrespondents are used to maximize response rates. To reduce respondent burden, web-based surveys use “smart skip” technology to ensure respondents only answer questions that are applicable to them. To deal with instances of nonresponse, OPA adjusts for nonresponse in the weighting methodology. To ensure the accuracy and reliability of responses, OPA conducts a nonresponse bias analysis every other survey cycle and will conduct one for the 2021 survey. OPA uses probability sampling and appropriate weighting to ensure the survey data can be generalized to the universe under study.

4.  Tests of Procedures

Describe any tests of procedures or methods to be undertaken.  Testing of potential respondents (9 or fewer) is encouraged as a means of refining proposed collections to reduce respondent burden, as well as to improve the collection instrument utility.  These tests check for internal consistency and the effectiveness of previous similar collection activities.

Not applicable.

5.  Statistical Consultation and Information Analysis

a. Provide names and telephone number of individual(s) consulted on statistical aspects of the design.

Mr. David McGrath, Branch Chief; Statistical Methods Team, Methods, Analysis, and Systems Support, Office of People Analytics (OPA); (571) 372-0983.

Ms. Wendy Barboza, Team Lead; Statistical Methods Team, Methods, Analysis, and Systems Support, Office of People Analytics (OPA); (571) 372-1099.

b. Provide name and organization of person(s) who will actually collect and analyze the collected information.

The data will be collected by Data Recognition Corporation (DRC), which is OPA’s operations contractor. Ms. Valerie Waller is the Senior Managing Director at DRC.

The data will be analyzed by OPA analysts. Robin Myers, Jason Haynes, and Amy Campbell are the lead operations analysts.

Attachment A. ADSS 2101 - Population and Sample Size by Strata

Stratum	Stratum Definitions	Population Size	Sample Size
All	Total	665,074	65,818
1	001 ARMY_E1-E4+E0_MALE+UNK_MAR+CHILD_	23,301	2,635
2	002 ARMY_E1-E4+E0_MALE+UNK_MARNOCHILD_	23,287	2,436
3	003 ARMY_E1-E4+E0_FEMALE_MAR+CHILD_	4,462	615
4	004 ARMY_E1-E4+E0_FEMALE_MARNOCHILD_	7,329	1,036
5	005 ARMY_E5-E6_MALE+UNK_MAR+CHILD_	55,935	2,721
6	006 ARMY_E5-E6_MALE+UNK_MARNOCHILD_	21,464	963
7	007 ARMY_E5-E6_FEMALE_MAR+CHILD_	5,512	337
8	008 ARMY_E5-E6_FEMALE_MARNOCHILD_	4,966	291
9	009 ARMY_E7-E9_MALE+UNK_MAR+CHILD_	34,129	2,438
10	010 ARMY_E7-E9_MALE+UNK_MARNOCHILD_	4,364	297
11	011 ARMY_E7-E9_FEMALE_MAR+CHILD_	2,551	225
12	012 ARMY_E7-E9_FEMALE_MARNOCHILD_	889	77
13	013 ARMY_W1-W5+W0_MALE+UNK_MAR+CHILD_	9,401	136
14	014 ARMY_W1-W5+W0_MALE+UNK_MARNOCHILD_	1,602	21
15	015 ARMY_W1-W5+W0_FEMALE_MARCHILD+NOCHILD_	866	20
16	016 ARMY_O1-O3+O0_MALE+UNK_MAR+CHILD_	11,858	986
17	017 ARMY_O1-O3+O0_MALE+UNK_MARNOCHILD_	7,879	617
18	018 ARMY_O1-O3+O0_FEMALE_MAR+CHILD_	1,625	163
19	019 ARMY_O1-O3+O0_FEMALE_MARNOCHILD_	2,812	253
20	020 ARMY_O4-O6_MALE+UNK_MAR+CHILD_	18,869	1,317
21	021 ARMY_O4-O6_MALE+UNK_MARNOCHILD_	2,920	198
22	022 ARMY_O4-O6_FEMALE_MAR+CHILD_	2,435	190
23	023 ARMY_O4-O6_FEMALE_MARNOCHILD_	1,042	79
24	024 NAVY_E1-E4+E0_MALE+UNK_MAR+CHILD_	7,875	1,359
25	025 NAVY_E1-E4+E0_MALE+UNK_MARNOCHILD_	16,557	2,651
26	026 NAVY_E1-E4+E0_FEMALE_MAR+CHILD_	2,633	552
27	027 NAVY_E1-E4+E0_FEMALE_MARNOCHILD_	6,666	1,319
28	028 NAVY_E5-E6_MALE+UNK_MAR+CHILD_	40,219	2,171
29	029 NAVY_E5-E6_MALE+UNK_MARNOCHILD_	19,892	984
30	030 NAVY_E5-E6_FEMALE_MAR+CHILD_	5,428	366
31	031 NAVY_E5-E6_FEMALE_MARNOCHILD_	4,907	295
32	032 NAVY_E7-E9_MALE+UNK_MAR+CHILD_	21,098	2,143
33	033 NAVY_E7-E9_MALE+UNK_MARNOCHILD_	3,316	311
34	034 NAVY_E7-E9_FEMALE_MAR+CHILD_	1,756	216
35	035 NAVY_E7-E9_FEMALE_MARNOCHILD_	620	71
36	036 NAVY_W1-W5+W0_MALE+FEMALE_MARCHILD+NOCHILD_	1,647	33
37	037 NAVY_O1-O3+O0_MALE+UNK_MAR+CHILD_	7,687	962
38	038 NAVY_O1-O3+O0_MALE+UNK_MARNOCHILD_	5,435	601
39	039 NAVY_O1-O3+O0_FEMALE_MAR+CHILD_	1,117	163
40	040 NAVY_O1-O3+O0_FEMALE_MARNOCHILD_	1,820	243
41	041 NAVY_O4-O6_MALE+UNK_MAR+CHILD_	13,090	1,168
42	042 NAVY_O4-O6_MALE+UNK_MARNOCHILD_	2,392	206
43	043 NAVY_O4-O6_FEMALE_MAR+CHILD_	1,589	154
44	044 NAVY_O4-O6_FEMALE_MARNOCHILD_	771	81
45	045 USMC_E1-E4+E0_MALE+UNK_MAR+CHILD_	4,754	1,483
46	046 USMC_E1-E4+E0_MALE+UNK_MARNOCHILD_	14,104	3,935
47	047 USMC_E1-E4+E0_FEMALE_MAR+CHILD_	588	268
48	048 USMC_E1-E4+E0_FEMALE_MARNOCHILD_	2,148	743
49	049 USMC_E5-E6_MALE+UNK_MAR+CHILD_	14,297	2,612
50	050 USMC_E5-E6_MALE+UNK_MARNOCHILD_	8,338	1,428
51	051 USMC_E5-E6_FEMALE_MAR+CHILD_	893	199
52	052 USMC_E5-E6_FEMALE_MARNOCHILD_	1,039	228
53	053 USMC_E7-E9_MALE+UNK_MAR+CHILD_	10,162	2,773
54	054 USMC_E7-E9_MALE+UNK_MARNOCHILD_	1,194	304
55	055 USMC_E7-E9_FEMALE_MARCHILD+NOCHILD_	550	183
56	056 USMC_W1-W5+W0_MALE+FEMALE_MARCHILD+NOCHILD_	1,909	90
57	057 USMC_O1-O3+O0_MALE+UNK_MAR+CHILD_	2,965	951
58	058 USMC_O1-O3+O0_MALE+UNK_MARNOCHILD_	2,764	822
59	059 USMC_O1-O3+O0_FEMALE_MARCHILD+NOCHILD_	463	166
60	060 USMC_O4-O6_MALE+FEMALE_MARCHILD+NOCHILD_	5,928	1,765
61	061 USAF_E1-E4+E0_MALE+UNK_MAR+CHILD_	9,719	1,175
62	062 USAF_E1-E4+E0_MALE+UNK_MARNOCHILD_	17,578	2,137
63	063 USAF_E1-E4+E0_FEMALE_MAR+CHILD_	2,407	396
64	064 USAF_E1-E4+E0_FEMALE_MARNOCHILD_	7,294	984
65	065 USAF_E5-E6_MALE+UNK_MAR+CHILD_	38,106	1,881
66	066 USAF_E5-E6_MALE+UNK_MARNOCHILD_	18,594	875
67	067 USAF_E5-E6_FEMALE_MAR+CHILD_	5,732	341
68	068 USAF_E5-E6_FEMALE_MARNOCHILD_	5,099	276
69	069 USAF_E7-E9_MALE+UNK_MAR+CHILD_	20,524	1,686
70	070 USAF_E7-E9_MALE+UNK_MARNOCHILD_	3,333	261
71	071 USAF_E7-E9_FEMALE_MAR+CHILD_	3,699	352
72	072 USAF_E7-E9_FEMALE_MARNOCHILD_	1,199	103
73	073 USAF_O1-O3+O0_MALE+UNK_MAR+CHILD_	8,416	741
74	074 USAF_O1-O3+O0_MALE+UNK_MARNOCHILD_	7,048	573
75	075 USAF_O1-O3+O0_FEMALE_MAR+CHILD_	1,592	165
76	076 USAF_O1-O3+O0_FEMALE_MARNOCHILD_	2,880	250
77	077 USAF_O4-O6_MALE+UNK_MAR+CHILD_	16,894	1,096
78	078 USAF_O4-O6_MALE+UNK_MARNOCHILD_	3,083	191
79	079 USAF_O4-O6_FEMALE_MAR+CHILD_	2,578	201
80	080 USAF_O4-O6_FEMALE_MARNOCHILD_	1,190	84

8