0704-sagr_ssb_9.29.21

SUPPORTING STATEMENT – PART B

1.  Description of the Activity

Describe the potential respondent universe and any sampling or other method used to select respondents.  Data on the number of entities covered in the collection should be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.  Indicate the expected response rates for the collection as a whole, as well as the actual response rates achieved during the last collection, if previously conducted.

The target population of the 2022 SAGR will consist of all students at the Military Service Academies (MSAs): U.S. Military Academy (USMA), U.S. Naval Academy (USNA), and U.S. Air Force Academy (USAFA), including the Preparatory Schools. Excluded are Service Academy Students who are (1) non-citizens and (2) are visiting from another MSA. Students under 18 years of age are also excluded. Working with the MSAs, we estimate the approximate numbers of cadets and midshipmen to be 14,200. The survey will be administered to all cadets/midshipmen (i.e., a census). Based on the 2018 SAGR survey that had a 73% response rate, we estimate a 70% response rate. To achieve sufficient statistical analytical power, we will include a census of the population of interest in the study to achieve sufficient coverage. A census allows OPA to generate reliable estimates of rare events among a small population (e.g., Unwanted Sexual Contact Rate for Women at USMA, etc.)

2.  Procedures for the Collection of Information

Describe any of the following if they are used in the collection of information:

1. Statistical methodologies for stratification and sample selection;

Stratification for sample selection does not apply for a census study.

2. Estimation procedures;

Data will be weighted, using an industry standard process, to reflect each Academy’s population as of the time of the survey in spring 2022. Weighting produces survey estimates of population totals, proportions, and means (as well as other statistics) that are representative of their respective populations. Unweighted survey data, in contrast, are likely to produce biased estimates of population statistics. The standard process of weighting consists of the following steps:

• Adjustment for selection probability—OPA typically adjusts for selection probability within scientific sampling procedures. However, in the case of the 2022 SAGR, all students will be selected to participate in the survey. Therefore, while adjustment for selection probability is usually performed as the first step in the weighting process, in this instance the selection probability will be 100%, hence the base weights will be calculated to be 1.

• Adjustments for nonresponse—Although the 2022 SAGR will be a census of all students, some students will not respond to the survey. Others might start the survey but not complete it, (i.e., not provide the minimum number of responses required for the survey to be considered complete). OPA adjusts for this nonresponse in creating population estimates by first calculating the base weights as the reciprocal of the probability of selection (in the 2022 SAGR, the base weights will take on the value one (1) since the survey is a census). Next, OPA adjusts the base weights for those who do not respond to the survey, then adjusts for those who started the survey but do not complete it.

• Adjustment to known population values—OPA typically adjusts the weights in the previous step to known population values to account for remaining bias. In the case of the 2022 SAGR, the weights in the previous step will be adjusted to known population values using the three known demographic variables (Academy, class year, and gender). The post stratification adjustments will be one (1) because the three demographic variables will already be accounted for in the previous step.

While the 2022 SAGR is a census of students, not everyone will respond to the survey; hence the weighting procedures described above were required to produce population estimates (e.g., percent female). Because of the weighting, conventional formulas for calculating margins of error will overstate the reliability of the estimate.

3. Degree of accuracy needed for the Purpose discussed in the justification;

OPA creates variance strata so precision measures can be associated with each estimate. We produce precision measures for reporting categories using 95% confidence intervals with the goal of achieving a precision of 5% or less (e.g. 80% (+/- 5%) of cadets/midshipmen are satisfied with their training).

4. Unusual problems requiring specialized sampling procedures; and

None.

5. Use of periodic or cyclical data collections to reduce respondent burden.

To reduce burden on this population, OPA conducts the SAGR in person every other year and is the only official survey to estimate prevalence rates of unwanted sexual contact, sexual harassment, and gender discrimination at the MSAs.

3.  Maximization of Response Rates, Non-response, and Reliability

Discuss methods used to maximize response rates and to deal with instances of non-response.  Describe any techniques used to ensure the accuracy and reliability of responses is adequate for intended purposes.  Additionally, if the collection is based on sampling, ensure that the data can be generalized to the universe under study.  If not, provide special justification.

When students are notified of the survey, their notification emphasizes the importance of their participation. Academy leaders continue to emphasize the survey’s importance for understanding how issues of sexual assault and sexual harassment impact students’ lives. Academy officials monitor attendance at the survey administration sessions and follow up to reschedule sessions for students who did not appear that their scheduled session. OPA staff members provide a briefing on the importance and purpose of the survey to encourage students to complete it. Given previous response rates, OPA predicts enough responses will be received within all important reporting categories to make estimates that meet confidence and precision goals.

4.  Tests of Procedures

Describe any tests of procedures or methods to be undertaken.  Testing of potential respondents (9 or fewer) is encouraged as a means of refining proposed collections to reduce respondent burden, as well as to improve the collection instrument utility.  These tests check for internal consistency and the effectiveness of previous similar collection activities.

Not applicable.

5.  Statistical Consultation and Information Analysis

a. Provide names and telephone number of individual(s) consulted on statistical aspects of the design.

Mr. David McGrath, Branch Chief; Statistical Methods Team, Methods, Analysis, and Systems Support, Office of People Analytics (OPA); (571) 372-0983.

Ms. Wendy Barboza, Team Lead; Statistical Methods Team, Methods, Analysis, and Systems Support, Office of People Analytics (OPA); (571) 372-1099.

b. Provide name and organization of person(s) who will actually collect and analyze the collected information.

The data will be collected by Fors Marsh Group (FMG), which is OPA’s analysis contractor. Ms. Amanda Barry is Director of Military Workplace Climate Research at FMG.

The data will be analyzed by OPA analysts. Ms. Lisa Davis and Dr. Rachel Breslin oversee this effort.

2