2023 National Survey of College Graduates Bridge Panel Analysis (DRAFT)

APPENDIX J

2023 National Survey of College Graduates Bridge Panel Analysis (DRAFT)

2023 National Survey of College Graduates Bridge Panel Analysis:

Preliminary Results for the Sexual Orientation and Gender Identity (SOGI) Questions¹

1. Background

An experimental panel (referred to as the Bridge Panel) was fielded concurrently with the 2023 National Survey of College Graduates (NSCG) production survey to determine the effect of proposed question changes and new content. The goal of this analysis is to understand the impact these changes had on response data and the respondent’s experience navigating the web survey. Specifically, this report discusses results of the sexual orientation and gender identity survey questions.

The 2023 NSCG production instrument included a gender identity question for the first time while the Bridge Panel tested a different way of asking about gender identity. The Bridge Panel also tested two versions of a sexual orientation question (respondents were randomly selected to receive one version); the only difference between the sexual orientation questions was the order of the first two response options. The production instrument did not ask about sexual orientation.

Additionally, to further measure respondent attitudes and comfort answering these questions, four question probes were added in the Bridge Panel: two for gender identity and two for sexual orientation.

Cases in the Bridge Panel only received web invitations and were not eligible to respond by paper or telephone. Comparisons to the production instrument were thus restricted to production cases that responded or used the internet.

2. Question Images

This section presents the survey questions related to gender identity and sexual orientation, as they appeared on the web instruments. The brief question names as used in the survey instruments and in the data are shown in capital letters within parentheses.

• Sex assigned at birth, asked on both the Bridge Panel and Production instruments (GENDER)

• Gender identity as asked in the production instrument (SOGI2)

• Gender identity as asked in the Bridge Panel (GENMIN)

• Sexual orientation version 1, asked in the Bridge Panel (SO1)

• Sexual orientation version 2, asked in the Bridge Panel (SO2)

• Qualitative questions on gender identity, asked in the Bridge Panel (GIP1 and GIP2 on the screen GI_PROBE)

• Qualitative questions on sexual orientation, asked in the Bridge Panel (SOP1 and SOP2 on the screen SO_PROBE)

1. Sex Assigned at Birth and Gender Identity Questions

Before asking about a respondent’s current gender identity, the survey asked about their sex assigned at birth. Both the production instrument and Bridge Panel instrument used the question shown in Figure 1.

Figure 1. GENDER (Sex assigned at birth) – Production and Bridge Panel instruments

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Web Survey

In the production instrument, the next question in the survey (SOGI2, shown in Figure 2) asked a respondent to describe their gender identity. Bridge Panel respondents were shown the alternative gender identity question (GENMIN, shown in Figure 3).

Figure 2. SOGI2 (Gender identity) – Production instrument

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Production Web Survey

Figure 3. GENMIN (Gender identity) – Bridge Panel

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Bridge Panel Web Survey

2. Sexual Orientation Questions

Bridge Panel respondents were randomly selected to receive one of the following two versions of a sexual orientation question: SO1 (which had “Gay or lesbian” as the first response option, shown in Figure 4) or SO2 (which had “Straight, that is not gay or lesbian” as the first response option, shown in Figure 5).

Figure 4. SO1 (Sexual orientation) – Bridge Panel

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Bridge Panel Web Survey

Figure 5. SO2 (Sexual orientation) – Bridge Panel

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Bridge Panel Web Survey

3. Qualitative Probes

Bridge Panel respondents were asked additional qualitative questions to further measure respondent attitudes and comfort answering these questions. Figure 6 shows the question probes regarding the gender identity question, and Figure 7 has the sexual orientation probes.

Figure 6. GI_PROBE (GIP1, GIP2) (Gender identity probes) – Bridge Panel

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Bridge Panel Web Survey

Figure 7. SO_PROBE (SOP1, SOP2) (Sexual orientation probes) – Bridge Panel

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Bridge Panel Web Survey

3. Analysis

The goal of this analysis was to understand the impact these survey changes had on response data and the respondent’s experience navigating the web survey.

Table 1 shows the universe for this analysis. There were 5,100 cases sampled into the Bridge Panel for the 2023 NSCG. Those cases were evenly distributed into two questionnaire version groups, which determined if they would see SO1 or SO2 when they accessed the web instrument. From the Bridge Panel, about 2,400 cases accessed at least one page in the web instrument (i.e., users) and 2,100 answered enough of the instrument to be considered respondents (i.e., respondents).

The full new cohort sample for the 2023 NSCG was about 56,000 cases. However, to make the universes comparable, some cases were excluded from the analytic comparison to the Bridge Panel cases; these included new cohort cases that were selected with certainty² and cases deemed temporarily ineligible in a prior cycle. This left about 39,000 new cohort cases for this analysis. About 19,500 of those accessed at least one page of the web instrument, and 17,000 answered enough of the web instrument to be considered respondents.

Table 1: Universe – Bridge Panel and Production

	Bridge Panel			Production New Cohort
	Version 1	Version 2	Total	Total
Overall sample	2,500	2,500	5,100	56,000
Analytic sample	2,500	2,500	5,100	39,000
Users	1,200	1,200	2,400	19,500
Respondents	1,100	1,000	2,100	17,000

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Web Survey Paradata and Master File Data

Demographic distributions of the Bridge Panel and Production universes (both sampled cases and respondents) are presented in Attachment A. The demographic variables that were compared were: age, citizenship at birth, disability status, highest degree, Hispanic origin, occupation group (broad category), oversample group eligibility indicator, race and ethnicity, science and engineering status, sex, and work status. These demographic variables were available for all sampled members from their American Community Survey response data. See Attachment A for comparisons between the universes.

When conducting our analyses, survey users were the universe of interest for all paradata metrics: changed answer rates, “Previous” click rates, and breakoff rates. Base weights were used for the paradata analysis. Survey respondents were the universe of interest for the item nonresponse analysis and response distributions. Final weights were used for this part of the analysis. Additionally, the response distributions for survey items (excluding the probing questions) come from edited and imputed data; if a respondent left a question blank, a value was imputed for them and the imputed value was used in the response distribution tables.³ Item nonresponse rates in Section 4 show the extent to which gender identity and sexual orientation were imputed.

To assess the gender identity questions, the following metrics were estimated for the production question (SOGI2) and the Bridge Panel question (GENMIN). Formulas and definitions for these metrics are provided in Attachment B.

• Changed answer rate

• “Previous” click rate

• Breakoff rate

• Item nonresponse

• Response distribution, specifically, the percentage of respondents who identify as a gender minority⁴

The production and Bridge Panel questions had different constructs; one asked about gender identity while the other asked about identifying as a gender minority. Additionally, the production question was a select-all-that-apply question while the Bridge Panel question was a multiple-choice question that included “I prefer not to answer” as a response option. Due to these question differences, we did not statistically test the metrics.

To assess the sexual orientation questions in the Bridge Panel instrument, the following metrics were estimated for each question (SO1 and SO2):

• Response distribution, including by sex and gender minority status

• “Previous” click rate

• Item nonresponse

• Breakoff rate

Comparisons using t-tests and chi-square tests with an alpha of 0.10 were conducted between the two versions of the sexual orientation question.

To assess the probing questions within the Bridge Panel (GIP1, GIP2, SOP1, SOP2), the following metrics were estimated:

• Item nonresponse, including by unedited response to GENMIN, SO1 and SO2

• Response distributions, including by response to GENMIN, SO1 and SO2

For the probing questions, we report point estimates and discuss their findings but did not conduct statistical tests.

4. Results

Sections 4.1 to 4.4 present the key results for each question concept. Additional results can be found in Attachment C.

1. Gender identity (SOGI2, GENMIN)

We compared the current gender identity question on production (SOGI2) to the gender minority status question on the Bridge Panel (GENMIN). This analysis sought to identify if one of these questions performed better than the other.

Given the difference in construct and question design between these two questions, no statistical testing was conducted on results in this section.

1. Breakoff rate

The number of breakoffs on both SOGI2 and GENMIN were too small to report, per the Census Bureau’s guidelines for disclosure avoidance and cell suppression.

2. Changed answer rate

The percent of users that changed an answer on the SOGI2 question in the production instrument was 0.9 percent and on the GENMIN question in the Bridge Panel was 3.6 percent (Table 2).

Table 2: Changed Answer Rate – Gender Identity

	Percent of Users who Changed an Answer on this Question
Production (SOGI2)	0.9 (0.1)
Bridge Panel (GENMIN)	3.6 (0.8)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Web Survey Paradata

Note: Standard errors are shown in parentheses.

3. “Previous” click rate

In the production instrument, 1.0 percent of all users who saw the SOGI2 screen clicked “Previous” to go backwards from that screen, while 0.8 percent of all Bridge Panel users who saw the GENMIN screen clicked “Previous” (Table 3).

Table 3: “Previous” Click Rate – Gender Identity

	Percent of Users who Clicked “Previous” on this Question
Production (SOGI2)	1.0 (0.2)
Bridge Panel (GENMIN)	0.8 (0.3)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Web Survey Paradata

Note: Standard errors are shown in parentheses.

4. Item nonresponse rate

In production, 0.6 percent of respondents left the SOGI2 question blank while 1.0 percent of Bridge Panel respondents left GENMIN blank (Table 4).

Table 4: Item Nonresponse Rate – Gender Identity

	Percent of Respondents who did not answer this Question
Production (SOGI2)	0.6 (0.1)
Bridge Panel (GENMIN)	1.0 (0.3)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

5. Response distributions

To conduct the response distribution analysis, we used additional data from the production instrument to identify those individuals who met our definition of being a gender minority. To do this, we used responses from the sex assigned at birth question (GENDER) in addition to the current gender identity question (SOGI2). A respondent from the production instrument was categorized as not a gender minority if their sex assigned at birth matched their current gender identity (and they only selected one option for their current gender identity). Otherwise, respondents from the production instrument (who answered both questions) were categorized as gender minorities. In the Bridge Panel, if a respondent selected “Yes” to GENMIN, they were categorized as a gender minority.

In production, 0.6 percent of respondents were considered a gender minority while 1.5 percent of respondents to the Bridge Panel identified as a gender minority (Table 5). We note that the 2021 NSCG Bridge Panel analysis reported that gender minorities had more bachelor’s or professional degrees, and fewer master’s or doctorate degrees, than non-minorities (Satisky, Reeves & Basdeo, 2023). As shown in Attachment A, Table 31, the 2023 Bridge Panel respondents had more advanced degrees than did the production respondents, and thus the difference in the percentage of respondents that identified as a gender minority could be greater between the two questions.

Table 5: Response Distribution – Gender Minority Status

	Percent of Respondents who Identified as a Gender Minority
Production (SOGI2)	0.6 (0.1)
Bridge Panel (GENMIN)	1.5 (0.5)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. These results use imputed data.

While Table 5 reports gender minority status as determined using GENDER and SOGI2 for production respondents, Table 6 and Table 7 show the raw response distributions of those questions, respectively. Additionally, Table 8, which reports the full gender minority status distribution, shows that 2.7 percent of Bridge Panel respondents chose the “Prefer not to answer” response option.

Table 6: Response Distribution – Sex assigned at birth (GENDER)

Survey	Sex Assigned at Birth	Percent of Respondents
Production	Female	53.6 (0.4)
	Male	46.4 (0.4)
	Total	100
Bridge Panel	Female	54.9 (0.5)
	Male	45.1 (0.5)
	Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. Imputed data were used for this analysis.

A Rao-Scott chi-square test resulted in a p-value of 0.05.

Table 7: Response Distribution – Gender Identity (SOGI2)

SOGI2	Percent of Respondents
Female	53.4 (0.4)
Male	46.2 (0.4)
Transgender	0.2 (<0.1)
I use a different term, specify	0.4 (0.1)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. These results use imputed data.

Table 8: Response Distribution – Gender Minority Status Full Distribution

Survey	Gender Minority	Percent of Respondents
Production (Derived from the questions GENDER and SOGI2)	No	99.4 (0.1)
	Yes	0.6 (0.1)
	Prefer not to answer	N/A
	Total	100
Bridge Panel (GENMIN)	No	95.8 (0.7)
	Yes	1.5 (0.5)
	Prefer not to answer	2.7 (0.6)
	Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. These results use imputed data.

To summarize, the analysis of the gender identity questions showed that three metrics (breakoff rates, “Previous” click rates, and item nonresponse rates) were comparable between survey instruments. However, while not statistically tested, there were slight differences in the changed answer rates and the response distributions.

2. Sexual orientation (SO1, SO2)

To compare SO1 and SO2 we calculated breakoff rates, “Previous” click rates, item nonresponse rates and response distributions, including response distributions by sex and gender minority status. As noted earlier, the only difference between the two survey questions was the order of the response options. In SO1, “Gay or lesbian” was the first response option and “Straight, that is, not gay or lesbian” was the second option. In SO2, the placement of those two options was flipped. The remaining three response options were the same in both versions of the question.

1. Response distributions

There was no statistical difference between the SO1 and SO2 response distributions (Χ² p-value = 0.62). Because there was no difference found, we combined SO1 and SO2 (Table 9) for greater statistical power in the cross tabulations with the qualitative probe questions.

Table 9: Response Distribution – Sexual Orientation

Sexual Orientation	Percent of Respondents
	SO1	SO2	SO1 and SO2 Combined
Bisexual	1.6 (0.4)	1.7 (0.4)	1.7 (0.3)
Gay or lesbian	3.0 (0.6)	2.9 (0.9)	2.9 (0.5)
Straight, that is not gay or lesbian	92.3 (1.3)	93.7 (1.1)	92.9 (0.9)
I don’t know	2.3 (1.0)	1.2 (0.4)	1.8 (0.7)
I use a different term, specify	0.8 (0.5)	0.6 (0.2)	0.7 (0.2)
Total	100	100	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. These results use imputed data.

Additional results with cross tabulations by sexual orientation are available in Attachment C. Table 39 and Table 40 in the attachments show sexual orientation crossed with sex and gender minority status, respectively.

In summary, the analysis of the sexual orientation questions found no statistical differences between the question versions, for any of the four metrics that were tested.

2. Breakoff rate

The number of breakoffs in both SO1 and SO2 were too small to report, per the Census Bureau’s guidelines for disclosure avoidance and cell suppression.

3. “Previous” click rate

The “Previous” click rate on the sexual orientation question (Table 10) was not affected by the order of the first two response options (p-value = 0.21).

Table 10: “Previous” Click Rate – Sexual Orientation

	Percent of Users who Clicked “Previous” from this Question
Bridge Panel (SO1)	2.4 (0.6)
Bridge Panel (SO2)	1.5 (0.4)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Web Survey Paradata

Note: Standard errors are shown in parentheses.

4. Item nonresponse rate

The item nonresponse rate for sexual orientation (Table 11) was not affected by the order of the first two response options (p-value = 0.24).

Table 11: Item Nonresponse Rate – Sexual Orientation

	Percent of Respondents who Left this Question Blank
Bridge Panel (SO1)	2.0 (0.9)
Bridge Panel (SO2)	4.4 (2.4)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

In summary, the analysis of the sexual orientation questions found no statistical differences between the question versions, for any of the four metrics that were tested.

3. Gender probes (GIP1, GIP2)

There were two qualitative questions to understand how respondents felt about answering the gender identity questions. The first question asked how comfortable the respondent was providing gender identity information in a federal government survey, while the second allowed respondents to choose multiple statements that accurately captured how they felt when thinking about the gender identity question.

We calculated overall item nonresponse rates and response distributions for the gender identity probing questions. We also report these results both by gender minority status and sexual orientation.

1. Item nonresponse rate

The item nonresponse rate for the first gender identity probe was 2.2 percent. The second gender identity probe had an item nonresponse rate of 22.6 percent (Table 12); however, if none of the statements were true for the respondent or if they had no concerns, there wasn’t a “none of the above” or “no concerns” option available to respondents so leaving the question blank was appropriate.

Table 12: Item Nonresponse Rate – Gender Identity Probes

	Percent of Respondents who Left this Question Blank
Gender Identity Probe 1 (GIP1)	2.2 (0.5)
Gender Identity Probe 2 (GIP2)	22.6 (2.1)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Only 3.6 percent of respondents who identified as a gender minority left the second gender identity probe blank while 22.7 percent of those respondents that answered “No” to the gender minority question left the second gender identity probe blank (Table 13). See Table 41 in Attachment C for gender identity probe 2 item nonresponse by sexual orientation.

Table 13: Item Nonresponse Rate – Gender Identity Probe 2 by Gender Minority

Gender Minority	Percent of Respondents who Left Gender Identity Probe 2 Blank
Yes	3.6 (2.1)
No	22.7 (2.3)
Prefer not to answer	8.9 (3.4)
Left GENMIN blank	72.3 (11.1)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

2. Response distributions

Most respondents felt very comfortable reporting their gender identity (68.5 percent) (Table 14); however, comfort varied greatly by gender minority status and sexual orientation (Table 15 and Table 16, respectively). While 70.6 percent of respondents who selected “No” to GENMIN felt very comfortable reporting their gender identity, only 31.2 percent who selected “Yes” felt very comfortable (Table 15).

Table 14: Response Distribution – Gender Identity Probe 1

Gender Identity Probe 1	Percent of Respondents
Very comfortable	68.5 (3.1)
Somewhat comfortable	12.5 (2.1)
Somewhat uncomfortable	8.1 (1.5)
Very uncomfortable	8.6 (1.6)
No response	2.2 (0.5)
Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Table 15 shows that 51.0 percent of people who marked “Prefer not to answer” on the Bridge Panel gender minority question (GENMIN) then reported they were ‘Very uncomfortable” with providing that information in a federal government survey. Interestingly, 17.9 percent said they were “Very comfortable.”

Table 15: Response Distribution – Gender Identity Probe 1 by Gender Minority

	Gender Minority
Gender Identity Probe 1	Yes	No	Prefer not to answer
Very comfortable	31.2 (13.7)	70.6 (3.0)	17.9 (6.9)
Somewhat comfortable	14.4 (10.2)	12.6 (2.2)	6.6 (4.0)
Somewhat uncomfortable	26.0 (14.7)	7.7 (1.6)	14.2 (5.8)
Very uncomfortable	D	7.1 (1.5)	51.0 (10.5)
No response	D	2.0 (0.5)	10.2 (4.0)
Total	100	100	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed response data for Gender Minority. Gender Identity Probe 1 was not edited or imputed.

(D) Suppressed for disclosure avoidance.

Table 16: Response Distribution – Gender Identity Probe 1 by Sexual Orientation

Sexual Orientation	Gender Identity Probe 1	Percent of Respondents
Bisexual	Very comfortable	58.4 (9.6)
	Somewhat comfortable	15.4 (5.2)
	Somewhat uncomfortable	15.8 (7.0)
	Very uncomfortable	D
	No response	D
	Total	100
Gay or lesbian	Very comfortable	43.7 (9.7)
	Somewhat comfortable	19.0 (6.4)
	Somewhat uncomfortable	13.2 (6.6)
	Very uncomfortable	D
	No response	D
	Total	100
Straight, that is not gay or lesbian	Very comfortable	70.8 (3.0)
	Somewhat comfortable	12.1 (2.2)
	Somewhat uncomfortable	7.5 (1.7)
	Very uncomfortable	7.4 (1.5)
	No response	2.2 (0.6)
	Total	100
I don’t know	Very comfortable	12.2 (6.0)
	Somewhat comfortable	21.2 (6.7)
	Somewhat uncomfortable	14.1 (7.5)
	Very uncomfortable	48.9 (12.4)
	No response	3.6 (2.4)
	Total	100
I use a different term, specify	Very comfortable	36.2 (16.0)
	Somewhat comfortable	7.3 (4.2)
	Somewhat uncomfortable	39.6 (23.6)
	Very uncomfortable	D
	No response	D
	Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed response data for Sexual Orientation. Gender Identity Probe 1 was not edited or imputed.

(D) Suppressed for disclosure avoidance.

Table 17 shows how respondents answered the second qualitive probe on gender identity. Since respondents could select more than one statement that resonated with them, the percentages shown in Table 17 do not sum to 100. For each response option, the percentages reflect how many respondents agreed with that statement.

The statement “I have questions about how this information would be useful to the NSCG” was marked most often, with 34.6 percent of respondents agreeing.

Table 17: Responses to Gender Identity Probe 2

Gender Identity Probe 2	Percent of Respondents
I consider my gender identity to be my own private information	27.8 (2.2)
I have questions about how this information would be useful to the NSCG	34.6 (2.8)
I have concerns about how this information may be used	22.0 (2.5)
The response options do not reflect how I think of myself	10.2 (2.0)
I do not understand the question	5.6 (1.5)
I have other concerns – specify+	7.4 (1.6)
No concerns or no response+	24.5 (2.2)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

⁺There were respondents who selected “I have other concerns” to Gender Identity Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no concerns or no response percentage. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

To explore who was selecting each of the probes in GIP2, Table 18 and Table 19 present how often a statement was marked on the gender identity probe 2 question, by a respondent’s gender minority status (Table 18) or sexual orientation (Table 19). While 30.7 percent of respondents who identified as a gender minority agreed with the second statement on the GIP2 question, 34.9 percent of people who did not identify as a gender minority also agreed with that statement.

Table 18: Response Distribution – Gender Identity Probe 2 by Gender Minority

Gender Identity Probe 2	Gender Minority
	Yes	No	Prefer not to answer
I consider my gender identity to be my own private information	59.0 (14.7)	27.1 (2.3)	33.6 (7.8)
I have questions about how this information would be useful to the NSCG	30.7 (13.5)	34.9 (2.9)	27.6 (8.1)
I have concerns about how this information may be used	40.3 (14.0)	21.0 (2.5)	48.5 (10.3)
The response options do not reflect how I think of myself	6.3 (4.1)	10.4 (2.0)	7.7 (4.2)
I do not understand the question	D	5.5 (1.6)	12.4 (5.1)
I have other concerns – specify+	6.4 (4.3)	7.3 (1.7)	9.1 (4.4)
No response or no concerns+	3.6 (2.1)	25.2 (2.3)	11.7 (4.4)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Gender Minority.

⁺There were respondents who selected “I have other concerns” to Gender Identity Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no concerns or no response percentage. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

(D) Suppressed for disclosure avoidance.

Table 19 shows how people responded to GIP2 by their sexual orientation. Again, the second statement stood out as being selected by people in a minority and majority group.

Table 19: Response Distribution – Gender Identity Probe 2 by Sexual Orientation

	Sexual Orientation
Gender Identity Probe 2	Bisexual	Gay or Lesbian	Straight	I don’t know	I use a different term
I consider my gender identity to be my own private information	34.2 (9.2)	36.9 (9.8)	27.1 (2.4)	40.6 (10.4)	32.2 (14.6)
I have questions about how this information would be useful to the NSCG	54.6 (8.8)	32.8 (8.8)	34.4 (3.0)	26.1 (6.9)	44.1 (22.4)
I have concerns about how this information may be used	30.4 (7.7)	32.1 (8.6)	20.7 (2.6)	52.1 (11.5)	59.5 (16.5)
The response options do not reflect how I think of myself	4.2 (2.2)	5.5 (3.7)	10.4 (2.1)	15.6 (5.7)	8.0 (6.7)
I do not understand the question	D	0.0 (N/A)	5.7 (1.6)	16.7 (9.4)	D
I have other concerns – specify+	D	3.5 (2.2)	7.5 (1.7)	14.4 (8.6)	7.4 (6.3)
No response or no concerns+	15.7 (5.5)	23.6 (7.5)	25.1 (2.3)	4.9 (2.9)	18.0 (12.2)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Sexual Orientation.

⁺There were respondents who selected “I have other concerns” to Gender Identity Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no response or no concerns percentages. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

(D) Suppressed for disclosure avoidance.

An additional table in Attachment C (Table 42) shows the response distribution to gender identity probe 2 based on response to gender identity probe 1.

4. Sexual orientation probes (SOP1, SOP2)

As with gender identity, there were two qualitative questions to understand how respondents felt about answering a sexual orientation survey question.

For the sexual orientation probes, we calculated item nonresponse rates and response distributions, including calculating both by gender minority status and sexual orientation. Because there were no differences found between the responses to SO1 and SO2, we combined them when calculating cross tabulations.

1. Item nonresponse rate

The item nonresponse rate for the first sexual orientation probe was 3.0 percent. The second probe had an item nonresponse rate of 20.6 percent (Table 20); however, as with gender identity probe 2, if none of the statements were true for the respondent or they had no concerns, there wasn’t a “none of the above” or “no concerns” option available and leaving the question blank was appropriate.

Table 20: Item Nonresponse Rate – Sexual Orientation Probes

	Percent of Respondents who Left this Question Blank
Sexual Orientation Probe 1 (SOP1)	3.0 (0.7)
Sexual Orientation Probe 2 (SOP2)	20.6 (2.3)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Table 21 shows that 13.5 percent of people who identified as gay or lesbian then left the second sexual orientation probe blank while 21.2 percent of those respondents that identified as straight left the second sexual orientation probe blank.

Table 21: Item Nonresponse Rate – Sexual Orientation Probe 2 by Sexual Orientation

Sexual Orientation	Percent of Respondents who Left Sexual Orientation Probe 2 Blank
Bisexual	14.8 (6.2)
Gay or lesbian	13.5 (6.0)
Straight, that is not gay or lesbian	21.2 (2.6)
I don’t know	2.2 (1.4)
I use a different term, specify	1.6 (1.3)
Left sexual orientation blank	29.1 (14.7)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Item nonresponse to the second sexual orientation probe is also presented by gender minority status in Table 43 in Attachment C.

2. Response distributions

Table 22 shows how Bridge Panel respondents answered the first qualitative probe about sexual orientation. Most respondents were comfortable providing this information; however, as with gender identity probe 1, comfort varied by gender minority status and sexual orientation (Table 23 and Table 24, respectively).

Table 22: Response Distributions – Sexual Orientation Probe 1

Sexual Orientation Probe 1	Percent of Respondents
Very comfortable	66.7 (3.0)
Somewhat comfortable	13.7 (2.1)
Somewhat uncomfortable	9.5 (1.5)
Very uncomfortable	7.1 (1.5)
No response	3.0 (0.7)
Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Table 23 shows how comfortable a respondent was answering sexual orientation by whether they were a gender minority. Respondents who were not a gender minority reported being more comfortable answering the sexual orientation question than were other respondents.

Table 23: Response Distribution – Sexual Orientation Probe 1 by Gender Minority

	Gender Minority
Sexual Orientation Probe 1	Yes	No	Prefer not to answer
Very comfortable	25.5 (12.7)	68.8 (3.0)	15.8 (6.6)
Somewhat comfortable	25.0 (11.4)	13.6 (2.2)	10.8 (4.4)
Somewhat uncomfortable	22.2 (14.2)	9.3 (1.6)	8.6 (4.7)
Very uncomfortable	D	5.5 (1.4)	52.4 (9.5)
No response	D	2.7 (0.7)	12.4 (4.7)
Total	100	100	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Gender Minority. Sexual Orientation Probe 1 is unedited and unimputed.

(D) Suppressed for disclosure avoidance.

Table 24 shows a respondent’s comfort answering the sexual orientation question by their reported sexual orientation. Of the people who marked “I don’t know” on the Bridge Panel sexual orientation questions (either SO1 or SO2), 55.8 percent reported they were ‘Very uncomfortable” with providing their sexual orientation in a federal government survey.

Table 24: Response Distribution – Sexual Orientation Probe 1 by Sexual Orientation

Sexual Orientation	Sexual Orientation Probe 1	Percent of Respondents
Bisexual	Very comfortable	49.9 (10.7)
	Somewhat comfortable	10.9 (3.8)
	Somewhat uncomfortable	32.2 (8.9)
	Very uncomfortable	D
	No response	D
	Total	100
Gay or lesbian	Very comfortable	40.0 (9.3)
	Somewhat comfortable	22.9 (6.3)
	Somewhat uncomfortable	21.7 (8.2)
	Very uncomfortable	D
	No response	D
	Total	100
Straight, that is not gay or lesbian	Very comfortable	69.2 (3.0)
	Somewhat comfortable	13.3 (2.3)
	Somewhat uncomfortable	8.4 (1.6)
	Very uncomfortable	6.0 (1.5)
	No response	3.0 (0.7)
	Total	100
I don’t know	Very comfortable	9.1 (4.8)
	Somewhat comfortable	20.3 (6.3)
	Somewhat uncomfortable	11.4 (6.3)
	Very uncomfortable	55.8 (10.0)
	No response	3.4 (2.3)
	Total	100
I use a different term, specify	Very comfortable	25.2 (13.7)
	Somewhat comfortable	19.4 (12.0)
	Somewhat uncomfortable	47.6 (18.9)
	Very uncomfortable	D
	No response	D
	Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Sexual Orientation. Sexual Orientation Probe 1 is unedited and unimputed.

(D) Suppressed for disclosure avoidance.

Table 25 shows how Bridge Panel respondents answered the second qualitative probe about sexual orientation. Since respondents could mark more than one statement that resonated with them, the percentages shown in Table 25 do not sum to 100. For each response option, the percentages reflect how many respondents agreed with that statement.

The statement “I have questions about how this information would be useful to the NSCG” was marked as true by 36.5 percent of respondents.

Table 25: Response Distributions – Sexual Orientation Probe 2

Sexual Orientation Probe 2	Percent of Respondents
I consider my sexual orientation to be my own private information	34.8 (2.2)
I have questions about how this information would be useful to the NSCG	36.5 (2.7)
I have concerns about how this information may be used	24.6 (2.6)
The response options do not reflect how I think of myself	7.7 (1.3)
I do not understand the question	3.1 (1.3)
I have other concerns – specify+	3.4 (0.8)
No response or no concerns+	24.5 (2.3)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

⁺There were respondents who selected “I have other concerns” to Sexual Orientation Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no concerns or no response percentage. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

Table 26 presents how often a statement was marked on the sexual orientation probe 2 question, by a respondent’s sexual orientation (Table 26). For instance, 53.8 percent of respondents who identified as bisexual agreed with the first statement on the SOP2 question. Notably, 39.0 percent of respondents who identified as gay or lesbian agreed with the second statement on the SOP2 question, but 36.1 percent of people who identified as straight also agreed with that statement.

Table 26: Response Distribution – Sexual Orientation Probe 2 by Sexual Orientation

	Sexual Orientation
Sexual Orientation Probe 2	Bisexual	Gay or Lesbian	Straight	I don’t know	I use a different term
I consider my sexual orientation to be my own private information	53.8 (10.0)	49.4 (8.6)	33.7 (2.4)	52.7 (13.3)	30.6 (14.2)
I have questions about how this information would be useful to the NSCG	55.8 (8.9)	39.0 (8.4)	36.1 (2.9)	32.0 (7.9)	40.5 (23.3)
I have concerns about how this information may be used	37.1 (8.3)	39.6 (9.4)	23.2 (2.8)	52.5 (11.6)	56.6 (17.0)
The response options do not reflect how I think of myself	1.0 (0.8)	2.0 (0.9)	7.7 (1.4)	20.5 (6.4)	25.5 (14.2)
I do not understand the question	D	0.0 (N/A)	3.2 (1.4)	7.0 (4.9)	D
I have other concerns – specify+	D	2.9 (2.4)	3.3 (0.8)	10.3 (6.3)	D
No response or no concerns+	15.3 (6.3)	18.5 (6.4)	25.4 (2.5)	4.3 (2.7)	5.1 (3.9)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Sexual Orientation.

⁺There were respondents who selected “I have other concerns” to Sexual Orientation Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no response or no concerns percentages. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

(D) Suppressed for disclosure avoidance.

Table 27 presents how often a statement was marked on the sexual orientation probe 2 question, by a respondent’s gender minority status. The second statement stood out as being selected at comparable rates by people regardless of gender minority status.

Table 27: Response Distribution – Sexual Orientation Probe 2 by Gender Minority

	Gender Minority
Sexual Orientation Probe 2	Yes	No	Prefer not to answer
I consider my sexual orientation to be my own private information	71.3 (12.9)	33.9 (2.3)	44.9 (10.8)
I have questions about how this information would be useful to the NSCG	30.1 (13.5)	36.5 (2.9)	37.4 (8.1)
I have concerns about how this information may be used	33.6 (13.3)	23.7 (2.7)	50.6 (10.6)
The response options do not reflect how I think of myself	6.3 (3.9)	7.7 (1.4)	11.7 (6.2)
I do not understand the question	D	3.1 (1.3)	5.0 (3.0)
I have other concerns – specify+	0.0 (N/A)	3.4 (0.8)	4.4 (2.4)
No response or no concerns+	3.6 (2.1)	25.2 (2.4)	11.8 (4.4)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Gender Minority.

⁺There were respondents who selected “I have other concerns” to Sexual Orientation Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no response or no concerns percentages. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

(D) Suppressed for disclosure avoidance.

Additional results from this question are presented in Attachment C, crossed by responses to the sexual orientation probe 1 (Table 44).

5. Conclusions

The gender identity questions in production (SOGI2) and in the Bridge Panel (GENMIN) measure different constructs as evidenced by the percent of respondents that identified as a gender minority (0.6 percent in production and 1.5 percent in the Bridge Panel). Additionally, given the nominal difference in the percent of users who changed an answer (0.9 percent in production and 3.6 percent in the Bridge Panel), users appeared to have more uncertainty when answering GENMIN than with SOGI2. SOGI2 in conjunction with the sex assigned at birth question is a generally accepted means of collecting gender identity information (Office of the Chief Statistician of the United States, 2023), and our analysis did not uncover any reason to change what is currently being used in production. However, given the differences found between SOGI2 and GENMIN and the evolving nature of this topic, additional testing (such as cognitive interviews or including a write-in box on GENMIN for gender minorities to provide more information) may be warranted to identify the reasons behind the differences. GENMIN had a higher percentage of respondents identifying as gender minorities and a higher percentage of users changing their answers. “Gender minority” can be interpreted in multiple ways; for example, females may consider themselves to be a minority since they are often underrepresented, and males may consider themselves a minority because there are slightly more females than males in the U.S. Additionally, a larger percentage of respondents chose the “Prefer not to answer” response option than the “Yes” response option to the GENMIN question. In the 2021 Bridge Panel analysis we recommended removing the “Prefer not to answer” response option from the current gender question (Satisky, Reeves & Basdeo, 2023).⁵ We continue to support the recommendation not to include a “Prefer not to answer” option when respondents can easily skip the question.

The order of the response options in the sexual orientation question had no effect. The “Previous” click rate, item nonresponse rate and response distributions (overall and by sex) were not significantly different. The breakoff rate for both was negligible.

There was a high percentage of respondents who agreed with the statement “I have questions about how this information would be useful to the NSCG,” regarding both the gender identity and sexual orientation questions. As a result, if this question is added to a production survey, it likely would be beneficial to the survey participant to include a sentence at the beginning of each question that says how this information will be useful (similar to what exists for the disability question within the NSCG questionnaire). Alternatively, an explanation could be included in a “Help” link and/or in the FAQs.

6. Plans for the 2025 NSCG Data Collection

For the examination of potential modifications to the gender identity question wording, the 2023 NSCG

Bridge Panel research did not uncover any reason to change the question wording used in the 2023 NSCG production data collection. As result, for the 2025 NSCG, NCSES will continue to use the two-step sex-at-birth/gender identity question from the 2023 NSCG survey cycle.

For the assessment of the response option ordering for the sexual orientation question, the 2023 NSCG

Bridge Panel research found no significant difference in the response distribution due to the response option ordering. The Bridge Panel findings have been shared with the areas of the Census Bureau that are working on SOGI research to help inform their on-going SOGI research efforts, including a proposed test of SOGI questions on the American Community Survey (ACS).

The Census Bureau’s ACS is an authoritative source for demographic data used by federal and non-federal studies for benchmarking purposes to ensure consistency in survey estimates across different data collection programs. This type of benchmarking is necessary to ensuring high-quality accurate estimates for large, nationally-representative demographic surveys like the NSCG. Given the critical role of the ACS in ensuring coordination and synchronization across the federal statistical system and its potential use as a benchmark for sexual orientation estimates, NCSES will await the findings from the Census Bureau’s ACS sexual orientation research to guide future data collection decisions. As a result, NCSES will not include a sexual orientation question on the 2025 NSCG and will use the Census Bureau’s ACS research, when completed, to inform its data collection plans for the 2027 NSCG.

7. References

Office of the Chief Statistician of the United States (2023). “Recommendations on the Best Practices for the Collection of Sexual Orientation and Gender Identity Data on Federal Statistical Surveys.” Office of Management and Budget. Accessed August 23, 2024 from https://www.whitehouse.gov/wp-content/uploads/2023/01/SOGI-Best-Practices.pdf.

Satisky, B., Reeves, R., & Basdeo, N. (2023). 2021 National Survey of College Graduates Bridge Panel Analysis Results. U.S. Census Bureau.

U.S. Census Bureau. (2023). Sampling Specification for the 2023 National Survey of College Graduates Bridge Panel - Revised.

Attachment A – Tables Of Demographic Distributions

Table 28 through Table 38 provide sample and respondent distributions of key demographics:

• age,

• citizenship at birth,

• disability status,

• highest degree,

• Hispanic origin,

• occupation group (broad category),

• oversample group eligibility indicator,

• race and ethnicity,

• science and engineering status,

• sex, and

• work status.

These demographic variables were available for all sampled members from their American Community Survey response. Sample distributions use base (or sampling) weights, and respondent distributions use final weights.

The following tables show demographic distributions within:

• The overall production new cohort sample

• The production new cohort sample that was used for this analysis (that is, with certainty cases removed as well as cases deemed temporarily ineligible in a prior cycle) in order to be comparable to the Bridge Panel sample (i.e., the production new cohort analytic sample)

• The Bridge Panel sample

• Respondents from the production new cohort analytic sample

• Respondents from the Bridge panel sample

Chi-square tests were conducted to compare the distributions of:

• The overall production new cohort sample against the production new cohort analytic sample

• The production new cohort analytic sample against the Bridge Panel sample

• Respondents from the production new cohort analytic sample against the respondents from the Bridge panel sample

Future data collection will be done on a universe akin to the overall production sample, so it is useful to note how the overall production sample differs from the analytic production sample. All demographic characteristic distributions, except sex, were significantly different between the production new cohort overall sample and the production new cohort analytic sample. However, while all the distributions were significantly different, a number were not meaningfully different; for instance, Table 30 shows that when examining disability status, 6.8 percent of the overall production sample had a disability while the corresponding result from the analytic sample was 6.9 percent. The category with the largest difference between the overall sample and the analytic sample was found in the 0 to 29 age group (1.6 percentage points, seen in Table 28).

However, the only difference between the production new cohort analytic sample and the Bridge Panel sample was in the oversample group eligibility indicator (Table 34).

When comparing production new cohort analytic sample respondents to the Bridge Panel respondents, the oversample group eligibility indicator (Table 34), sex (Table 37), broad occupation group (Table 33), and highest degree (Table 31) show significantly different distributions. For the first two demographics, oversample group and sex, we notice that differences between distributions are one percentage point or less. For the latter two demographics (broad occupation group and highest degree), the differences between distributions are more substantial. The differences as it relates to the SOGI questions is discussed in Section 4.1.5.

Table 28: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Age

Age (CAGEGROUP)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: 0 to 29	12.5 (0.3)	10.9 (0.3)	11.0 (0.9)	12.1 (0.6)	12.4 (1.1)
2: 30 to 39	23.2 (0.7)	23.3 (0.7)	23.1 (2.0)	24.5 (1.0)	24.4 (2.5)
3: 40 to 49	21.1 (0.7)	21.5 (0.7)	21.4 (2.3)	21.2 (0.9)	22.5 (2.6)
4: 50 to 59	18.8 (0.7)	19.2 (0.7)	20.1 (1.9)	18.5 (0.9)	18.9 (2.4)
5: 60 to 75	24.5 (0.6)	25.1 (0.6)	24.4 (1.7)	23.8 (0.8)	21.8 (1.9)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.99		0.94

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 29: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Citizenship Status at Birth

Citizen status at birth (CSAB)	Sample				Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample		Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: U.S. citizen at birth	82.9 (0.1)		83.2 (0.1)	83.1 (0.3)	84.8 (0.4)	85.3 (1.0)
2: Not a U.S. citizen at birth	17.1 (0.1)		16.8 (0.1)	16.9 (0.3)	15.2 (0.4)	14.7 (1.0)
Total	100		100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall			*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample				0.94		0.66

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 30: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Disability Status

Disability status (DIS)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: With disability	6.8 (<0.1)	6.9 (<0.1)	6.9 (<0.1)	6.1 (0.2)	5.1 (0.6)
2: No disability	93.2 (<0.1)	93.1 (<0.1)	93.1 (<0.1)	93.9 (0.2)	94.9 (0.6)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.60		0.12

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 31: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Highest Degree

Highest degree (HIDEG)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: Bachelor’s or professional degree	69.0 (0.1)	69.4 (0.1)	69.4 (0.1)	68.7 (0.5)	64.0 (2.0)
2: Master’s degree	26.8 (0.1)	26.5 (0.1)	26.5 (0.1)	27.0 (0.5)	31.2 (1.9)
3: Doctorate degree	4.2 (<0.1)	4.1 (<0.1)	4.1 (<0.1)	4.3 (0.1)	4.7 (0.4)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.70		*0.007

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 32: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Hispanic Origin

Hispanic origin (HISP)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
	Percent	Percent	Percent	Percent	Percent
1: Hispanic	10.6 (0.2)	9.9 (0.2)	9.9 (0.4)	9.5 (0.2)	9.8 (0.2)
2: Not Hispanic	89.4 (0.2)	90.1 (0.2)	90.1 (0.4)	90.5 (0.2)	90.2 (0.2)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.996		0.31

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 33: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Broad Occupation Group

Broad occupation group (OCC_BROAD)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
11: Mathematical scientists	0.6 (<0.1)	0.6 (<0.1)	0.4 (0.1)	0.7 (0.1)	0.4 (0.1)
12: Computer and information scientists	5.0 (0.1)	4.8 (0.1)	4.9 (0.2)	5.9 (0.2)	5.7 (0.4)
20: Life scientists	0.5 (<0.1)	0.4 (<0.1)	0.3 (0.1)	0.5 (0.1)	0.4 (0.1)
30: Physical scientists	0.8 (<0.1)	0.6 (<0.1)	0.7 (0.1)	0.7 (0.1)	0.7 (0.2)
40: Social scientists, except psychologists	0.2 (<0.1)	0.1 (<0.1)	0.1 (<0.1)	0.2 (<0.1)	0.2 (0.1)
41: Psychologists	0.3 (<0.1)	0.3 (<0.1)	0.3 (0.1)	0.3 (<0.1)	0.2 (0.1)
50: Engineers	2.7 (0.1)	2.5 (0.1)	2.5 (0.2)	2.9 (0.1)	2.5 (0.3)
61: S&E-related health occupations	9.7 (0.1)	9.4 (0.1)	9.4 (0.2)	9.0 (0.2)	11.1 (1.1)
62: S&E-related non-health occupations	2.4 (0.1)	2.3 (0.1)	2.6 (0.2)	2.7 (0.1)	3.0 (0.4)
71: Postsecondary teacher in an S&E field	1.1 (<0.1)	0.9 (<0.1)	1.0 (0.1)	1.1 (0.1)	1.2 (0.2)
72: Postsecondary teacher in a non-S&E field	0.7 (0.2)	0.8 (0.2)	0.6 (0.1)	0.7 (0.2)	0.6 (0.2)
73: Secondary teacher in an S&E field	0.6 (<0.1)	0.6 (<0.1)	0.5 (0.1)	0.7 (0.1)	0.7 (0.2)
74: Secondary teacher in a non-S&E field	1.2 (0.3)	1.3 (0.3)	1.8 (0.9)	1.2 (0.3)	3.0 (1.4)
81: Non-S&E high interest occupation, S&E FOD	12.1 (0.1)	12.1 (0.1)	11.7 (0.4)	13.1 (0.3)	12.1 (0.8)
82: Non-S&E low interest occupation, non-S&E FOD	8.9 (0.1)	8.6 (0.1)	8.5 (0.4)	8.7 (0.3)	7.5 (0.7)
83: Non-S&E occupation, non-S&E FOD	34.8 (0.4)	35.8 (0.4)	34.9 (1.0)	34.2 (0.7)	33.5 (2.1)
91: Not working, S&E FOD or S&E previous occupation	8.9 (0.1)	9.1 (0.1)	9.2 (0.4)	8.6 (0.3)	9.1 (0.7)
92: Not working, non-S&E FOD and non-S&E previous occupation or never worked	9.5 (0.2)	9.7 (0.2)	10.6 (0.6)	8.7 (0.4)	8.2 (1.5)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.89		*0.08

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 34: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of the Oversample Group Eligibility Indicator

Oversample group eligibility indicator (OVERSAMP)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: S&E case that has earned a bachelor’s or master’s degree in the last five years	9.6 (0.1)	8.0 (0.1)	8.7 (0.4)	9.0 (0.2)	10.1 (0.7)
2: Non-S&E case or S&E case that has not earned a bachelor’s or master’s degree in the last five years	90.4 (0.1)	92.0 (0.1)	91.3 (0.4)	91.0 (0.2)	89.9 (0.7)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			*0.07		*0.098

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 35: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Race and Ethnicity

Race and Ethnicity (RACETH)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: Hispanic	10.6 (0.2)	9.9 (0.2)	9.9 (0.4)	9.5 (0.2)	9.8 (0.2)
2: Black	8.9 (0.1)	8.3 (0.1)	8.1 (0.3)	7.8 (0.2)	7.9 (0.4)
3: Asian	10.7 (0.3)	10.5 (0.3)	11.0 (0.9)	10.7 (0.1)	10.6 (0.2)
4: American Indian, Alaska Natives	1.3 (0.3)	1.3 (0.3)	1.3 (0.8)	0.9 (0.1)	0.9 (<0.1)
5: Native Hawaiian, Pacific Islander	0.2 (0.1)	0.2 (0.1)	0.1 (0.1)	0.2 (<0.1)	0.2 (<0.1)
6: White	68.3 (0.4)	69.8 (0.4)	69.6 (1)	71.0 (0.3)	70.6 (0.4)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.98		0.90

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 36: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Science and Engineering Status

Science and engineering status (SE)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: S&E degree or S&E occupation	54.5 (0.2)	53.2 (0.2)	52.7 (0.3)	55.9 (0.5)	55.2 (1.3)
2: No S&E degree nor S&E occupation	45.5 (0.2)	46.8 (0.2)	47.3 (0.3)	44.1 (0.5)	44.8 (1.3)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.19		0.63

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 37: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Sex

Sex (SEX)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: Male	46.1 (0.1)	46.1 (0.1)	46.0 (0.1)	46.6 (0.4)	45.5 (0.5)
2: Female	53.9 (0.1)	53.9 (0.1)	54.0 (0.1)	53.4 (0.4)	54.5 (0.5)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		0.77
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.46		*0.08

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Table 38: New Cohort and Bridge Panel Sample and Respondent Percent Distributions of Work Status

Work status (WORKSTAT)	Sample			Respondents
	Production New Cohort Overall	Production New Cohort Analytic Sample	Bridge Panel	Production New Cohort Analytic Sample	Bridge Panel
1: Employed	76.7 (0.0)	76.3 (0.1)	76.3 (0.1)	77.8 (0.6)	78.8 (1.6)
2: Unemployed	3.2 (0.2)	3.3 (0.2)	2.6 (0.4)	3.2 (0.3)	2.6 (0.5)
3: Not in the labor force	20.0 (0.2)	20.5 (0.2)	21.1 (0.4)	19.1 (0.5)	18.6 (1.6)
Total	100	100	100	100	100
Rao-Scott chi-square p-value Comparison to Production New Cohort Overall		*<0.0001
Rao-Scott chi-square p-value Comparison to Production New Cohort Analytical Sample			0.23		0.66

Source: U.S. Census Bureau, 2023 National Survey of College Graduates

Note: Standard errors are shown in parentheses.

*Indicates statistical significance at alpha 0.10.

Attachment B – Formulas

Breakoffs

A breakoff occurs when a survey user logs into the survey but does not complete it. If the individual leaves the survey on a particular screen but returns later and submits the survey, they are not included in the breakoff rates.

RQ: Of users who visited a screen, how many broke off the survey on that screen?

Screen-level Breakoff Rate:

Changed Answers

Changed answers to a survey question can occur while a respondent remains on the screen or if a respondent returns to a screen after advancing. Changed answers are generally flagged at the item-level. In the case of select-all-that apply questions, all checkboxes are considered together when determining the changed answer rate for that question.

RQ: Of all users that saw a question, how often did they change their answer to each item of the question?

Item-level Changed Answer Rate:

Previous Clicks

The “Previous” button allows the respondent to return to the prior screen in the survey instrument. Respondents generally use the “previous” button to verify or change a prior response, or to check the context of the previous question before answering the current one. “Previous” button clicks and changed answers to a survey question can both indicate respondent confusion with the question.

RQ: Of all the users who visited a specific screen, what percentage clicked “previous” on that screen?

Screen-level “Previous” Click Rate:

Item Nonresponse

The item nonresponse rate uses response data and provides insight into how burdensome or problematic a question might be for respondents. Rates are calculated using final weights.

RQ: Of respondents who were logically on-path to answer a given survey question, how many did not provide a response?

Item Nonresponse Rate:

Response Distribution

Response distributions use survey respondents’ answers and are generally edited and/or imputed. Each table notes where results are edited and/or imputed. Distributions are calculated using final weights.

Attachment C – Additional Results Tables

A.1 Additional Results of Sexual Orientation

Table 39 and Table 40 report a respondent’s sexual orientation, by their sex assigned at birth (Table 39) and their gender minority status (Table 40).

Table 39: Response Distribution – Sexual Orientation by Sex Assigned at Birth

Sex Assigned at Birth	Sexual Orientation	Percent of Respondents
Female	Bisexual	2.3 (0.5)
	Gay or lesbian	2.8 (0.7)
	Straight, that is not gay or lesbian	91.7 (1.4)
	I don't know	2.2 (1.1)
	I use a different term, specify	1.0 (0.4)
	Total	100
Male	Bisexual	0.9 (0.3)
	Gay or lesbian	3.2 (0.8)
	Straight, that is not gay or lesbian	94.4 (1.0)
	I don’t know	1.2 (0.4)
	I use a different term, specify	0.3 (0.2)
	Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table uses imputed data for Sex Assigned at Birth and Sexual Orientation.

A Rao-Scott chi-square test resulted in a p-value of 0.13.

Table 40: Response Distribution – Sexual Orientation by Gender Minority Status

Gender Minority	Sexual Orientation	Percent of Respondents
Yes	Gay or lesbian or bisexual	41.1 (17.5)
	Straight, that is not gay or lesbian	30.5 (13.8)
	I don't know	6.0 (4.4)
	I use a different term, specify	22.4 (13.4)
	Total	100
No	Gay or lesbian or bisexual	3.9 (0.6)
	Straight, that is not gay or lesbian	95.1 (0.6)
	I don’t know	0.7 (0.2)
	I use a different term, specify	0.4 (0.1)
	Total	100
Prefer not to answer	Gay or lesbian or bisexual	8.8 (4.8)
	Straight, that is not gay or lesbian	53.5 (10.8)
	I don’t know	37.7 (11.7)
	I use a different term, specify	0.0 (N/A)
	Total	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses. This table used imputed data for Gender Minority and Sexual Orientation.

A.2 Additional Results from the Gender Identity Probes

Table 41 reports the item nonresponse rates to GIP2 based on the respondent’s sexual orientation. For instance, Table 41 shows that, of the people who reported they were bisexual, 14.7 percent left GIP2 blank.

Table 41: Item Nonresponse Rate – Gender Identity Probe 2 by Sexual Orientation

Sexual Orientation	Percent of Respondents who Left Gender Identity Probe 2 Blank
Bisexual	14.7 (5.4)
Gay or lesbian	19.7 (7.3)
Straight, that is not gay or lesbian	23.1 (2.3)
I don’t know	2.3 (1.4)
I use a different term, specify	7.2 (6.0)
Left sexual orientation blank	29.2 (14.5)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Table 42 presents the gender identity probe 1 responses by gender identity probe 2 responses. For instance, of the respondents who agreed with the first statement on the GIP2 question, 51.0 percent said they were “Very comfortable” answering about their gender identity.

Table 42: Response Distribution – Gender Identity Probe 1 by Gender Identity Probe 2

	Gender Identity Probe 1
Gender Identity Probe 2	Very comfortable	Somewhat comfortable	Somewhat uncomfortable	Very uncomfortable	No Response GIP1	Total
I consider my gender identity to be my own private information	51.0 (3.8)	16.3 (3.6)	18.3 (4.3)	13.8 (2.0)	0.6 (0.4)	100
I have questions about how this information would be useful to the NSCG	51.2 (5.4)	21.1 (4.2)	15.9 (3.6)	D	D	100
I have concerns about how this information may be used	39.0 (5.5)	D	22.8 (5.2)	21.3 (5.9)	D	100
The response options do not reflect how I think of myself	90.8 (3.3)	3.0 (1.6)	1.4 (0.7)	4.9 (2.5)	0 (0)	100
I do not understand the question	63.0 (12.5)	10.8 (9.0)	15.6 (8.3)	D	D	100
I have other concerns – specify+	55.3 (10.7)	D	15.0 (9.5)	25.9 (9.9)	D	100
No response or no concerns+	88.3 (2.4)	3.8 (1.7)	0.8 (0.8)	0.7 (0.5)	6.4 (1.4)	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

(D) Suppressed for disclosure avoidance.

⁺There were respondents who selected “I have other concerns” to Gender Identity Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no response or no concerns percentages. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

A.3 Additional Results from the Sexual Orientation Probes

Table 43 reports the item nonresponse rate to SOP2, based on the respondent’s answer to the gender identity question.

Table 43: Item Nonresponse Rate – Sexual Orientation Probe 2 by Gender Minority

Gender Minority	Percent of Respondents who Left Sexual Orientation Probe 2 Blank
Yes	2.5 (1.7)
No	20.7 (2.5)
Prefer not to answer	8.1 (3.3)
Left GENMIN blank	77.1 (10.6)

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

Table 44 presents the sexual orientation probe 1 responses by sexual orientation probe 2 responses. For instance, of respondents who agreed with the first statement on the SOP2 question, 50.0 percent said they were “Very comfortable” answering about their sexual orientation.

Table 44: Response Distribution – Sexual Orientation Probe 1 by Sexual Orientation Probe 2

	Sexual Orientation Probe 1
Sexual Orientation Probe 2	Very comfortable	Somewhat comfortable	Somewhat uncomfortable	Very uncomfortable	No Response SOP1	Total
I consider my sexual orientation to be my own private information	50.0 (3.1)	18.9 (3.6)	18.1 (3.7)	12.7 (1.8)	0.2 (0.2)	100
I have questions about how this information would be useful to the NSCG	48.7 (5.2)	22.2 (4.1)	17.6 (3.7)	D	D	100
I have concerns about how this information may be used	40.7 (5.7)	D	25.2 (5.0)	18.6 (5.5)	D	100
The response options do not reflect how I think of myself	87.3 (3.9)	5.1 (2.3)	0.8 (0.5)	6.7 (3.0)	0 (0)	100
I do not understand the question	87.9 (8.0)	1.1 (0.9)	D	3.8 (2.8)	D	100
I have other concerns – specify+	51.2 (10.9)	9.3 (5.4)	11.3 (5.2)	13.8 (5.6)	14.4 (11.6)	100
No response or no concerns+	86.8 (2.6)	4.0 (1.8)	D	D	8.7 (1.9)	100

Source: U.S. Census Bureau, 2023 National Survey of College Graduates Response Data

Note: Standard errors are shown in parentheses.

⁺There were respondents who selected “I have other concerns” to Sexual Orientation Probe 2 and wrote in that they had no concerns or wrote in an unrelated response. These are included in the no response or no concerns percentages. Additionally, write-in responses were blanked and “backcoded” to a response option, when appropriate.

(D) Suppressed for disclosure avoidance.

1 The Census Bureau has reviewed this data product to ensure appropriate access, use, and disclosure avoidance protection of the confidential source data used to produce this product (Data Management System (DMS) number: P-7533594, Disclosure Review Board (DRB) approval number: CBDRB-FY24-POP001-0100).

2 The 2023 NSCG production cases were selected from the new cohort sampling frame, then the Bridge Panel was sampled from the remaining cases (U.S. Census Bureau, 2023). Cases with specific demographic characteristics were selected into the NSCG new cohort production sample with certainty (i.e., “take all” or “self-representative”) and were therefore not available to be included in the Bridge Panel. For example, all persons on the new cohort sampling frame who recently earned a bachelor's or professional degree in a social sciences field and who do not work in a science or engineering occupation were automatically included in the NSCG production sample. Thus, none of these persons were left on the sampling frame for the Bridge Panel.

3 The imputation approach used for these variables follows a similar process as for other NSCG survey questions, notably utilizing a hot deck allocation method. SO1 and SO2 used the same class and sort variables.

4 In the production instrument, if respondents provided the same answer on both the sex assigned at birth question and the gender identity question (e.g., “female”) and only selected one response for the gender identity question (SOGI2), they were not counted as a gender minority. If the answers to those questions did not match and were not blank, or if more than one option was selected for current gender identity, the respondent was considered to be a gender minority.

5 An excerpt from the 2021 National Survey of College Graduates Bridge Panel Analysis Results document: “A recent report released by the National Academies of Sciences stated that, “For data collections where respondents can easily skip over items if they do not wish to answer, it is not necessary to provide an explicit “prefer not to answer” response.” (National Academies of Sciences, Engineering, and Medicine, 2022). Other national surveys with questions about gender identity are testing questions that do not include this response option (National Center for Health Statistics (U.S.), 2022; Office of Management and Budget, 2022).”