Supplemental Supporting Statement Part B
2022 CENSUS OF AGRICULTURE
Substantive Change
OMB No. 0535-0226
COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided for the universe as a whole and for each stratum in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection, was conducted previously, include the actual response rate achieved during the last collection.
The target population for the Farm Producer Study (FPS) are all U.S. farmers. The FPS sampling frame – like the Census of Agriculture’s sampling frame - comprises all active farms on NASS’s list frame. The purpose of this study is to assess (1) the impact of the presence of disability and sexual orientation and gender identity (SOGI) questions and (2) the additional measurement error, if any, arising from the presence of the gender identity questions. Four treatment groups are to be identified: (1) control (none of the test questions are included); (2) only disability questions; (3) only SOGI questions; and (4) both disability and SOGI questions. The total sample size will be about ~75,000. The sample size for each of the control group (group 1) and the treatment group receiving only the disability questions (group 2) will be about 12,500. The sample size for each treatment group with the SOGI questions (groups 3 and 4) will be about 25,000. The two treatment groups with SOGI questions will each be split into two subgroups: respondents in one subgroup will be asked a confirmation question if their recorded gender at birth differs from their present gender and respondents in the other subgroup will not be asked the confirmation question. Notice that the total sample size for the disability and SOGI questions will be, respectively, about 37,500 and 50,000. With this design, it will be possible to assess whether each set of new questions (disability or SOGI) has an impact on response rates and whether the presence of two sets of new questions has an additive impact on response rates or whether some interaction is present. Dividing the treatment groups with SOGI questions into two subgroups allows measurement error to be assessed when respondents report a difference in their gender recorded at birth and their gender at present.
To address the potential that some groups may be less—or more—likely to respond when asked one or more of the new questions (disability or SOGI), the FPS Sampling Frame will be stratified.
A separate stratum will be formed for LGBTQ+ operations and for groups with historical response rates lower than the overall census response rate: American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanics, and women (farms with only women producers) with farm records being assigned to strata in the order specified. For example, if a farm has an Asian female principal producer, then that farm would be assigned to the Asian stratum. All other farm records will be in the final stratum. Each of these seven strata, will be further stratified by Census region, state, farm type groups, and farm value of sales groups. The control group will receive a standard 4-page questionnaire. In contrast, each treatment group will receive one or more sets of test questions (disability or SOGI). For one of the two subgroups of each treatment group with SOGI questions, those responding via a web or phone interview and reporting a different gender than the one recorded at birth will be asked an additional confirmation question.
This is a new experimental data collection. There are no previous response rates to report. NASS will attempt to collect as many positive reports as possible in the allotted amount of time during data collection.
2. Describe the procedures for the collection of information including:
• statistical methodology for stratification and sample selection,
• estimation procedure,
• degree of accuracy needed for the purpose described in the justification,
• unusual problems requiring specialized sampling procedures
To address the potential that some groups may be less—or more—likely to respond when asked one or more of the new questions (disability or SOGI), the FPS Sampling Frame will be stratified. A separate stratum will be formed for each of the following groups: LGBTQ+, American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanics, and women, with farm records being assigned to strata in the order specified. For example, if a farm has an Asian female principal producer, then that farm would be assigned to the Asian stratum. All other farm records will be in the final stratum. Each of these six strata, will be further stratified by Census region, state, farm type groups and farm value of sales groups. The LGBTQ+ stratum consists of about 5,000 records that are more likely to have at least one LGBTQ+ principal producer based on responses to the 2017 Census of Agriculture. The LGBTQ+ stratum will be sampled with certainty. The next five strata (American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanics, and women) will be sampled at a relatively high rate with about 7,500 farms being selected from each of these strata. Thus, the first six strata will have about 42,500 records in the sample. The sample from the strata will be drawn proportional to the size of the strata. The remaining ~32,500 farm records will be drawn from the last stratum. This portion of the sample will be allocated proportional to the 2017 Census of Agriculture published number of farms by Census region-farm type-farm value of sales combinations. Each stratum will be sorted by Census region, state, farm type and farm value of sales before assigning replicates and randomly selecting replicates for the treatment and control groups.
The data collection plan for this experimental study will consist of the following:
Send a traditional paper questionnaire with a cover letter. This mailing will also encourage reporting on the web.
For all non-respondents, send a second traditional paper questionnaire with a cover letter. This mailing will also encourage reporting on the web.
Conduct non-response follow-up with enumerators via computer assisted telephone interviewing (CATI).
Respondents in the confirmation question subgroup of any treatment group with SOGI questions will be asked a confirmation question if their recorded gender at birth differs from their reported present gender. Respondents in the other subgroup of those treatment groups will not be asked the confirmation question. This should be result in approximately half of web or CATI respondents in treatment groups with SOGI questions who have differing responses to the two gender questions receiving the confirmation question.
3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
Both mailings of the questionnaires will include a cover letter with information explaining the purpose and importance of the data being collected. Respondents will be able to reply either by mail or by using the web instrument. After the questionnaire mailings, NASS will attempt to collect data from non-respondents via a computer-assisted phone interview.
4. Describe any tests of procedures or methods to be undertaken.
The proposed experimental data collection (The Farm Producer Study) will collect data on two additional demographic topics, as compared with demographic data NASS has previously collected. The additional demographic topics are: (1) disability status and (2) sexual orientation and gender identity (SOGI). This will provide baseline information to potentially incorporate questions pertaining to these topics into future Censuses of Agriculture and other surveys NASS conducts. The Farm Producer Study will have the objectives of determining the impact on response rates when including questions on these topics and to determine the measurement error associated with mail response to the gender identity questions.
The experimental data collection will have the following goals. The associated analysis and decision process are explained for each goal.
Goal: To determine whether the presence of questions on disability and SOGI impact response rates.
Analysis: Response rates for the four treatment groups will be compared using a generalized linear model with a Bernoulli response, weights equal to the sampling weight, and fixed effects of indicators for the presence of each of the set of questions on disability or SOGI and the interaction between the disability and SOGI questions, response mode, and indicators for respondents who are LGBTQ+, American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanic, or women. Note that the indicator for SOGI questions will be one if the set of SOGI questions occurs on the questionnaire, whether disability test questions are on the questionnaire or not. Also, as an example, a respondent who is an Asian woman will have ones for the indications of Asian, and women and zeroes for the indicators of LGBTQ+, American Indian/Alaska Native, Hispanic and Black. If sufficient data are present, the effects of Census region, state, farm type and farm size and additional interactions among these effects will be explored. In addition, a similar analysis of item response rates for the disability and SOGI questions will be conducted to determine whether the item response rates differ significantly with treatment group.
Significant differences in the unit and item response rates for the main effects will be determined for the presence of disability or SOGI questions. In addition, tests will be conducted to determine whether the response rates to the disability or SOGI questions differ for the demographic groups with historically lower response rates: LGBTQ+, American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanics, and women. If a significant difference in unit or item response rates is observed for one or more of the sets of questions either overall or for one of the demographic groups with historically lower response rates, the hypotheses H0: r1 – r2 ≤ 5% versus H1: r1 – r2 > 5% will be conducted for the significant question sets. Here r represents a response rate (either unit or item) and the subscript 1 corresponds to the group that did not have the set of questions of interest, and the subscript 2 represents the group that did have that set of questions.
Decision: For each set of questions (disability or SOGI), if the presence of that set of questions results in a significant decrease in unit and item response rates of more than 5% for all farm producers or for any of the groups of interest (American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanics, and women), then that set of questions will not be asked on the 2027 Census of Agriculture.
Goal: To determine whether measurement error is associated with estimating the proportion of producers who have a current gender identity different from the one recorded at birth.
Analysis: Only the treatment groups receiving the SOGI questions (treatments 3 and 4) will be considered in this analysis. The proportion of persons responding via the web, CAPI, or CATI and identifying as transgender or other (non-binary, for example) within the two subgroups (with and without a confirmation question) will be compared to each other and to that proportion reported via mail. The analysis will be a generalized linear model with a Bernoulli response reflecting whether the respondent has a current gender identity differing from that recorded at birth, weights equal to the sampling weight, and fixed effects of treatment, subgroup, treatment by subgroup interaction, and indicators for respondents who are LGBTQ+, American Indians/Alaska Natives, Asians/Native Hawaiians and Pacific Islanders, Blacks, Hispanics, or women. It is anticipated that no significant difference will be observed in the gender identity proportions in the mail and the treatment subgroups with no confirmation question. However, a difference may be observed in the gender identity proportions for the treatment subgroups with a confirmation question and the other two groups. If so, the measurement error associated with a lack of a confirmation question will be estimated.
Decision: If the measurement error is more than 100%, NASS will not move forward with gender identity questions on the 2027 Census of Agriculture.
Prior to implementation of the Farm Producer Study, nine cognitive interviews were conducted by NASS survey methodologists; five respondents were members of the LGBTQ+ community and four respondents were persons with functional limitations and/or disabilities. These cognitive interviews were intended to assess the respondents’ question response process and their reactions to the disability and SOGI questions.
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name agency unit, contractor(s), grantee(s), or others who will actually collect and/or analyze the information for the agency.
Several NASS units contributed to developing census of agriculture methodology, each containing staff members with prior agriculture census experience. Contributing senior staff and unit leaders are:
Mark Apodaca, Chief, Sampling, Editing, and Imputation Methodology Branch: (202) 720-2857),
Jeff Bailey, Chief, Summary, Estimation, and Disclosure Methodology Branch: (202) 720-6468,
Dan Beckler, Chief, Standards and Survey Development Methodology Branch: (202) 720-8858,
Donald Buysse, Chief, Census Planning Branch: (202) 690-8747,
Data collection is carried out by NASS Field Offices; Eastern Field Operation’s Director is Jody McDaniel (202) 720-3638 and the Western Field Operation’s Director is Troy Joshua, (202) 720-8220,
Tony Dorn, Chief, Environmental, Economics, and Demographics Branch: (202)720-6146,
Suzette Qualey, Deputy Director, National Operations Division: (314) 595-9502,
Virginia Harris, Demographer: (502) 907-3211,
Linda J. Young, Director, Research and Development Division: (202) 690-1401.
September 2021
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | hancda |
| File Modified | 0000-00-00 |
| File Created | 2021-10-04 |