Generic Testing – Local Foods
OMB No. 0535-0248
This mini-supporting statement is being submitted to OMB to define the need for conducting qualitative research, in the form of exploratory interviews and cognitive interviews, in an effort to reduce measurement error in the next iteration of the Local Foods Marketing Practices Survey. The next iteration of the survey will most likely be in January 2021 – April 2021, referencing the 2020 calendar year. No more than 90 operations will be interviewed in this research effort.
The 2015 Local Food Marketing Practices Survey was conducted as a new information collection in 2016 to gather data related to the production and marketing of food from the producers to consumers. The questionnaire collected information on market channels, value of sales, marketing expenses, practices, third party certification and food safety, and personal characteristics. Responses to the survey were voluntary.
Evidence from data analysis of the 2015 data suggests some respondents had difficulty comprehending some questions on the survey and forming responses that satisfied those data requests. As such, survey methodologists at NASS would like to use exploratory and cognitive interviews to refine the existing questionnaire prior to the next survey iteration. Specifically, NASS survey methodologists would like to answer the following research questions:
What do respondents grow and where do they sell it?
When they think of local marketing, what marketing channels come to mind?
What local marketing channels do they use?
How are their records kept to document what is sold where?
How do respondents think of the local foods marketing practices concepts identified in the survey?
When they talk about marketing their products locally, what terminology do they use?
Is it the same as our terminology or something different?
A. JUSTIFICATION
Circumstances making collection of information necessary.
Prior to the launch of the 2015 Local Food Marketing Practices Survey, nine cognitive interviews were conducted to assess the efficacy of the new information collection. These nine interviews uncovered potential problems that may have influenced the responses received. For example, during the cognitive interviews, many respondents were unfamiliar with the term “intermediate markets.” This was further corroborated during the follow-on Local Foods Marketing Practices Misclassification Survey (the primary purpose of this survey being weighting). Findings from analysis of the misclassification data illustrated that a large number of operations changed their status (in-scope to out-of-scope and vice-versa) during the question that probed on the intermediate markets construct. This example, along with several other problems identified with the concepts, questions, and question ordering on the questionnaire during cognitive testing, submission of post survey suggestions from NASS staff, and data analysis, illustrates the need for further research to reduce potential measurement error related to the questions and concepts in this survey and supplemental materials.
How, by whom, and for what purpose information is to be used.
The information gathered through the qualitative research will be analyzed by NASS’s Methodology Division and Census and Survey Division to determine the content, as well as the best way to ask questions about the constructs of interest on the next Local Foods Survey.
The types of questions used for the qualitative research are shown in Attachments A and B. The types of questions proposed for the exploratory interviews are shown in Attachment A and a draft cognitive interview guide are shown in Attachment B.
Use of improved information technology.
The qualitative research proposed would be done on the phone or in person with NASS or NASDA staff.
4. Efforts to identify duplication.
No additional efforts will be taken to identify duplication. Operators who are selected for the qualitative research interviews will be drawn from NASS list of known farm operations who fit, or are likely to fit, the screening criteria for the survey. Duplication is removed before operators are added to the NASS list frame.
Methods to minimize burden of small businesses.
For the qualitative research interviews, no special efforts will be made to minimize burden of small businesses.
Consequence if information collection were less frequent.
The qualitative research is planned to be conducted during the two and a half year period between July 2017 and June 2019, as resources become available. This docket expires in June 2019, so interviews not completed by that time will be resubmitted with a mini-supporting statement after this docket (0535-0248) is renewed
Special circumstances.
There are no special circumstances associated with this information collection.
Federal register notice and consultation with outside persons.
Not applicable.
Payments or gifts to respondents.
There are no payments or gifts to respondents.
Confidentiality provided to respondents.
The same confidentiality that is applied to the Local Foods Survey will be applied to data collected during the qualitative research interviews.
Questions of a sensitive nature.
The Local Foods Survey contains questions related to farm related income, some operating expenses, and demographics that could be considered sensitive. These are similar questions to those that have been used on the Local Foods Survey, the Census of Agriculture, and other surveys in the past.
12. Hour burden and annualized costs to respondents.
All interviews will be conducted by trained cognitive interviewers and survey methodologists at NASS headquarters to explore how respondents understand the underlining constructs within the questionnaire, and make sure that the respondents understand the intent of each question and are able to provide the requested information.
Qualitative Interviews |
Number of Respondents |
Hours per Interview |
Total Burden Hours |
Exploratory Interviews |
50 |
0.5 hours |
25 |
Cognitive Interviews |
40 |
1.5 hours |
60 |
Total |
90 |
|
85 |
NASS uses the Bureau of Labor Statistics’ Occupational Employment Statistics (most recently published on March 31, 2017 for the previous May) to estimate an hourly wage for the burden cost. The May 2016 mean wage for bookkeepers was $19.34. The mean wage for farm managers was $36.44. The mean wage for farm supervisors was $23.47. The average of the three is $26.42. The annual estimated reporting time of 85 hours is multiplied by $26 per hour for a total cost to the public of $2,210.
Total annual cost burden to respondents.
There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.
Annualized costs to federal government.
Costs for conducting the qualitative research interviews are estimated at $75,000. This will cover expenses for staff payroll, travel, survey analysis, and any other expenses that may be incurred while updating survey materials based on our findings. Several NASS employees who have been trained in conducting cognitive interviews in several states will conduct the cognitive interviews.
Reasons for changes in burden.
This mini-supporting statement addresses the use of burden to conduct testing for the Local Foods Survey.
Tabulation, analysis, and publication plans.
No data will be published from these tests. Data are for internal use only, but results may be presented at outside conferences or seminars.
Request for approval of non-display of expiration date.
There is no request for approval of non-display of the expiration date.
18. Exceptions to certification statement.
There are no exceptions to the certification statement.
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS:
Respondent universe, sampling, and response rate.
NASS will use list frame data and survey data to identify operations or operators that meet the screening criteria for the Local Foods Survey. The operators will not be statistically sampled; rather, they will be hand selected based on whether they market foods locally, as well as their characteristics, such as demographics, size, number and types of commodities produced, and geographic location. HQ staff, NASS State Statisticians, Regional Field Office staff, and NASDA interviewers will recruit respondents for the interviews.
Procedures for the collection of information.
Interviewers will follow standard pretesting techniques as defined in the original Supporting Statement Part A for the Generic Clearance docket (0535-0248).
Information collected adequate for intended uses.
Operations will be selected based on specific criteria as stated above.
Test of procedures or methods.
Not applicable.
Individuals consulted on statistical aspects of survey.
Selection of methods of testing for this qualitative research was done by the Methodology Division; Chief Cognitive Research Methodologist is Jaki McCarthy, (202) 690-2389. Senior Survey Methodologist is Kathy Ott, (202) 720-1114.
April 2017
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | HancDa |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |