SuptStmt-Part B Local Food Directory and Survey Final-072016

2016 Supporting Statement – Part B

Local Food Directories and Survey

OMB NO. 0581-0169

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. DESCRIBE (INCLUDING A NUMERICAL ESTIMATE) THE POTENTIAL RESPONDENT UNIVERSE AND ANY SAMPLING OR OTHER RESPONDENT SELECTION METHOD TO BE USED. DATA ON THE NUMBER OF ENTITIES (E.G., ESTABLISHMENTS, STATE AND LOCAL GOVERNMENT UNITS, HOUSEHOLDS, OR PERSONS) IN THE UNIVERSE COVERED BY THE COLLECTION AND IN THE CORRESPONDING SAMPLE ARE TO BE PROVIDED IN TABULAR FORM FOR THE UNIVERSE AS A WHOLE AND FOR EACH OF THE STRATA IN THE PROPOSED SAMPLE. INDICATE EXPECTED RESPONSE RATES FOR THE COLLECTION AS A WHOLE. IF THE COLLECTION HAD BEEN CONDUCTED PREVIOUSLY, INCLUDE THE ACTUAL RESPONSE RATE ACHIEVED DURING THE LAST COLLECTION.

The respondent universe is the markets listed in USDA’s National Farmers Market Directory, around 8,500 markets at present. USDA annually sends reminder emails to: market managers of farmers markets listed in the Directory that have email addresses, State Departments of Agriculture, and State Farmers Market Associations. The Farmers Market Coalition also assists the Department with the annual update push each year by encouraging its membership to list and update their listing in the Directory. The Public Affairs Office of AMS writes blogs and tweets about the importance of listing and updating existing listings in the Directory. AMS conducts internet searches to determine if markets that have not updated their directory listings are still in operation and to collect current email addresses for these farmers markets where possible. Invitations to request that these markets update their directory listings are then sent to the email addresses collected from the website searches. AMS also requests that State farmers market representatives and State Farmers Market Association representatives review the USDA National Farmers Market Directory listings for their state and identify those markets that are no longer in operation.

The survey is a census of an opt-in sampling frame. Every farmers market manager that enters information into the directory update system is invited to participate in the survey. There are no limits for the sample size during the managers' survey period. In data analysis and interpretation stage, we will conduct an analysis (e.g., Chi-Square test) to examine whether the distribution of survey responses by region and market size represents the distribution of markets by region and market size listed in the directory (Table 1). If not, a weight score based on the region and farmers market size in the directory will be applied to the survey dataset for further analyses. Other methods, e.g., non-response adjustment, may be applied in the future based on the distribution of the responses and the research topics and needs.

Limitations of the Study

The findings of this collection are limited to describing the farmers markets that are included in USDA’s National Farmers Market Directory during the data collection year, and should not be generalized to the entire population of farmers’ markets in the US. Care should be taken when generating trends or comparing results across years as any differences may reflect changes in the composition of the sample rather than changes in the general population.

Sampling of the Farmers Markets and Survey Administration

Element 1: Farmers Market Directory Update Form

All farmers market respondents will be asked to complete the USDA’s National Farmers Market Directory form first. Responding farmers market managers that completed the form in 2016 will have the ability to access input forms that contain portions of their previously submitted market information (This will reduce the time burden associated with completing the form, since it is anticipated that the vast majority of respondents will have completed a form during the previous year. Based on market managers that updated their listing information in 2015, it is estimated that it will take an average of 8 minutes for new markets to complete the form. Once they complete the form, the data will automatically be stored in a database.

On the last page of the Directory update form, they will be instructed that to make changes in their directory information during the course of the year.

Annual Farmers Market Managers Survey

Farmers markets that complete and submit the directory update form will be brought to a review page for their Directory listing. Here they will see their submitted data in a format similar to how it will appear in the USDA Directory search engine, and will have the opportunity to edit the data to correct errors (e.g., typos, omissions). On this page, they will also be informed that the USDA is also in the process of conducting its 2016 National Farmers Market Managers Survey and asked if they would be willing to complete it. They will be able to: (1) refuse, in which case they will exit the survey and will not receive any further email correspondence concerning the survey, (2) agree, but at a later time, in which case they will receive an email with a personal link to the survey and three reminders over the course of the next three weeks if they do not complete it, or (3) agree to complete the survey at the present time, in which case they will be immediately connected to USDA’s National Farmers Market Managers Survey.

Element 2: Core Component of the Annual Farmers Market Managers Survey

The core component of the Annual National Farmers Market Managers Survey will be completed first and by all respondents, who agreed to take it. It will be data-driven insofar as respondents will only see questions that are specifically relevant to them based on a combination of the data provided on the Directory update form and answers to earlier questions on the survey. It is estimated that it will take 8 minutes for the average respondent to complete. If they do not complete all the core questions, they will be sent a reminder asking whether they would be willing to complete the remainder of the survey, and a link that will allow them to resume at the place where they left off.

Element 3: Special Modules

Nine different modules covering various topics have been submitted for OMB approval. The modules contain a different number of questions and will require different amounts of time on the part of respondents. Three of the modules, Market Performance, Budget and Finance, and Competition, will be administered on an annual basis to proportional random samples of the responding farmers markets. The other six modules will alternatively be administered. In odd years the three modules that will be included are business incubator, health and healthy eating and diets, and farmers market vendors. In even years the three questions modules that will be included are future market success, farmers market managers, and fresh locally-grown fruits and vegetables. Qualified respondents will be assigned to modules in real time. Before a modular assignment is made, the computer program takes into account the respondents that have already been assigned to modules while the survey is being administered. Often a respondent qualifies for multiple modules; the computer program makes the modular assignment that best maintains proportionality, so as to keep the regional and size proportions of the sample similar to the regional and size proportions of farmers markets listed in USDA’s National Farmers Market Directory during the data collection year.

Respondents that complete and submit the core survey will automatically proceed to three blocks consisting of 2-3 different special modules. In subsequent years, different modules will be incorporated as part of the survey. They will be combined in 2 blocks so that the time requirement does not exceed 22 minutes and the total time to complete the Directory update form (8 minutes), core survey (8 minutes) and modules (no more than 6 minutes). For example in 2016 there will be three blocks as follows: Block 1 includes modules on Competition and Vendors, Block 2 includes modules on Market Performance and Expansion , and Block 3 will include modules on Budget/Finance and Market Enhancement.

The survey is based on a census of the opt-in sampling frame. Every farmers market manager that enters information into the directory update system is invited to participate in the survey. There are no limits for the sample size during the managers' survey period. In data analysis and interpretation stage, we will conduct an analysis (e.g., Chi-Square test) to examine whether the distribution of survey responses by region and market size represents the distribution of markets by region and market size listed in the directory (Table 1). If not, a weight score based on the region and farmers market size in the directory will be applied to the survey dataset for further analyses. Other methods, e.g., non-response adjustment, may be applied in the future based on the distribution of the responses and the research topics and needs. Non-response bias adjustments have not been needed for previous reports. The sample(s) have had similar regional and size distributions as the USDA’s National Farmers Market Directory.

TABLE 1. Distribution of Markets in the Directory

	9 or Fewer Vendors on a Typical Market Day	10 or More Vendors on a Typical Market Day
Region 1: Northeast	7.8%	15.5%
Region 2: Midwest	7.1%	21.6%
Region 3: South	7.4%	19.8%
Region 4: West	2.6%	18.3%

The estimate for the respondent universe pertaining to the three direct to customer marketing directories that will be merged into this collection is as follows:

On-Farm Markets 50,000 operations

Community Supported Agriculture (CSA’s) 6,500 operations

Food Hubs 250 operations

Total 56,750 operations

The data developed by this information collection will be used to populate three on-line direct marketing directories in these three marketing channels.

There is no sampling procedure used in this section of the collection.

2. DESCRIBE THE PROCEDURES FOR THE COLLECTION OF INFORMATION INCLUDING:

- STATISTICAL METHODOLOGY FOR STRATIFICATION AND SAMPLE SELECTION;

- ESTIMATION PROCEDURE;

- DEGREE OF ACCURACY NEEDED FOR THE PURPOSE DESCRIBED IN THE JUSTIFICATION;

- UNUSUAL PROBLEMS REQUIRING SPECIALIZED SAMPLING PROCEDURES, AND

- ANY USE OF PERIODIC (LESS FREQUENT THAN ANNUAL) DATA COLLECTION CYCLES TO REDUCE BURDEN.

The e-mail invitation to participate in the directory update and the national farmers market managers’ survey will be sent to all participate in the Directory. The invitation will include a web link to the directory update. After market managers/market representatives complete their listing or listing update they will be invited to participate in the national farmers market managers’ survey. AMS has made every effort to gather a complete listing of e-mail addresses of farmers markets when farmers market managers update their market contact information on the Directory. Currently AMS has e-mail addresses for 6,845 markets, representing 80.5 percent of the approximately 8,503 markets in the Directory. In addition AMS will announce the Directory update and the voluntary market manager survey on the Department’s website, contact farmers market associations, State Departments of Agriculture, and the Farmers Market Coalition. Theses stakeholders will be informed that the survey can be completed online after their market is listed or updated in the Directory and we will provide them with the address of the farmers market directory update/listing link. Any market that lists or updates their listing in the farmers market directory can choose to participate in the national farmers market managers’ survey.

We will again be working with Michigan State University’s Department of Community Agriculture Recreation and Resource Studies to develop the survey website as we did in 2014. In addition to website development, MSU will develop the database used to save data from survey respondents and will import that raw data into an SPSS database for subsequent analysis.

3. DESCRIBE METHODS TO MAXIMIZE RESPONSE RATES AND TO DEAL WITH ISSUES OF NON-RESPONSE. THE ACCURACY AND RELIABILITY OF INFORMATION COLLECTED MUST BE SHOWN TO BE ADEQUATE FOR INTENDED USES. FOR COLLECTIONS BASED ON SAMPLING, A SPECIAL JUSTIFICATION MUST BE PROVIDED FOR ANY COLLECTION THAT WILL NOT YIELD "RELIABLE" DATA THAT CAN BE GENERALIZED TO THE UNIVERSE STUDIED.

We estimate that 3,500 market managers will update their market listing on USDA’s National Farmers Market Directory. We believe that 2,000 market managers, (57 percent) will choose to complete USDA’s National Farmers Market Managers Survey, and that a little more than half of the market managers that list their market in the Directory will be willing to spend the additional 14 minutes to complete the market manager questionnaire. AMS requests that State Department of Agriculture, farmers market representative(s), and State Farmers Market Associations review the Directory listing for their state to identify any markets that are no longer in operation. In addition when AMS invites market managers to update their listing a number of the managers’ report that their market is no longer in operation. AMS removes markets from the Directory that are reported closed. AMS also performs website searches to determine if markets that have not updated are still in operation and to collect updated email addresses for farmers markets that have not updated their listing and invites them to update their information so that their customers can have the most current information about their market.

Those managers who listed their market in the Directory but choose not to complete the Survey will be analyzed to determine non-response bias. The sample was not found to be different in terms of the regional and size distribution of the listings in USDA’s National Farmers Market Directory.  AMS compared the sample distribution to all listings in the Directory that did not respond to the survey.  Some markets that did not participate in the survey did make updates to their listing in the same year that the survey was administered while other markets that did not participate in the survey updated their listing in previous years. The number of markets that reported size and location information was approximately 5,000 listings.  Currently there are 5,328 listings in the Directory report with region and size data.

The five questions in the Directory update form that we plan to focus on in our non-response survey analysis are:

3. Are you a manager or a representative of another farmers market in 2016?

12. In 2016, how many different locations will this farmers market operate with the same market name?

13. Please provide information about the physical location (actual place) where the market will operate. If there is not a street address, number or zip code for this location, or you do not know it, please the state, city/ town and a description of the location with the nearest road intersection listed (e.g., on the town center green, mall parking lot at Main St. x Tree St.). The Directory will include this description to assist customers in locating the market. After you enter the information click on the box below and an arrow will appear pointing to the location of the market. If it is not in the correct location of the market in 2016, please drag the arrow to the correct location. If you lose sight of the arrow (by zooming in, for example), you can click anywhere on the map and a new arrow will appear.

18. During which months will this farmers market operate at this location in 2016?

22. On a typical market day, how many producers/ vendors do you expect will be selling at this farmers market at this location in 2016?

The answer to these questions in the Directory update, which captures various elements of market scale and scope, should enable us to identify any major sources of bias that might exist in our sample pool.

4. DESCRIBE ANY TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN. TESTING IS ENCOURAGED AS AN EFFECTIVE MEANS OF REFINING COLLECTIONS OF INFORMATION TO MINIMIZE BURDEN AND IMPROVE UTILITY. TESTS MUST BE APPROVED IF THEY CALL FOR ANSWERS TO IDENTICAL QUESTIONS FROM 10 OR MORE RESPONDENTS. A PROPOSED TEST OR SET OF TESTS MAY BE SUBMITTED FOR APPROVAL SEPARATELY OR IN COMBINATION WITH THE MAIN COLLECTION OF INFORMATION.

AMS has tested this collection instrument with MSD staff that manages USDA’s farmers market to evaluate the time necessary to complete the questionnaire and to evaluate the instrument for understanding and clarity.

5. PROVIDE THE NAME AND TELEPHONE NUMBER OF INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS OF THE DESIGN AND THE NAME OF THE AGENCY UNIT, CONTRACTOR(S), GRANTEE(S), OR OTHER PERSON(S) WHO WILL ACTUALLY COLLECT AND/OR ANALYZE THE INFORMATION FOR THE AGENCY.

AMS has consulted with David Hancock of USDA’s National Agricultural Statistical Service (202-690-2388) and Dr. Yue Cui of Michigan State University-Community Recreation and Resource Studies Department (571-432-0285) on developing the survey strategy. The AMS/MSD employee who will collect and/or analyze the survey information is Edward Ragland (202-690-1327).

7