NASS comment--Farmers Market Questionnaire

Review of OMB Documentation for the AMS Farmers Market Questionnaire

March 15, 2016

Reviewed By: Sarah Goodale, NASS, Summary Estimation and Disclosure Methodology Branch

Part A.

No comment on this section of the document

Part B.

Upon reading the documentation it appears as if all the farmers markets that are on the USDA National Farmers Market Directory are eligible to participate. In the first part of the process the farmers market is asked to update their information. Then they are asked if they would like to participate in the National Farmers Market Managers Survey. By doing this all farmers markets on the National Farmers Market Directory are eligible for this additional survey (no sampling design needed). The document only refers to farmers markets already on the National Directory. Will there be any new markets added to Directory during this process and how they will be treated?

The document then refers to 9 different special modules covering various topics. Three of the modules will be administered on an annual basis and have a proportional random sample of responding farmers markets. There is no estimate of sample size. Shouldn’t there be a minimum and maximum number of units to be sampled? I know that this is based on who responds but is there some guide or expected number of responses? There is also no mention on how the additional 6 special modules will be handled. How they will be sampled? How often they will occur?

After mention the proportional random sample the document then says that these modules will be assigned using a simple random stratified sampling method. This is slightly different the proportional random sample. It appears as if the directory is separated into the strata and the proportion of farmers markets that fall into the strata was calculated. The intention appears to then select within the strata the same proportion of farmers markets. Based on the Table 1 in the document, 8% of the farmers markets on the directory are in strata 1. What if 8% of this strata does not respond? However the document then states that the proportion of completed surveys within the strata will be selected. Taking only the proportion of the strata that responded for a further sample will create an under covered biased sample. Also the document says that survey results will be weighted using the directory. What methodology will be used to determine the weight?

The document also says “There is no sampling procedure used in this section of the collection” Are they referring to the National Survey that is voluntary? Or the special modules where a sample design name is mentioned with no other details?

It appears that the farmers markets will only be contacted by email. This is only approximately 80% of the farmers markets on the National Directory. What about the other 20% that do not have a valid email address associated with it; will they be excluded?

The document does provides approximate numbers and percentages of markets that will respond to update their information and who will respond to the survey. Again no sample numbers for the special projects are given.

Beyond the small mention in the document that the sample will be weighted to the 2016 directory there are no other weighting factors mentioned in the supporting statement. Will there be any weighting adjustments? Non-response adjustment?