2014 Organic Survey - Promotional Materials Usability Testing

2014 Organic Survey – Promotional Materials Usability Testing
October 20, 2014
Heather Ridolfo, Project Lead
Research and Development Division

Interviewing Team: Kathy Ott, Dale Hawks, Cheryl Turner, Bruce Eklund, & Gary Keough
Background
The 2014 Organic Survey is a data collection project conducted by NASS for the Risk
Management Agency (RMA); Census funds will also be used as directed by the 2014 Farm
Appropriations Bill. The survey is an organic production and practices survey, parts of which
have been previously conducted in 2008 and 2011. The population of interest is any operation
that produces organic products according to USDA’s National Organic Program (NOP)
standards and is certified or exempt from certification. The population also contains operations
that reported in the 2012 Census of Agriculture that they were transitioning to organic. The
questionnaire collects information on organic production of field crops, vegetables, fruits, tree
nuts, berries, livestock and poultry, production practices, production expenses, marketing
practices and value-added production and processing.
Prior to receiving the survey, respondents will all receive a postcard indicating the survey will be
arriving soon and reasons it is important to respond. At the start of the data collection period,
respondents will receive the survey questionnaire with a cover letter that details who is
conducting the survey, the purpose of the survey, how to respond, and who to contact for more
information. Respondents are also informed that the survey is required and protected by law. At
the end of the data collection period, respondents will receive a reminder letter which will
encourage them to respond to the survey if they have not already done so. The reminder letter
will also stress the importance of responding, how they can complete the survey, who to contact
for more information and that the survey is required and protected by law. Respondents are also
informed in this letter that a NASS representative will follow up with them if we do not receive
their data.
Before sending these materials out to the public, we wanted to ensure that we are using effective
messaging in these materials. That is, we want to make sure the information presented in these
materials is understandable, accessible, valuable, and credible. The following report presents
findings from six usability interviews conducted to assess these factors.
Methods
Sample
Six interviews were conducted in the following states: VA, MD, OH, NJ, and NH. The sample
included operations that grow organic crops and are certified or exempt from certification.
1

Interviewers in each state were responsible for recruiting respondents. Respondents were
informed of the intent of the testing during the recruitment process and that their participation
was voluntary.
Interviewing Procedures
All interviews were conducted in October 2014. Interviewers were provided an interview
protocol and interview guide prior to data collection (See Appendix A). Copies of the postcard,
cover letter and reminder letter used during testing can be found in Appendix B.
At the start of the interview, respondents were informed of the task. Respondents were told we
are testing the promotional materials that will be used for the 2014 Organic Survey and we
wanted to make sure the materials were informative and that everyone understands the
information presented. Respondents were then handed the postcard and instructed to pretend they
had just received it in the mail. Interviewers were instructed to record any initial reactions the
respondents had to the postcard and note how the respondents went about reviewing the material
on the postcard. When respondents were done reviewing the postcard they were asked how much
of the postcard they read. Respondents were then asked a series of questions to assess their
reactions to the information provided. Respondents were asked such things as: What information
stands out? Is there any information you think is missing? Is it helpful to receive a postcard like
this? (See Interview Guide in Appendix A for a complete list of questions).
After providing feedback on the postcard, respondents were then asked to review the cover letter.
Respondents were handed the cover letter and a copy of the survey questionnaire and asked to
review it as if they had just received it in the mail. Interviewers were instructed to record any
initial reactions the respondents had to the cover letter and note how the respondents went about
reviewing the information on the cover letter. When respondents were done reviewing the cover
letter, the interviewers took the cover letter back and asked the respondents how much of the
cover letter they read. Respondents were then asked a series of questions to test their recall of
important information presented in the letter. Respondents were asked if they could recall such
things as: who was conducting the survey; what was the purpose of the survey; how could they
respond to the survey; when was the survey due? Respondents were then handed the cover letter
back. If respondents were not able to answer the recall questions, they were then instructed to try
and find that information on the cover letter and interviewers noted their ability to do so.
Respondents were then asked a series of questions to assess their reactions to the letter.
Respondents were first asked about interpretation and reactions to two messages in the letter.
The first message stated: This survey is part of the Census of Agriculture Program which is
required and protected by law (Title 7, U.S. Code and the Confidential Information Protection
and Statistical Efficiency Act). These federal laws require that NASS keeps your identity and
your answers confidential?” The second message stated: “Responding online is the easiest,
fastest, and safest way to complete your questionnaire and it saves taxpayer dollars too.”
Respondents then answered question regarding their general reactions to the letter, such as: what
information was most important to them; what information was missing; what information was
unnecessary?

2

Finally, respondents were handed the reminder letter. Once again, respondents were instructed to
review the letter as if they had just received it in the mail and interviewers noted respondents’
initial reactions to the letter and how they went about reviewing the information in the letter.
When respondents finished reading the letter, they were asked how much of the cover letter they
read. Respondents were then asked about their reactions to the letter. They were asked probing
questions such as: what information stood out; what information was most important to you;
what information was missing or unnecessary? Respondents were also probed on their
interpretation and reaction to the following statement in the letter: “This survey is part of the
Census of Agriculture Program which is required by and protected by law (Title 7, U.S. Code
and the Confidential Information Protection and Statistical Efficiency Act.”
Results
Postcard
All six respondents looked over the front and back of the postcard. When first handed the
postcard, some respondents only skimmed over the information, while others read more
thoroughly.
A couple of respondents commented that the postcard was attractive. One respondent
commented that the postcard was attractive and the word “organic” and the USDA logo grabbed
his attention. Another respondent thought the picture on the front should be changed to an image
of a tractor. He asked, “Do you people think all organic growers are hippies with a hoe?”
For the most part, all of the respondents felt the information provided on the postcard was easy to
understand; however, one respondent questioned why it said “Your 2014 Organic Survey,” since
it wasn’t his survey. Another respondent also commented that it was hard to make out the NASS
logo. He had to look at it very closely to make out what it was. This same respondent thought it
would be better to identify NASS using its acronym rather than writing out the agency name. He
was also expecting to see the USDA organic logo on the postcard.
Respondents felt that providing information about the survey and reasons why they should
participate was helpful but they didn’t agree with the information that is currently being
provided. In particular, a couple of respondents did not think the second reason given was
effective. Reason #2 states: “Your input will help re-evaluate Federal crop insurance programs to
ensure organic producers receive equitable insurance prices.” These respondents said a lot of
organic growers do not use crop insurance. Therefore, the price of crop insurance was not
important to them. On a related topic, in cognitive testing of Section 9 (Organic Acres under
Crop Insurance) in the Organic Survey, respondents often commented that they did not know
crop insurance was available for the types of crops they grew or that it was too expensive for
them to purchase. Crop insurance might not be a good motivator to convince operators to
participate in the survey. One respondent didn’t feel reason #3 was a reason to cooperate. Reason
#3 stated: “Your responses are completely confidential, as required by federal law.” This
respondent said it was a reason not to avoid cooperating, not a reason to respond.
Several respondents commented that more effective reasons needed to be given. Two
respondents provided suggestions. One respondent suggested stating that more federal dollars
3

would be provided to organic programs for things such as certification reimbursements and
research as a reason to participate in the survey. Another respondent suggested including results
from past surveys on the postcard.
When asked if they felt it was helpful to receive a postcard like this prior to the survey, three
respondents indicated that it was helpful and three respondents said it was not helpful; however
one of these respondents stated that this was because he reads all of his mail and responds to all
legitimate requests. The two other respondents, who did not think it was helpful to receive a
postcard, both would rather receive an email about the upcoming survey instead of a postcard.
Recommendation:
Remove Reason #2 and #3 from the list of the top 3 reasons to respond to the survey and add
findings from the previous Organic survey to the postcard that demonstrate why the survey data
is beneficial to a broad range of organic growers. Approved.
Cover Letter
Four respondents skimmed the contents of the cover letter, one respondent read it in full and one
respondent read “most” of the content. After respondents initially read the cover letter, we
probed them to see if they could recall important pieces of information included in the letter,
including who was conducting the survey, the purpose of the survey, how they could respond,
and the due date.
After looking at the letter once, all respondents could easily recall who was conducting the
survey. Three respondents recalled that NASS was conducting the survey; two respondents
recalled that the USDA was conducting the survey and another respondent recalled “USDA, Ag
Statistics.”
When asked if they recalled the purpose of the survey, four respondents indicated that the survey
was being conducted to gather information/statistics on organic farming. One respondent
indicated the survey was being conducted to evaluate the crop insurance policy and risk
management. Another respondent also commented that they survey data would support RMA.
One respondent could not recall reading information on the cover letter regarding the purpose of
the survey but was able to locate this information after looking at the cover letter a second time.
All respondents could recall reading that the survey could be completed online or by mail. One
respondent commented that the information about responding online should be made more
visible on the letter. Three respondents indicated that they preferred to respond online and three
respondents indicated that they would prefer to respond to the survey by mail. One of these
respondents indicated that he prefers to respond by mail because he can use a pencil and fix his
mistakes. Another respondent who prefers to respond by mail commented that he likes to flip
back and forth through the pages of the survey.
Three respondents recalled that the due date was March 1. Two respondents could not recall the
due date. Another respondent also could not recall the due date but stated, “They always call me
before they are due, so I don’t need to know when they are due.”
4

We also probed respondents on their interpretations and reactions to two important statements in
the cover letter. First, respondents were probed on their understanding of the following
statement, “This survey is part of the Census of Agriculture Program which is required and
protected by law (Title 7, U.S. Code and the Confidential Information Protection and Statistical
Efficiency Act). These federal laws require that NASS keeps your identity and your answers
confidential?” Three respondents interpreted this statement as saying that NASS was required to
keep their information confidential. Two respondents interpreted this statement as saying they
were required by law to complete the survey and both questioned whether anyone had ever been
fined for not completing the survey. One of these respondents then questioned whether this
statement meant that NASS was required by law to conduct the survey or if it meant that he as a
respondent was required by law to participate in the survey.
Respondents generally trusted that NASS would keep their responses confidential. However, one
respondent commented that although he was confident NASS would keep his data confidential,
he was still not happy that we publish the total numbers. This respondent felt that our survey data
only benefited big corporations. Another respondent commented that he was confident his data
would be kept secure but he thought other growers might have concerns.
Respondents were also probed on the effectiveness of the following statement encouraging them
to respond online, “Responding online is the easiest, fastest, and safest way to complete your
questionnaire and it saves taxpayer dollars too.” Two respondents indicated that they would
respond online regardless of this statement. After reading this statement, another respondent
stated, “it sounds like they really want me to do it online, so I guess I will give it a whirl.”
However, this respondent preferred to respond on paper. Two other respondents indicated that
this statement would have no effect on their decision to respond online. One respondent
commented, “This does not convince me to respond online… it makes me wonder when did the
government ever worry about saving the taxpayer money?” The other respondent, who was not
persuaded, stated that computers put people out of work and therefore he would prefer not to
respond online.
All respondents reported that the information presented in the letter was generally easy to
understand. When asked what information they felt most important in the cover letter
respondents reported the following: the due date, who is conducting the survey, the purpose of
the survey, why they should participate, and the web address for responding online. Three
respondents felt there was unnecessary information included in the letter. One respondent felt it
was not necessary to include the name of Title 7 (Title 7, U.S. Code and the Confidential
Information Protection and Statistical Efficiency Act). Two other respondents felt the second
paragraph, which states that the results of the survey will help RMA shape policy, was not
necessary. These same respondents indicated that most small organic farmers do not qualify for
crop insurance.
Respondents generally did not feel that anything was missing from the letter; however, one
respondent would like a NASS address where he could contact Joe Reilly. Another respondent
noted that the code/password for replying online was missing, although he presumed it would be
in the official letter.
5

After reading the letter, all six respondents reported they would be willing to participate in the
survey; however, two indicated this with some reluctance. One of these respondents indicated he
would participate so he wouldn’t get “pestered” about it. Another respondent indicated he would
only complete the survey because he is required to, but we would have to catch him at a time that
“works for him.”
Recommendations:
1) Revise the following statement for clarity: “This survey is part of the Census of
Agriculture Program which is required and protected by law (Title 7, U.S. Code and the
Confidential Information Protection and Statistical Efficiency Act). These federal laws
require that NASS keeps your identity and your answers confidential.” Consider splitting
into three sentences. Approved.
2) Remove the second paragraph and add a paragraph similar to the second paragraph in the
reminder letter, which more generally states how all organic growers can benefit from
participating in the survey. Approved.
3) To decrease the amount of text, remove the statement “Responding online is the easiest,
fastest, and safest way to complete your questionnaire and it saves taxpayer dollars too.” Approved.
Reminder Letter
Five respondents reported that they skimmed the contents of this letter and one respondent read it
in full. All of the respondents thought the information presented in the letter was generally easy
to understand. After reading the letter, respondents were probed to see what information in the
letter stood out to them initially. Respondents commented on different information presented in
the letter. One respondent noted that the letterhead and the USDA logo made the letter appear
legitimate. Two respondents noted that the letter was reminding them of the Organic survey and
that they had not completed it yet. Another respondent noted that the letter stresses the
importance of responding to the survey. Another respondent noticed the results from the 2012
Census were included in this letter, although he felt they could be more pronounced. One
respondent felt there was a “subliminal message” in the letter, which indicated that his
participation in the survey is mandatory. Finally, one respondent commented that it seemed like
the reminder letter was written by a different person than the cover letter because the messages
were different. The cover letter seemed to focus on how the survey benefited the RMA and the
reminder letter stressed Federal programs.
Respondents were probed to see what information in the letter was more important to them.
Three respondents felt the information that stated someone would follow up with them if they
did not complete the survey was most important to them. However, one respondent commented
that this statement was not definitive because of the word “may.” He asked if this meant
someone will follow up with him or someone may follow up with him. Two respondents felt the
second paragraph in the letter was most important. One of these respondents felt this information
should be in the first paragraph of the letter. He felt this was a better justification for
participating in the survey than the messages provided in the postcard and the cover letter.

6

Another respondent also commented that the information stating he could respond online was
also important to him.
We also probed respondents on their interpretations of and reactions to the following statement,
“This survey is part of the Census of Agriculture Program which is required by and protected by
law (Title 7, U.S. Code and the Confidential Information Protection and Statistical Efficiency
Act).” Respondents interpreted this statement in different ways. Two respondents interpreted
this statement as saying that NASS is required by law to keep their answers confidential. One
respondent interpreted this statement as saying he is required by law to complete the survey;
however he questioned whether anyone was ever fined for not completing the survey. Once
again, one respondent questioned whether it meant that NASS was required by law to conduct
the survey or if it meant that he as a respondent was required by law to participate in the survey.
Finally, one respondent interpreted this statement as saying he is required to do the survey and
that his responses will be kept confidential. In general, respondents felt this statement had no
influence on their decision to participate in the survey. One respondent commented that this was
just “high powered legal talk.” One respondent felt this statement was not necessary because he
knows he will get a phone call if he doesn’t send his data in. This doesn’t convince him at this
point to participate. Another respondent said it would have no impact on his decision because he
would respond to the survey anyway.
Respondents probed to see if there was any information in the reminder letter that they felt was
unnecessary. As with the cover letter, one respondent felt the name of Title 7 (Title 7, U.S. Code
and the Confidential Information Protection and Statistical Efficiency Act) was not necessary.
Finally, one respondent commented that there was a lot of text on the letter but he wasn’t sure
what could be removed.
Respondents were also probed to see if there was any information that they felt was missing from
the reminder letter. Once again, one respondent would like an address for NASS to be included
in the letter. Two respondents commented that there was no due date on this letter and one of
these respondents also pointed out that there were no repercussions stated for not completing the
survey. This respondent stated that he could look up Title 7 to see if there are any repercussions
for not completing the survey; however, he doesn’t think most people would do this. Finally, one
respondent felt more information could be included that explained how the results of this survey
would benefit the growers.
Respondents were then asked for their general comments on the letter. Two respondents felt it
was unnecessary to send a reminder letter. One respondent felt it was better to just call him if we
had not received his survey data. Another respondent felt it was better to send a reminder
postcard or an email. Finally, one respondent commented that he hopes he would only be sent
this letter if he hadn’t sent the survey in yet. He said he does not like receiving these letters when
he has already completed his survey.
Recommendations:
1) If possible, only send this letter if we have not received respondents’ data by the desired
due date. Approved.
7

2) Revise the following statement for clarity: “This survey is part of the Census of
Agriculture Program which is required and protected by law (Title 7, U.S. Code and the
Confidential Information Protection and Statistical Efficiency Act). These federal laws
require that NASS keeps your identity and your answers confidential.” Consider splitting
into three sentences. Approved.
3) Based on cover letter findings, remove the statement “Responding online is the easiest,
fastest, and safest way to complete your questionnaire and it saves taxpayer dollars too.” Approved.

8

Appendix A: Interview Protocol and Guide
2014 Organic Survey – Promotional Materials
Usability Testing Protocol
September 26, 2014
Please read this entire document before conducting any interviews to become familiar with the
data collection procedures and promotional materials.
Background
The 2014 Organic Survey is a data collection project conducted for the Risk Management
Agency (RMA); Census funds will also be used as directed by the 2014 Farm Appropriations
bill. The survey is an organic production and practices survey, parts of which have been
previously conducted in 2008 and 2011. The population of interest is any operation that produces
organic products according to USDA’s National Organic Program (NOP) standards and is
certified or exempt from certification. The population also contains operations that reported in
the 2012 Census of Agriculture that they were transitioning to organic. The questionnaire
collects information on organic production of field crops, vegetables, fruits, tree nuts, berries,
livestock and poultry, production practices, production expenses, marketing practices and valueadded production and processing.
Research Goals
The purpose of this research study is to assess respondents’ comprehension of and reactions to
the Organic Survey promotional materials. In this study, we will assess the postcard, cover letter
and reminder letter.
Sample
Since OMB clearance was not obtained for this testing, we will only be conducting a total of 9
interviews. Interviews will be conducted in the following states: CA, MD, OH, NH, and NJ.
The sample will include operations that grow organic crops and are certified or exempt from
certification.
Recruitment
Interviewers in each state are responsible for recruiting respondents. Interviewers may recruit
known organic operations in their area that are certified or exempt from certification. We also
encourage interviewers to search local CSAs for organic producers. See screener questionnaire at
the end of this document.

Respondents should be informed of the intent of the interviews during the recruitment process
and that their participation is voluntary. Respondents should also be told the interviews will be
conducted in person and are expected to last 1 hour.
Interview Procedures
Each interviewer is expected to conduct 1 interview. Interviewers should bring a copy of the
interview guide, postcard, cover letter, blank questionnaire, and reminder letter to the interview.
The following procedures should be followed during the interview.
Interviewers should explain the task to the respondent at the start of the interview. Tell the
respondent we are interested in their impressions of the letters we will send out for this survey.
We want to know how to make these letters more informative and useful to respondents (see
script at top of interview guide.)
Postcard
After explaining the task to the respondent, hand them the postcard and ask them to look it over.
Tell respondent: Pretend you just received this postcard in the mail. I want you to look it over
however you normally would. If the respondent doesn’t look at the postcard at all and says
something like “I’d throw it away” make a note of this and then ask them to look over the
postcard.
Note any comments they make while looking it over (positive or negative). After they have
finished looking at it ask the following questions:
If you received this postcard in the mail what would be your initial reaction?
Is there any information that stands out to you?
Is there any information on this postcard that you find confusing or have a question about?
If there any information you think is missing?
Do you feel it is helpful to receive a postcard like this?
Organic Cover Letter
Next, hand the respondent a copy of the cover letter and the survey instrument. We will not be
testing the survey instrument. The reason we are giving the survey instrument along with the
cover letter is we want to mimic the procedures in the field (respondents receive these two items
together). The instrument will provide additional context to help respondents understand the
information presented in the letter.
Have respondents look over the cover letter. Tell respondent: Pretend you just received this letter
and survey in the mail. I want you to look the letter over however you normally would. If the
respondent doesn’t look at the letter at all and says something like “I’d throw it away” make a
note of this and then ask them to look over the letter.

Note any comments they make while reading the letter. When they are finished take the cover
letter and survey instrument away and ask the following questions:
(Recall/Comprehension)
How much of this letter did you read? Did you read this letter in full, skim, didn’t read anything?
Do you remember who is conducting the survey? Who?
In your own works, what is the purpose of the survey?
How can you respond to this survey?
When are responses to the survey due?
Would you participate in this survey if you were selected to do so? Why or why not?
If so, How would you respond to survey (i.e., mail or web)? Why? If not, how would you
normally prefer to participate in surveys?
After the respondents have answered the recall questions, hand the cover letter back to them. If
respondents could not recall the information asked about above, ask if they can locate the
information after reviewing the information a second time. Note their ability/inability to do so.
Then ask the following questions:
(Reactions)
Is there anything on this letter than you don’t understand or are confused about?
When this statement reads, “This survey is part of the Census of Agriculture Program which is
required and protected by law (Title 7, U.S. Code and the Confidential Information Protection
and Statistical Efficiency Act). These federal laws require that NASS keeps your identity and
your answers confidential?” What does this mean to you??
How confident are you that your data will be kept confidential? Can you tell me more about
that? (note positive and negative reactions)
What is your initial reaction to the following “Responding online is the easiest, fastest, and
safest way to complete your questionnaire and it saves taxpayer dollars too”? Does this
persuade you in any way to respond to the survey online? How so? (note positive and negative
reactions)
What information on this letter do you feel is the most important to you?
Is there any information that you feel is unnecessary?
Is there any information that you feel is missing?
Organic Reminder Letter
Finally, hand the respondent a copy of the reminder letter and the survey instrument.

Have respondents look over the reminder letter. Tell respondent: Pretend you just received this
letter and survey in the mail. I want you to look the letter over however you normally would. If
the respondent doesn’t look at the letter at all and says something like “I’d throw it away” make
a note of this and then ask them to look over the letter.
Note any comments they make while reading the letter. When they are finished take the reminder
letter away and ask the following questions:
(Reactions)
How much of this letter did you read? Did you read this letter in full, skim, didn’t read anything?
What information stood out to you when reading this letter?
What information on this letter do you feel is the most important to you?
Please look at the following statement: This survey is part of the Census of Agriculture Program
which is required and protected by law (Title 7, U.S. Code and the Confidential Information
Protection and Statistical Efficiency Act).Can you tell me in your own words what this statement
is saying? Does this persuade you in any way to respond to the survey? Why or why not?
If the respondent does not discuss the part of the statement that says the survey is required by
law, ask them about it. What is your initial reaction to this? Does this influence your decision to
participate?
Is there any information that you feel is unnecessary?
Is there any information that you feel is missing?
Is there any information in this letter that you find confusing or have a question about?
If respondent indicated they would not complete this survey during testing of the cover letter,
ask: Would this letter persuade you to complete the survey? Why or why not?
Do you have any other general comments about these materials?
End the interview and thank the respondent for their time.
Interview Notes
Interviewers are expected to spend up to one hour per interview writing and summarizing their
notes from that interview. Please provide detailed notes form the interviews. Notes should be
emailed to Heather Ridolfo ([email protected]) no later than October 3, 2014.

Screener Questionnaire
Hi, my name is and I am calling from the United States Department of Agriculture,
National Agricultural Statistics Service. We are contacting operations to ask for help in testing
the 2014 Organic Survey. This survey will provide important information on organic production
across the United States. Is this something that you’d be willing to speak to us about?
In order to improve the information we distribute on this survey, we would like to get your
feedback on the promotional materials we will be sending out. This would take about an hour of
your time and I could schedule a time that is convenient for you. During this time I will ask you
to look over our promotional materials and provide your feedback on how we can improve them.
Could I schedule a time to meet with you and get your input on this?
One of the criteria for this testing is you must have organic crops according to USDA’s National
Organic Program standards and are either certified or exempt from certification (under $5,000 in
annual sales).
Does your operation have any organic production according to NOP standards and is certified or
exempt from certification?
YES – [Continue]
NO - [Thank respondent for their time and hang up]
This means you are eligible to help us out. I have the following days and timeframes open.

Which of these days and times works best for you?
I will call you to confirm the appointment a day or two before. If you need to change the
appointment time, please call me back at .

2014 Organic Survey – Promotional Materials
Usability Testing Interview Guide
September 26, 2014
Interviewer:

Respondent:

Date:

Before we begin, I want to tell you a little more about the project and what we will be doing today. I
have here the promotional materials that will be used for the 2014 Organic Survey. We want to make
sure that these materials are informative and that everyone understands the information presented in
these materials. I’m going to have you look them over and then I will ask you some follow up questions.
I’m interested in your general reactions to these materials – good and bad. Again, everything we discuss
today will be kept completely confidential. Do you have any questions before we get started?

Postcard
[Hand the respondent the postcard and tell them to take a look at it. Tell respondent: Pretend you just
received this postcard in the mail. I want you to look it over however you normally would. If the
respondent doesn’t look at the postcard at all and says something like “I’d throw it away” make a
note of this and then ask them to look over the postcard.
Note any comments they make while looking it over. After they have finished looking at it ask the
following questions.]
Comments:

If you received this postcard in the mail what would be your initial reaction?

Is there any information that stands out to you?

Is there any information on this postcard that you find confusing or have a question about?

If there any information you think is missing?

Do you feel it is helpful to receive a postcard like this?

Organic Cover Letter
[Hand the respondent a copy of the cover letter and the survey instrument. Have respondents read
the cover letter. Tell respondent: Now I want you to pretend you received this letter and survey in the
mail. I want you to look it over however you normally would. If the respondent doesn’t look at the
letter at all and says something like “I’d throw it away” make a note of this and then ask them to look
over the letter.
Note any comments they make while reading the letter. When they are finished take the cover letter
away and ask the following questions.]
Comments:

How much of this letter did you read? Did you read this letter in full, skim, didn’t read anything?

Do you remember who is conducting the survey? Who?

In your own works, what is the purpose of the survey?

How can you respond to this survey?

When are responses to the survey due?

Would you participate in this survey if you were selected to do so? Why or why not?

If so, How would you respond to survey (i.e., mail or web)? Why? If not, how would you normally

prefer to participate in surveys?

[Hand the cover letter back to respondents. If respondents could not recall the information asked
about above, ask if they can locate the information after reviewing the information a second time.
Note their ability/inability to do so. Then ask the following questions.]
Notes:

Is there anything on this letter than you don’t understand or are confused about?

When this statement reads, “This survey is part of the Census of Agriculture Program which is
required and protected by law (Title 7, U.S. Code and the Confidential Information Protection and
Statistical Efficiency Act). These federal laws require that NASS keeps your identity and your answers
confidential?” What does this mean to you??

How confident are you that your data will be kept confidential? Can you tell me more about that? (note
positive and negative reactions)

What is your initial reaction to the following “Responding online is the easiest, fastest, and safest way to
complete your questionnaire and it saves taxpayer dollars too”? Does this persuade you in any way to
respond to the survey online? How so? (note positive and negative reactions)

What information on this letter do you feel is the most important to you?

Is there any information that you feel is unnecessary?

Is there any information that you feel is missing?

Organic Reminder Letter
[Hand the respondent a copy of the reminder letter and survey instrument. Have respondent look
over the reminder letter. Tell respondent: Pretend you just received this letter and survey in the mail. I
want you to look the letter over however you normally would. If the respondent doesn’t look at the
letter at all and says something like “I’d throw it away” make a note of this and then ask them to look
over the letter.
Note any comments they make while reading the letter. When they are finished take the reminder
letter and instrument away and ask the following questions.]

Comments:

How much of this letter did you read? Did you read this letter in full, skim, didn’t read anything?

What information stood out to you when reading this letter?

What information on this letter do you feel is the most important to you?

Please look at the following statement: This survey is part of the Census of Agriculture Program
which is required and protected by law (Title 7, U.S. Code and the Confidential Information
Protection and Statistical Efficiency Act).Can you tell me in your own words what this statement
is saying? Does this persuade you in any way to respond to the survey? Why or why not?

If the respondent does not discuss the part of the statement that says the survey is required by
law, ask them about it. What is your initial reaction to this? Does this influence your decision to
participate?

Is there any information that you feel is unnecessary?

Is there any information that you feel is missing?

Is there any information in this letter that you find confusing or have a question about?

If respondent indicated they would not complete this survey during testing of the cover letter, ask:
Would this letter persuade you to complete the survey? Why or why not?

Do you have any other general comments about these materials?

Appendix B – Promotional Materials Used During Testing
(Postcard – Front)

(Postcard – Back)

United States Department of Agriculture
National Agricultural Statistics Service
In Cooperation with Risk Management Agency

January 02, 2015

Dear Agricultural Producer,
The U.S. Department of Agriculture’s (USDA) National Agricultural Statistics Service (NASS) is
conducting the 2014 Organic Survey. This survey gathers detailed production, sales and marketing
information on organic farming in the United States. Please take the time to fill out your survey form.
The 2014 Organic Survey is important because the results will help USDA’s Risk Management
Agency shape policy decisions, including re-evaluating Federal crop insurance programs to ensure
organic producers receive equitable insurance prices.
Your response is critical to help protect your investments and support your future success. Please
respond promptly in either of the following ways:
•
•

Online at www.agcounts.usda.gov. Responding online is the easiest, fastest and safest way to
complete your questionnaire and it saves taxpayer dollars too. All you need is the identification
number on the enclosed form to begin.
By mail. Complete the enclosed form and mail it back in the return envelope provided.

This survey is part of the Census of Agriculture Program which is required and protected by law
(Title 7, U.S. Code and the Confidential Information Protection and Statistical Efficiency Act). These
federal laws require that NASS keeps your identity and your answers confidential.
If you have questions, please call (888) 424-7828. Thank you in advance for your participation.
Sincerely,

Joseph Reilly
Administrator, National Agricultural Statistics Service
United States Department of Agriculture
Enclosures

USDA is an equal opportunity provider and employer.

United States Department of Agriculture
National Agricultural Statistics Service
In Cooperation with Risk Management Agency

February 17, 2015

Dear Agricultural Producer,
According to our records, the U.S. Department of Agriculture’s National Agricultural Statistics
Service (NASS) has not received your completed 2014 Organic Survey. Please complete and return
your survey today. If you have recently responded, thank you.
Responses to the most recent Census of Agriculture let us know that organic agriculture is growing in
the United States, with sales increasing 82 percent between 2007 and 2012. Your participation in this
survey is important because every response is a step closer to Federal programs that better meet the
needs of organic producers like you.
If you have not yet responded, you can do so in either of the following ways:
•
•

Online at www.agcounts.usda.gov. Responding online is the easiest, fastest and safest way to
complete your questionnaire and it saves taxpayer dollars too. All you need is the identification
number on the form to begin.
By mail. Complete the form and mail it back in the return envelope provided.

If a response is not received a NASS representative may personally follow up with you to gather this
information. This survey is part of the Census of Agriculture Program which is required and
protected by law (Title 7, U.S. Code and the Confidential Information Protection and Statistical
Efficiency Act). These federal laws require that NASS keeps your identity and your answers
confidential.
To learn more about this survey visit www.nass.usda.gov. If you have questions or need assistance,
call (888) 424-7828.
Sincerely,

Joseph T. Reilly
Administrator, National Agricultural Statistics Service
United States Department of Agriculture
Enclosures

USDA is an equal opportunity provider and employer.