NASS Review of PACA Customer Survey

NASS Review

August 27, 2008

This document summarizes my review of the OMB clearance request for the AMS customer satisfaction survey relating to the Perishable Agricultural Commodities Act (PACA).

This survey is designed to provide a low-cost (estimated annual cost of @ $1,000) subjective measure of customer satisfaction for services provided by the PACA Customer Service Line. It appears that AMS has adequately planned what it wants to evaluate, and intends to use the results to improve its service. The data will be for internal use only by AMS and USDA’s Fruit and Vegetable Industry Advisory Committee. I believe the potential benefits are worth the token monetary investment.

I have just a few comments. AMS states it expects a response rate of about 80 percent, but doesn’t explain how it arrived at that estimate. It also states that data collection could not be conducted less frequently than once a month, but doesn’t explain why. AMS doesn’t explain how it arrived at a desired usable sample size of 20 responses per month. However, estimating percents doesn’t usually require a large sample. AMS leaves the door open to increases in sample size based on results from the initial few surveys. For example, greater than expected variation in responses could call for a larger sample.

This telephone survey will ask customers to rate services they received using a five-point scale ranging from very satisfied to very dissatisfied. There will also be some “yes-no” questions. “No opinion” will be an option for some questions. The question wording is good. I would suggest re-ordering the questions as follows:

1. Was this the first time you have contacted the PACA?

2. Did you find the PACA Customer Service Line recording easy to use and follow?

3. How satisfied were you with the staff’s knowledge of the subject?

4. How satisfied were you with the staff’s professionalism and courtesy?

5. How would you rate the staff’s helpfulness?

6. Did the staff’s information help you to resolve your dispute?

7. What was your overall impression of the PACA Customer Service Line?

8. How comfortable would you be calling the PACA Customer Service Line again?

9. Do you have any further comments or suggestions concerning the PACA Customer Service Line or other aspects of PACA customer service?

AMS doesn’t discuss how it will summarize the responses. The correct summarization method is to tabulate counts of responses in each category (e.g., number of customers who say they are very satisfied with the service, number who say they are somewhat satisfied, etc.). AMS should not assign a numerical rating to each level of customer satisfaction and then calculate a mean satisfaction level.

AMS should have a paper version of the questionnaire to provide to OMB. It should also store the questionnaire responses and summarization in electronic form for archival.

If AMS plans to use this as a client evaluation tool, it should be sure the questions the survey asks are consistent with the rating criteria.