SS Part B 2009

October 2009

The Supporting Statement for OMB

Customer/Stakeholder Satisfaction Survey – Part B

0579-XXXX

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Introduction: The information contained in this document is to describe how a collection of data, for the use of qualitative purposes, will be analyzed. The feedback collected will be used to improve the Quality Management System for the Registration, Identification, Permits, and Plant Safeguarding Unit and the Regulatory, Permits and Manuals Unit. The collection of this data will assist in the continual improvement efforts to satisfy our customers’ requests and wishes.

Respondent Universe

United States Department of Agriculture (USDA), Animal and Plant Health Inspection Service (APHIS), Plant Protection and Quarantine (PPQ), Plant Health Programs (PHP), Permits Registrations, Imports and Manuals (PRIM), Permit Services (PS) and Pest Permit Evaluations (PPE) are responsible for the issuance of more than 12,000 permits for plants, plant products, and plant pests annually. These permits are required for the importation, transit, and domestic movement of plant pests and the importation and transit of plants and plant products under regulatory authority. The issuance of permits affects numerous stakeholders including a diverse set of permit applicants from importers of plants and plant products to university scientists conducting research on plant pathogenic bacteria, and cooperating Federal and State agencies.

United States Department of Agriculture (USDA), Animal and Plant Health Inspection Service (APHIS), Plant Protection and Quarantine (PPQ), Plant Health Programs (PHP) is responsible for making quarantine action status recommendations. An urgent pest interception consists of the interception of a potential plant pest on a perishable shipment being held at a U.S. port of entry. These determinations directly affect PPQ Identifiers who are responsible for identifying more than 200,000 plant pests each year, which assists to make the aforementioned recommendation. Other individuals that may be affected by recommendations include PPQ Safeguarding Specialists, Customs and Border Protection (CBP) Officers, Managers in PPQ and CBP, PPQ/SITC Officers, importers, and shippers.

Obtaining an estimate of stakeholder satisfaction with the services of PS, PPE is the primary goal for this information collection.

Sampling

There will not be a sampling system in place. This information will be made available to everyone who requests services from PS or PPE. The information received is on a voluntary basis through the use of an automated phone system, and self-selected during conferences. APHIS thinks a small percentage of people will actually respond, but we are unable to predict this type of information. Eventually, APHIS may have an online questionnaire, where individuals who contact PS and PPE would be given the option to take the survey. This is an unbiased method, because there is no systematic number of daily calls for PS and PPE; thus, the selection of respondents reflects impartiality. Emphasis on the recommendation alone would not provide an accurate view of the individual respondents. With relation to the permittees, which represents easily definable discrete units, the permits themselves are not used solely for measurement because a single current or potential permit applicant can have more than one permit.

Here is how the information collected will be used:

Table 1

Number of people surveyed
Frequency of survey
Cumulative frequency over each quarter
Cumulative frequency over one year

The analysis will be conducted after each quarter to show the results of the number of people surveyed.

How will this be interpreted and used to improve the permit process. Table 2 shows the value of the information collected.

Table 2

How are we doing?	Number of permit problems	Duration of permit in process	How can we improve?
% of problems which have been resolved vs. % of problems still unresolved	% of permit problems dealing with ePermits vs. # of permits not resolved under specific timeframes suggested in the process	# of permits issued for every process vs. # of permits sitting stagnate and length of time taken to process those permits	Overall, how many issues were resolved, vs. length of time permit issues took to resolve

Expected Response Rates

The stakeholder/customer satisfaction survey instrument will be used as an opportunity for the customer to provide feedback to the PS and PPE units. The estimated response rate may vary considerably because this is a self-selected sample. The estimated response rate may vary from 1-5 percent depending on the survey methodology and willingness of the participants.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical Methodology

We will not use specific methodology for stratification and sample selection. All individuals inquiring about quarantine action status recommendations or PS and PPE permits are considered eligible members of the sample population.

Estimation Test Procedure

The following table describes the information needs and statistical procedures used to estimate them:

Customer Feedback	Simple descriptive statistics, to interpret raw data. Using SPC (Statistical Process Control)	Excel Spreadsheet; measuring the quality and improvement of the current processes
a.) Quarantine Action Status Recommendation Impact b.) Permit Application and Issuance Impact	Input/Output Analysis	Graphs and Charts

Test procedures used will be considered Cost of Quality/ Cost Benefit or the cost of doing business as it pertains to the collection of survey information, underlying the positives or negatives in the permitting process. Cost implications would be considered when recurring issues continued.

(i.e., Quality Metrics Collection: focusing on defect rates (permit problems, length of permit stagnation in the system) or service problem reports (service to our customers).

The fundamental approach has the advantages of straightforward measurement and ease of understanding. It allows comparison of dissimilar products.)

In order to analyze the effectiveness of our cost of quality, this has to be evaluated over a long-range timeframe. Our first two years will show quality cost history without any knowledge of, or emphasis on, its reduction. That is reduction of customer service complaints, permit issued complaints, etc. The third year is the start of quality cost measurements and use. Years four through nine show actual progress accomplished. Year 10 is a projection of the expected continued progress.

Degree of Accuracy

We have not established degrees of error or confidence levels for the survey since the data is nominal and the analysis is descriptive in nature.

Unusual Problems

No unusual problems are anticipated due to the use of multiple survey methodologies (i.e., interview, phone, online), which will utilize minimal resources.

Periodic Data Collection

This survey will initially be offered to everyone but can be changed to scale back the sampling intervals as deemed appropriate.

3. Describe methods to maximize response rates and to deal with issues on non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

In the past, there have not been any observed problems with the physical surveys. We anticipate few problems with non-English speakers because quarantine action status recommendations from the plant pest, plants, and plant products permits are for domestic use only by American citizens who are required to speak functional English to utilize our services. For this reason, it is expected that all of the respondents will have functional English skills; therefore, language should not be an issue. Additionally, all survey methodologies include a brief set of questions with an optional comment section. The shortness of the surveys should assist in the maintenance of respondent interest and an increase in survey responses.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

No method testing is required for this information collection approval period because:

1. There are no methods to test from previous information collection approval periods.

2. The questions to be used in the information collection approval periods have been approved and reviewed previously for use in survey methodologies by internal top management.

3. The effectiveness of the collection methods has been pre-determined.

4. There are no concerns with response bias due to past review of qualitative data from stakeholder/customer comments.

5. There are no concerns with response bias due to the variety in the recipients of quarantine action status recommendations for permits from PS and PPE.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Principal design and analysis consultants include:

• Terry Di Leone: (301) 734-0655; USDA-APHIS-PPQ, Plant Health Programs, Registration, Investigation, Permits and Plant Safeguarding

• Cheron Dalrymple: (301) 734-5215; USDA-APHIS-PPQ, Plant Health Programs, Regulations, Permits and Manuals

The principal individual in charge of sample design, data collection, analysis, and reporting:

Georgia Cornelison: (301) 734-0841; USDA-APHIS-PPQ, Plant Health Programs,

Regulations, Permits and Manuals

Statistical Consultants: (possibilities)

Robert M McDowell, USDA Riverdale, MD

Ned Jones, USDA Raleigh, NC

5