VA SPS Part B

Supporting Statement for Paperwork Reduction Act Submission

Part B

Collection of Information Employing Statistical Methods for

VA Supplier Perception Survey (SPS)

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

The Department of Veterans Affairs (VA) is in the beginning stage of undergoing its transformation process. In this stage of transformation, VA is primarily acquiring data on the perception and experience of its suppliers. VA Office of Acquisition and Logistics (OAL) has enlisted the help of Michigan State University (MSU) to conduct a supplier perception survey. VA suppliers (for the purpose of this study) are identified in two categories. The respondents will have an impact on VA’s improvement initiative in regards to customer satisfaction. Suppliers’ feedback will be anonymous and survey responses will be held in strictest confidence. Only personnel at MSU will see the replies. MSU Professors Ragatz and Sandor will report summary results, analysis and findings to VA.

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each strata. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

Data will be collected from a list of two categories of suppliers. The first category hereby referred to as Category 1 (C1) is the top 400 vendors with the greatest VA spend: the second category hereby referred to as Category (C2) is roughly 15,000 VA suppliers. All suppliers listed in these two categories are done so without regard to business size or socio-economic status. This type of supplier survey has not been conducted before by VA OAL. We estimate the response rate for the C-1 to be 70% or greater and the response rate for C-2 to be 30% or greater.

2. Describe the procedures for the collection of information, including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed

• Unusual problems requiring specialized sampling procedures

• Any use of less frequent than annual data collection to reduce burden

For each of the two categories, VA OAL will provide MSU each company’s name, point of contact, and email address. MSU will send out the survey electronically to each supplier. The survey will have several pull down menus to allow the respondent to denote different information such as demographics, socio-economic type, and geography. VA will provide pre-notice to C1 suppliers (from a va.gov address) the day before MSU provides the actual survey. C-2 will not get VA pre-notification.

Statistical methodology for stratification and sample selection

In C-1, the population of interest is the 400 vendors who represent the greatest VA annual spend. This entire group will be included in the sample. In C-2, the sample of 15,000 suppliers will be selected randomly from the remainder of the supplier base. We anticipate that the size of this sample will be sufficient to achieve adequate representation of all of the subgroups of suppliers that are of interest in the analysis.

Estimation procedure

Overall supplier ratings of VA on individual items in the survey will be estimated using simple confidence intervals. Differences in ratings across sub-groups of suppliers (e.g., suppliers of different categories of goods and services) will be estimated using Analysis of Variance.

Degree of accuracy needed

All items in the body of the survey ask for respondent ratings on a 5-point Likert scale. With this type of scale, we believe an accuracy on individual item scores of + 0.15 (α = 0.05) is, from a practical perspective, adequate. In past administrations of the survey (in other organizations), item variances typically range between 0.70 and 1.00. With this level of variation, a sample size of n > 175 will provide needed accuracy. This sample size will require a response rate of 43.8% from the C1 suppliers. Based on previous administrations of the survey (again, in other organizations) we anticipate a substantially higher response rate.

Unusual problems requiring specialized sampling procedures

No such issues are anticipated.

Any use of less frequent than annual data collection to reduce burden

Annual or biannual data collection is needed to track the effectiveness of VA’s transformation efforts. Burden on respondents should be very modest. Based on MSUs prior use of the survey with other clients, the typical time required for a supplier to respond to the survey is less than 30 minutes.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

All data will be collected electronically and, based on MSUs experience with other clients, total time for a supplier to respond to the survey should average less than 30 minutes.

An initial e-mail invitation to the suppliers will be followed up at ten-day intervals with two “reminder” messages to non-respondents. We do not anticipate a high non-response rate beyond the non-response rate that we identified and estimated in item # 1. However, if such need arises, MSU can allow suppliers to go several days beyond the deadline for response to ensure we get our targeted minimum response rate. Post hoc testing will be conducted to evaluate the representative nature of the respondent group and to identify any limitations on the generalizability of the results.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

Data from MSUs previous administration of the survey with other clients have shown good internal consistency – Chronbach alpha values of 0.74 to 0.92 for the six subgroups of items on the survey.

Further analysis will be done with the VA data, with an eye toward further simplification of the survey instrument for future data collections.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Data analysis will be performed primarily by MSU professor Joe Sandor, Hoagland-Metzler Professor of Practice in Supply Management, The Eli Broad School of Business; N464 North Business Complex, Michigan State University, East Lansing, MI 48824-1122; Phone (517) 432-6386.

Page 3