2900-0751_VA Supplier Survey Justification Part B r4-28

OMB 2900-0751

Part B

Collection of Information Employing Statistical Methods for

VA Supplier Perception Survey (SPS)

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

The Department of Veterans Affairs (VA) is primarily acquiring data on the perception and experience of its suppliers. VA Office of Acquisition and Logistics (OAL) has enlisted the help of Barbaricum to conduct a supplier perception survey. VA suppliers (for the purpose of this study) are identified in two categories. The respondents will have an impact on VA’s improvement initiative in regards to customer satisfaction. Suppliers’ feedback will be anonymous and survey responses will be held in strictest confidence. Only personnel at Barbaricum will see the replies. Barbaricum will report summary results, analysis and findings to VA.

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each strata. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

For the last iteration of the Supplier Perception Survey in January 2015, 29,185 unique emails were provided to the contractor conducting the survey. A total of 1,150 completed surveys were obtained, for a response rate of 3.95%

2. Describe the procedures for the collection of information, including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed

• Unusual problems requiring specialized sampling procedures

• Any use of less frequent than annual data collection to reduce burden

For each of the two categories, VA OAL will provide Barbaricum each company’s name, point of contact, and email address. Barbaricum will send out the survey electronically to each supplier. The survey will have several pull down menus to allow the respondent to denote different information such as demographics, socio-economic type, and geography. VA will provide pre-notice to C1 suppliers (from a va.gov address) the day before Barbaricum provides the actual survey. C-2 will not get VA pre-notification.

Statistical methodology for stratification and sample selection

In C-1, the population of interest is the 1,150 vendors who represent the greatest VA annual spend. This entire group will be included in the sample. In C-2, the sample of 29,185 suppliers will be selected randomly from the remainder of the supplier base. We anticipate that the size of this sample will be sufficient to achieve adequate representation of all of the subgroups of suppliers that are of interest in the analysis.

Estimation procedure

Overall supplier ratings of VA on individual items in the survey will be estimated using simple confidence intervals. Differences in ratings across sub-groups of suppliers (e.g., suppliers of different categories of goods and services) will be estimated using Analysis of Variance.

Degree of accuracy needed

All items in the body of the survey ask for respondent ratings on a 5-point Likert scale. With this type of scale, we believe an accuracy on individual item scores of + 0.15 (α = 0.05) is, from a practical perspective, adequate. In past administrations of the survey (in other organizations), item variances typically range between 0.70 and 1.00. With this level of variation, a sample size of n > 175 will provide needed accuracy.

Unusual problems requiring specialized sampling procedures

No such issues are anticipated.

Any use of less frequent than annual data collection to reduce burden

Annual or biannual data collection is needed to track the effectiveness of VA’s transformation efforts. Burden on respondents should be very modest. Based on Barbaricum prior use of the survey with other clients, the typical time required for a supplier to respond to the survey is less than 30 minutes.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

In order to increase sample size, an email is sent from VA to all potential respondents, alerting them to the upcoming survey invitation and ensuring respondents that the survey is a legitimate request. All data will be collected electronically and, based on MSUs experience with other clients, total time for a supplier to respond to the survey should average less than 30 minutes.

An initial e-mail invitation to the suppliers will be followed up at ten-day intervals with two “reminder” messages to non-respondents. We do not anticipate a high non-response rate beyond the non-response rate that we identified and estimated in item # 1. However, if such need arises, Barbaricum can allow suppliers to go several days beyond the deadline for response to ensure we get our targeted minimum response rate. Post hoc testing will be conducted to evaluate the representative nature of the respondent group and to identify any limitations on the generalizability of the results.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

N/A

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The contractor that will collect and analyze the information for VA is Barbaricum. The point of contact is:

Kyle Andrews

[email protected]

202-393-0873

Barbaricum

819 7^th Street NW, Suite 500

Washington, Dc 20001

Page 3