2900-0701_Part_B. Post-Eng. (MO 9.8.15)

Post-Engagement Survey

VA Form 10-21081(NR)

2900-xxxx

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each strata. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

The Post-Engagement Survey will collect information from attendees from the National Veterans Small Business Engagement (NVSBE) during each year of this project. The table below shows data for 2014. During that year, the NVSBE had an attendance of 3,582, while 975 participants completed the Post-Engagement questionnaire for a response rate of 27%. The Department of Veterans Affairs (VA), Office of Small and Disadvantaged Business Utilization (OSDBU) estimate an annual response rate increase of 2% for the 2015 NVSBE and subsequent years. It is also expected a slight increase in the number attendees to the NVSBE each year.

Historical Universe of potential respondents, response rates, and completed surveys (Year 2014)

	Universe	QTRs	Annual total	Response Rate	Completed surveys
2014	3,582	n/a	3,582	27%	975†

Projected Universe of potential respondents, response rates, and completed surveys (Years 2015, 2016, 2017)

	Universe	QTRs	Annual total	Response Rate	Completed surveys
2015	4,000	n/a	4,000	29%	1,160∆
2016	4,350	n/a	4,350	31%	1.349∆
2017	4,700	n/a	4,700	33%	1,551∆

†actual number of completed surveys

∆ projected number of completed surveys

2. Describe the procedures for the collection of information, including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed

• Unusual problems requiring specialized sampling procedures

• Any use of less frequent than annual data collection to reduce burden

The Post-Engagement data is collected once a year, the day the NVSBE ends. In order to be consistent with the 2014 methodology approach, VA OSDBU will use a whole-population survey of the business attendees rather than a sampling procedure. The survey will be deployed to all NVSBE attendees, being November 19, 2015 the date where the survey will be deployed this year. The use of a whole-population survey is particularly important to capture the unique aspects of responses from set-aside groups within the target population, such as small businesses, large corporations, VA procurement decision makers (PDM), and corporate PDM.

Frequency and cross tabulations are the analysis that will be conducted. Some of these analyses will be posted at the 2015 NVSBE website (http://nvsbe.com/) in October 2015, when registration opens for this year’s event. The following are examples of the information to be analyzed: the percentage of attendees stating to have made new and valuable connections, type of connections with greatest ROI potential, value of procurement opportunities learn at the event, and overall ROI.

The responses of this web-based survey are collected over secured, encrypted SSL/TLS connections. Secure Sockets Layer (SSL) and Transport Layer Security (TLS) technology protect communications using both server authentication and data encryption. This ensures that user data in transit is safe, secure, and available only to intended recipients. All responses are stored in secure data centers in accordance with industry best practices. The data repository is protected from unauthorized access.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

The following actions will be taken in order to maximize response rate and to ensure accuracy and reliability of the Post-Engagement survey data:

• Reminders and follow-ups are employed to minimize non-response bias.

• Data and results will be fully documented along with the methodology employed.

• Consistent systems of data collection will be employed throughout the data collection process to ensure year-to-year monitoring is comparable.

• Quality control checks and internal peer review are performed on data and analysis products to ensure they are clear, concise, and accurate representations of the survey results.

• Monitor for and account for blank or "N/A only" non-responses in the data set to ensure effectively empty responses are not recorded as a response.

• Use tokenized anonymous links that ensure only a single response is submitted by each respondent.

• Employ technical measures to ensure the survey is completed correctly, such as to prevent multiple answers to multiple choice questions.

• Ensure questions have a null response, such as "N/A", where a respondent might not have a meaningful response.

The questionnaire employed for the Post-Engagement survey was tested after the 2014 NVSBE and the responses analyzed to understand the data collected and verify alignment with VA OSDBU research objectives.

The intended use of the data is to improve the short-term ROI and satisfaction level for all NVSBE attendees and identify areas for improvement and areas of success. Non-responses will be compared to the demographics of the target population to determine if any known demographics factors may be correlated with non-responses. The demographics, registration data, and other administrative data available to VA OSDBU from all NVSBE attendees will allow careful comparison of the response demographics to the demographics of all attendees to identify any bias.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

The Post-Engagement survey and its administration procedures have been studied over the last year. OSDBU does not anticipate major changes in either the administration procedure or the survey itself. One reason to minimize any additional changes is to allow for comparison of results over time. Refining the survey to minimize burden can only be achieved by reducing the number of questions asked. Due to the survey’s use as a performance measure, any deletions of the questions will result in data that are not comparable year to year.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Matthew Roy
Statistician

Turn Around Factor

7847 Riverside Dr.

Richmond, Virginia 23225

Tel: 804-554-2149

Email: [email protected]