VBOC Supporting Statement Part A_Final 5-11-15

Appendix A-1 Deliverables Schedule
Appendix B-1 Director Survey Instrument
Appendix B-2 Director Survey Construct Justification
Appendix B-3 Director Survey Invite E-mail
Appendix B-4 Director Survey Reminder E-mail
Appendix B-5 Director Survey Thank you E-mail
Appendix C-1 Client Outcome Survey Instrument
Appendix C-2 Client Outcome Survey Construct Justification
Appendix C-3 Client Outcome Survey (Cohort 1) announcement to directors
Appendix C-4 Client Outcome Survey (Cohort 2) announcement to directors
Appendix C-5 Client Outcome Survey Invite E-mail
Appendix C-6 Client Outcome Survey Reminder E-mail
Appendix C-7 Client Outcome Survey Thank You E-mail
Appendix D-1 Site Visit Invite E-mail, Follow-up E-mail, and Consent Form, Director Interview Consent Form, Protocol, and Construct Justification
Appendix E-1 Staff Interview Consent Form, Protocol, and Construct Justification
Appendix F-1 E-mail Scripts for Arranging Client Interviews, Phone Scripts for Arranging Client Interviews, Client Interview Consent Form, Protocol, and Construct Justification
Appendix G-1 Data Management Security Protocols

PART A. JUSTIFICATION

Circumstances Making the Collection of Information Necessary

Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The Small Business Administration (SBA) Office of Veterans Business Development’s (OVBD’s) Veterans Business Outreach Program (VBOP) provides entrepreneurial development services, which includes business training, counseling, mentoring, and referrals, for eligible veterans owning or considering starting a small business. Through the VBOP, up to 15 organizations will participate in a cooperative agreement with SBA in the new 2015 granting period. These organizations will operate what OVBD refers to as Veterans Business Outreach Centers (VBOCs)¹. VBOCs provide outreach, assessment, business training, short-term and long-term counseling, directed referrals, and other technical assistance services to veterans, service-disabled veterans, and Reserve Component business owners and entrepreneurs, including discharged service members eligible for participation in the Department of Defense/Department of Labor Transition Assistance Program (TAP).

Currently, VBOCs systematically collect intake data from clients; however, there is no process in place for VBOCs to systematically collect client outcomes. VBOCs collect client outcomes in an ad-hoc fashion, by following up with clients at arbitrary times (e.g., when a client returns for another service, calling clients at non-standardized intervals after service receipt) instead of using an established methodology across all centers. A centrally coordinated data collection effort is essential for developing consistent outcome reporting across all VBOCs. Also, OVBD plans to collect additional performance metrics in order to comprehensively assess how VBOCs operate and measure the outcomes and impacts of VBOCs. Lastly, the collected metrics lay the groundwork for future systematic impact evaluations.

SBA has hired an independent contractor, Optimal Solutions Group, LLC (Optimal), to conduct a performance evaluation of VBOCs in 2015. The methodology of this evaluation includes conducting a VBOC director survey, a client² outcome survey, and on-site, in-person interviews with directors, staff, and clients. (See Appendix B-2, C-2, D-1, E-1, F-1 for the justification per instrument by topic.)

The survey instruments are in response to Office of Management and Budget (OMB) Memoranda M-13-17³ and OMB Memorandum M-14-06⁴ requirements for evidence-based evaluations. The memoranda call for agencies to link program participant data with already existing administrative data to find new, innovative approaches for program delivery. The data collected through the VBOC survey instruments will complement, not replicate, the information available in administrative data.

Purpose and Use of the Information Collected

Indicate how, by whom, how frequently, and for what purpose, the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

This is a new collection effort. For the VBOC evaluation, the SBA has developed a total of five new data collection tools, including 1) a director survey, 2) a client survey, 3) a director interview protocol, 4) a staff interview protocol, and 5) a client interview protocol. SBA’s data collection methodology was developed with the following research questions in mind:

What is the client population of VBOCs?
What kinds of services are provided to clients, and how satisfied are the clients with the services received?
What are the programmatic inputs?⁵
What processes are used by VBOCs for determining client goals, challenges, and needs?
What are the programmatic outputs?⁶
What are the client outcomes?⁷
What kinds of data and reporting standards should be used by VBOCs to allow for a future impact study?
How can VBOCs better serve their clients?
How are VBOCs different from other resource partners and why?
How can SBA communicate guidelines and goals to resource partners?

The ultimate goals of the evaluation are to develop specific guidelines for systematic outreach methods, how to better deliver services, and systematic data collection of client outcomes, as well as to incentivize VBOC collaboration with other resource partners. The major research topics covered by the surveys, site visits, and interviews are presented in Exhibit 1. (The surveys and interview protocols are attached in Appendix B-1, C-1, D-1, E-1, F-1.) Optimal will collect, store on secure servers, and analyze all data collected; Optimal will provide a final report based on the analysis to SBA. At the end of the contract, Optimal will transfer the data collected to SBA through a secure server for them to store and use in future evaluations.

Exhibit 1. Instrument Research Topics

Research Topics	Source of Information
Research Topics	Director Survey	Client Outcome Survey	Director Interview	Staff Interview	Client Interview
	Business Ownership
Business status		x			x
Business stage		x			x
Industry		x			x
Business plan development		x			x
Number of employees		x			x
Number of veteran employees		x			x
Income		x			x
Loan and credit acquisition		x			x
Online sales		x			x
Home-based business		x			x
Business certifications		x			x
Legal entity type		x			x
Woman-owned business		x			x
Service-disabled veteran owned business		x			x
8(a) certified business		x			x
Boots to Business participation		x			x
Business goals		x			x
Small-business challenges		x			x
	VBOC Services
Referrals from other resource partners	x	x	x	x	x
Referrals to other resource partners	x	x	x	x	x
VBOC outreach methods	x	x	x	x	x
Services offered/ received	x	x	x	x	x
Client satisfaction		x			x
Client demographics		x			x
	VBOC Capacity
Staff experience and capacity	x		x	x
Internal VBOC goals	x		x	x
Documented staff training	x		x	x
VBOC website information	x		x	x
Method of service delivery	x		x	x
Client needs assessment	x		x	x
Location of service delivery	x		x	x
Training topics	x		x	x
Counseling topics	x		x	x
VBOC-specific initiatives	x		x	x
VBOC method and frequency of reporting	x		x	x
Inter-VBOC collaboration	x		x	x

Director Survey. This is a new data collection effort to elicit feedback across VBOC from the center directors. The information will enable SBA to evaluate VBOC performance as well as compile their challenges, recommendations, best practices, and lessons learned. The director survey will primarily collect quantitative information on program implementation (e.g., services provided, program staff, outreach efforts, intake procedures, tracking procedures), and program outputs (e.g., clients served, services utilized, referral networks).

Client Outcome Survey. The client outcome survey is a new data collection effort that will assist SBA with conducting an impact evaluation in the long-term of those small businesses or nascent small businesses that received services from a VBOC. The client outcome survey will collect data on client demographics, client business stage, client business needs and goals, services received by clients, client outcomes, clients’ perceptions about the impact of services, and client satisfaction with VBOCs.

Interviews. Interviews are essential for collecting comprehensive qualitative data as part of Optimal’s performance evaluation of OVBD. This qualitative data will contextualize and help explain the data collected through the director and client outcome surveys. In comparison to the surveys, therefore, different questions and data will be collected from the interviews. Information will be collected in person from VBOC directors, staff, and clients during site visits to the centers. During the two days spent at each selected site, an Optimal researcher will conduct interviews with directors, staff, and clients in order to identify VBOC client demographics, services provided, intake protocols, reporting protocols, service networks, referral networks, feedback mechanisms, best practices, lessons learned, and challenges.

Director Interviews. Director interviews will provide context for the interpretation of the quantitative data from the director survey regarding services provided, VBOC capacity, populations served, implementation strategies, service providers, referral networks, challenges, best practices, and ideas for improvement.

Staff Interviews. Staff interviews will provide context for interpreting the quantitative data from the director survey regarding services provided, VBOC capacity, populations served, implementation strategies, service providers, referral networks, challenges, best practices, and ideas for improvement. The interview will also be valuable to have an additional perspective to the directors’ perspective. Each VBOC has a variety of staffing arrangements, including full-time, part-time, salaried, and un-salaried employees. All staff members will be invited to participate in individual interviews (see Appendix E-1).

Client Interviews. From each selected VBOC, Optimal will interview two clients who received counseling services and two clients who received only training services. Purposive sampling will be used. For the counseling clients, at least one pre-venture and one in-business client will be selected; both clients should represent “typical” clients in terms of services received, business needs, and business outcomes. Training clients will be selected based on the topic of training attended and no receipt of counseling services; one client will be selected from each of the two most highly attended training topics offered. Client interviews will provide context for the quantitative data from the client survey regarding client goals, client needs, client challenges, client services received, referrals, business outcomes of clients, and client satisfaction with services received.

Use of Information Technology and Burden Reduction

Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden. Insert any applicable electronic web address.

Surveys. SBA is committed to complying with the E-Government Act of 2002 to promote the use of technology in order to lessen the burden of data collection. Survey data will be collected via web-based surveys, which facilitate the efficient collection and analysis of data. Web-based surveys are cost- and time-efficient, and they allow for flexibility in the survey’s format. The web-based survey also allows respondents to stop at any time and resume the survey at the point at which they last stopped from. Notice regarding the survey, interviews and site visits, as well as the consent forms, interview protocols and other information related to this information collection will be sent electronically to help minimize the burden on respondents.

Efforts to Identify Duplication and Use of Similar Information

Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The SBA OVBD does not currently employ a consistent methodology for collecting information about either directors or client outcomes in order to evaluate the VBOC program (both how it is performing and its impact on clients). The director and client outcomes surveys will be the first of their kind, collecting information that allows for a rigorous evaluation of the program. Similarly, the data items to be collected via the client survey are not currently available from a large number of clients in any administrative data or other data source. The results of the client survey will be used to complement administrative data, but they in no way duplicate administrative data.

Impacts on Small Businesses or Other Small Entities

If the collection of information impacts small business or other small entities, describe any methods used to minimize burden.

Clients. VBOC clients are either interested in starting small businesses, are in the process of starting a small business, or wish to grow their small business currently in operation. Considering the population that the VBOC serves, client-owned small businesses will be impacted by the surveys. To minimize the burden, efforts have been made to simplify and streamline the client outcome survey. The survey will be disseminated via an online platform, which will allow small business owners to take the survey at a time that is convenient, exit the survey and resume from the point that they last stopped at, and only answer questions that pertain to their experiences. The survey collects minimal information that is already captured in administrative data.

VBOC Directors and Staff. VBOCs, as small entities, will be impacted by OVBD’s study because directors and staff will be asked to participate in data collection, taking them away from their VBOC work. As with the client survey, the director survey has been designed to minimize burden. It is hosted on an online platform that allows directors to skip questions and open and exit the survey at their convenience.

Four centers will be asked to hold two-day site visits during which staff members will be asked to participate in interviews. This data collection effort will affect the time that staff and directors typically dedicate to providing business assistance. While interviews will take away from staff’s typical work time, each interview protocol has been designed to minimize burden and allow for questions to be skipped if not applicable to the respondent. In addition, since the interviewer is at the site over two days, interviews will be held when staff members are available.

Consequences of Collecting the Information Less Frequently

Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Across all five instruments, there are no technical or legal obstacles to reducing burden.

Director Survey. This data collection effort will enable OVBD to evaluate the program, identify lessons learned and best practices across VBOC centers, and find ways of improving the program. In the survey, directors will provide information on their operations as well as their best practices, challenges, and lessons learned. If this survey were not administered once to all directors, OVBD would not have a systematic understanding of the variety of implementation practices across VBOCs, nor could it provide guidance on best practices.

Client Outcome Survey. The client outcome survey will be administered to clients from the previous and current granting periods. Previously, VBOCs collected partial outcome data for a small sample of clients that either VBOCs followed up with on their own ad hoc timelines or came from clients contacting VBOCs and reporting outcome data. However, representative client information has not been collected in the past, preventing SBA from conducting a rigorous program evaluation. The standardized data collection process proposed here is part of a framework that will allow SBA to conduct future performance and impact evaluations of the program. If the client outcome survey is not conducted, OVBD will lack the individual level data that they will need to match VBOC clients to already existing federal datasets for the purpose of conducting an impact evaluation. In addition, after learning about the challenges clients face in starting small businesses and about any possible gaps in the VBOC services received by clients, OVBD will be able adjust their approach to service provision to better address the expressed needs of veterans. If the client outcome survey is not conducted, OVBD will lack the wide-ranging individual level data on a large number of clients, making it much more difficult to evaluate the program and finding ways of improving it so that the needs of veterans are better served.

Interviews. VBOC on-site interviews will only be held once as part of Optimal's performance evaluation of OVBD. The qualitative data generated during interviews will contextualize and help with the interpretation of the mostly quantitative results from the client and director surveys. If Optimal is unable to conduct VBOC site visits, they, as evaluators, will not be able to fully assess the individual variation in the VBOC assistance process, the best practices, challenges, and lessons learned of VBOCs and the distinctive military culture of VBOCs that distinguishes them from other similar business assistance programs.

Special Circumstances Relating to the Guideline of 5 CFR 1320.5

Explain any special circumstances that would cause an information collection to be conducted in a manner:

requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary, trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no special circumstances that would cause information to be collected in the manner described above. The collection of information will be conducted in a manner consistent with the guidelines in 5 CFR 1320.5.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency.

If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

The required public comment notice was published in the Federal Register on Thursday, December 11, 2014 at 79 FR 73684.⁸ All comments must have been received on or before February 9, 2015.

One set of comments was received. Comments emphasized the importance of collecting information from service disabled small business owners, eliciting more information about Boots to Business (B2B) course, and more connection to the Veteran’s Administration (VA) programs for entrepreneurs (e.g., Vocational Rehabilitation and Employment Program). The current instruments already address questions specific to service disabled small business owners; therefore no changes were made to instruments in that regard. However, response options in both the client and director surveys were expanded to include options specifying B2B and VA programs as sources for outreach and referral.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.

Apart from the request for comment through the FR notice, the only other consultation outside of the agency was with Optimal, the company SBA has contracted with to design the methodological approach for data collection and the survey instruments. Optimal functions as an independent third party to provide recommendations on the deliverables. Optimal’s team is composed of three members with experience in diverse disciplines including veteran families, labor economics, and program evaluation. The contact information for these individuals is presented in Section B of this document.

Explanation of Any Payment or Gift to the Respondents

Explain any decision to provide any payment or gift to respondents, other than re-enumeration of contractors or grantees.

Respondents will not be provided payments or gifts.

Assurance of Confidentiality Provided to Respondents

Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

Optimal, an independent contractor, will be collecting, compiling, and analyzing all data collected from VBOC sites and participants (those interviewed and surveyed). Optimal was hired to ensure that the information provided by clients, directors, and staff will in no way compromise their receipt of service, VBOC funding, or employment. The data collected will be stored on Optimal’s secure servers, ensuring that the data remain confidential. No survey or interview participants will be identified in any public reports. Assurance of confidentiality is authorized by the Privacy Act of 1974 and the Freedom of Information Act, including Exemption 4, which allows SBA to withhold from disclosure certain information that could cause substantial competitive harm. Contractors are required to adhere to the Privacy Act, which is stated in the contract with Optimal. Optimal implements data management security protocols to protect personal and sensitive information. Secure intranets are used to maintain project-related files, and secure servers use industry-standard methods such as firewalls, monitored access logs, virus protection, encrypted connections, password-protected accounts, and user authentication mechanisms to ensure the privacy of personal data. Optimal maintains a biometrically (physically) secure environment and employs a data security officer who oversees Optimal’s data. The security approach used to protect the restricted-use data (RUD) is based upon Defense in Depth principles. In general, the RUD enclave consists of a single web portal accessible only via a virtual private network (VPN) and remote desktop software. The server does not have access to the Internet, nor does it have public IP addresses open. Access is only granted via the secure VPN connection. The detailed description of data handling, storage and disposition, and the procedures for privacy safeguards and protection are presented in Appendix G-1.

Directors and Staff. All information collected from VBOC directors and staff during surveys and interviews will be reported in the aggregate. Optimal will conceal staff members' identities in any report provided to any organization inside or outside the government so that participants will feel comfortable to provide researchers with accurate and candid answers.

Clients. In the reports produced by Optimal, client data will be reported in the aggregate. However, SBA will require that the data set from the client outcome survey contain personally identifying information (PII) for the purpose of linking the client outcome data set to administrative datasets, allowing SBA to conduct an impact evaluation in the future. Therefore, Optimal will provide SBA with client PII; however, VBOCs will not see the client PII. Clients are informed of these conditions in the consent form.

Justification for Sensitive Questions

Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

VBOC clients will be asked about their businesses’ income, profits, loans, and lines of credit. An essential component of VBOC service is helping veterans generate funding to start or expand their own businesses, and clients will be asked about their recent and current financial situations to help assess VBOC effectiveness with helping clients secure financing for starting, continuing, and growing businesses. Although sensitive in nature, questions regarding business performance are essential in determining whether SBA is fulfilling its mission.

Client survey and interview questions also include questions about race, ethnicity, and disability in order to determine whether or not VBOC is fulfilling its mission of serving service-disabled veterans, as well as service to minority veterans. These questions were written in compliance with the OMB Standards for the Classification of Federal Data on Race and Ethnicity. Clients are allowed to not answer any question during an interview or on a survey.

Estimates of Hour Burden Including Annualized Hourly Costs

Provide estimates of the hour burden of the collection of information. Indicate the affected public, number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated.

SBA estimates that 2313.08 hours are needed to conduct this study (see Exhibit 2) and that the total annualized hourly costs are $49,015.46 (see Exhibit 3). The affected public and respondent types for this data collection is composed of VBOC directors, staff, and clients. Preliminary pre-tests were conducted for all data collection instruments.

Director Survey. The director survey and scripts were internally tested by Optimal to develop burden estimates (see Exhibit 2). For pretesting, Optimal conducted one cognitive interview with a VBOC director. Based on this, the total burden for the director survey is 28.50 hours. Optimal will conduct an additional cognitive interview and internet test after receiving conditional OMB approval of the instruments and methodology.

Client Outcome Survey. The client survey and scripts were internally tested by Optimal to develop burden estimates (see Exhibit 2). For pretesting, Optimal conducted one cognitive interview with a VBOC client. Based on this, the total burden for cohort 1 of the client survey is 1023.00 hours, and the burden for cohort 2 is 1186.00 hours. Optimal will conduct an additional two cognitive interviews and two web-based burden tests after receiving conditional OMB approval of the instruments and methodology.

Interviews. The interviews were pre-tested by an Optimal researcher during two VBOC site visits. Each site visit lasted two days. Across both site visits, the director interview protocol was pretested twice, and the client interview protocol was pretested five times. The pretests provided burden estimates. No staff interviews were pretested during the site visits; interview protocols were developed based on the observations and results of the preliminary site visits. No additional pretests will be conducted for interview protocols.

The interview protocols were revised based on pretesting feedback that suggested clarifying the instructions, revising the wording of some questions, and providing examples for unclear categories or phrases, all of which will reduce interview burden. The estimates in Exhibit 2 include the receipt and reading of e-mail invitations, introductory letters, reminder e-mails, and the duration of two-day site visits and interviews. The total burden for the on-site interviews is 75.58 hours.

Exhibit 2. Reporting Estimates of Hour Burden

Respondent Type	Instrument	Number of Respondents	Freq. of Response (annual)	Total Annual Response	Avg. Hours per Response	Annual Burden	Number of Non-Respondents	Freq. of Response (annual)	Total Annual Response	Avg. Hours per Response	Annual Burden	Total Burden Hours
		Respondents					Non-respondents
Director	Survey pretests	3	1	3	1	3	0	0	0	1	0	3
Director	Survey instrument	15	1	15	1.3	19.5	0	0	0	1.3	0	19.5
Director	Survey invite e-mail	15	1	15	0.08	1.2	0	0	0	0.08	0	1.2
Director	Introduction e-mail	15	1	15	0.08	1.2	0	0	0	0.08	0	1.2
Director	Reminders	15	3	45	0.08	3.6	0	0	0	0.08	0	3.6
Director Survey subtotal		15				28.5	0				0	28.5
Cohort 1 Client	Survey pretest	5	1	5	1	5	0	0	0	1	0	5
Cohort 1 Client	Survey invite	2000	1	2000	0.08	160	0	1	0	0.08	0	160
Cohort 1 Client	Reminders	2000	3	6000	0.08	480	0	3	0	0.08	0	480
Cohort 1 Client	Survey instrument	900	1	900	0.42	378	1100	0	0	1	0	378
Cohort 1 subtotal		900				1023	1100				0	1023
Cohort 2 Client	Survey invite	2000	1	2000	0.08	160	0	1	0	0.08	0	160
Cohort 2 Client	Reminders	2000	3	6000	0.08	480	0	3	0	0.08	0	480
Cohort 2 Client	Survey instrument	1300	1	1300	0.42	546	700	0	0	0.42	0	546
Cohort 2 subtotal		1300				1186	700				0	1186
Client	Site visit interview pretesting	5	1	5	0.5	2.5	0	0	0	0.5	0	2.5
Client	E-mail/ Phone Scheduling	16	1	16	0.08	1.28	0	0	0	0.08	0	1.28
Client	Consent form	16	1	16	0.08	1.28	0	0	0	0.08	0	1.28
Client	Site visit interview	16	1	16	0.5	8	0	0	0	0.5	0	8
Staff	Consent Form	16	1	16	0.08	1.28	0	0	0	0.08	0	1.28
Staff	Site visit interview	16	1	16	1	16	0	0	0	1	0	16
Director	Site visit pretesting	2	1	2	1.5	3	0	0	0	3	0	3
Director	Intro letter	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32
Director	Follow-up E-mail	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32
Director	Consent form	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32
Director	Reminders	4	3	12	0.08	0.96	0	0	0	0.08	0	0.96
Director	Site visit interview	4	1	4	1.5	6	0	0	0	1.5	0	6
Director	Site visit	4	1	4	8.5	34	0	0	0	8.5	0	34
Director	Thank you	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32
Site Visit Subtotal		36				75.58	0				0	75.58
GRAND TOTAL		2251				2313.08	1800				0	2313.08

Exhibit 3. Annualized Cost of Burden

Respondent Type	Instrument	Number of Respondents	Freq. of Response (annual)	Total Annual Response	Avg. Hours per Response	Annual Burden	Number of Non-Respondents	Freq. of Response (annual)	Total Annual Response	Avg. Hours per Response	Annual Burden	Total Burden Hours	Average Hourly rate	Total Annual Respondent Cost
		Respondents					Non-respondents
Director	Survey pretests	3	1	3	1	3	0	0	0	1	0	3	31.61⁹	$94.83
Director	Survey instrument	15	1	15	1.3	19.5	0	0	0	1.3	0	19.5	31.61	$616.40
Director	Survey invite e-mail	15	1	15	0.08	1.2	0	0	0	0.08	0	1.2	31.61	$37.93
Director	Introduction e-mail	15	1	15	0.08	1.2	0	0	0	0.08	0	1.2	31.61	$37.93
Director	Reminders	15	3	45	0.08	3.6	0	0	0	0.08	0	3.6	31.61	$113.80
Director Survey subtotal		15				28.5	0				0	28.5	31.61	$900.89
Cohort 1 Client	Survey pretest	5	1	5	1	5	0	0	0	1	0	5	20.85¹⁰	$104.25
Cohort 1 Client	Survey invite	2000	1	2000	0.08	160	0	1	0	0.08	0	160	20.85	$3336.00
Cohort 1 Client	Reminders	2000	3	6000	0.08	480	0	3	0	0.08	0	480	20.85	$10008.00
Cohort 1 Client	Survey instrument	900	1	900	0.42	378	1100	0	0	1	0	378	20.85	$7881.30
Cohort 1 subtotal		900				1023	1100				0	1023	20.85	21329.55
Cohort 2 Client	Survey invite	2000	1	2000	0.08	160	0	1	0	0.08	0	160	20.85	$3336.00
Cohort 2 Client	Reminders	2000	3	6000	0.08	480	0	3	0	0.08	0	480	20.85	$10008.00
Cohort 2 Client	Survey instrument	1300	1	1300	0.42	546	700	0	0	0.42	0	546	20.85	$11384.10
Cohort 2 subtotal		1300				1186	700				0	1186	20.85	$24728.10
Client	Site visit interview pretesting	5	1	5	0.5	2.5	0	0	0	0.5	0	2.5	20.85	$52.13
Client	E-mail/ Phone Scheduling	16	1	16	0.08	1.28	0	0	0	0.08	0	1.28	20.85	$26.69
Client	Consent form	16	1	16	0.08	1.28	0	0	0	0.08	0	1.28	20.85	$26.69
Client	Site visit interview	16	1	16	0.5	8	0	0	0	0.5	0	8	20.85	$166.80
Staff	Consent Form	16	1	16	0.08	1.28	0	0	0	0.08	0	1.28	20.52¹¹	$26.27
Staff	Site visit interview	16	1	16	1	16	0	0	0	1	0	16	20.52	$328.32
Director	Site visit pretesting	2	1	2	1.5	3	0	0	0	3	0	3	31.61	$94.83
Director	Intro letter	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32	31.61	$10.12
Director	Follow-up E-mail	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32	31.61	$10.12
Director	Consent form	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32	31.61	$10.12
Director	Reminders	4	3	12	0.08	0.96	0	0	0	0.08	0	0.96	31.61	$30.35
Director	Site visit interview	4	1	4	1.5	6	0	0	0	1.5	0	6	31.61	$189.66
Director	Site visit	4	1	4	8.5	34	0	0	0	8.5	0	34	31.61	$1074.74
Director	Thank you	4	1	4	0.08	0.32	0	0	0	0.08	0	0.32	31.61	$10.12
Site Visit Subtotal		36				75.58	0				0	75.58		$2056.92
GRAND TOTAL		2251				2313.08	1800				0	2313.08		$49015.46

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information.

There is no other total annual cost burden to respondents or record keepers.

14. Annualized Costs to Federal Government

Provide estimates of annualized cost to the Federal government. Provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

Annualized cost to the federal government is $358,344.40. This includes the costs associated with the contractor conducting the project ($349,995.64) and the salary of the assigned OVBD employees ($8,348.77). The cost of the OVBD employee, Research Associate, involved in project oversight is estimated at GS-13, step 1 at $43.67 per hour based on 2,080 hours per year. OVBD anticipates that this person will work 80 hours per year for 1 year. The annual cost for this OVBD employee over the course of this study is $3,493.19. The cost of the OVBD employee, Associate Administrator, involved in project oversight is estimated at GS-15, step 1 at $60.70 per hour based on 2,080 hours per year. OVBD anticipates that this person will work 80 hours per year for 1 year. The annual cost for this OVBD employee over the course of this study is $4,855.58. The total annual cost for both of these OVBD employees over the course of this study is $8,348.77. Federal employee pay rates are based on the General Schedule of the Office of Personnel Management for 2015 for the Washington DC locality.¹²

All costs for conducting the study are included in the contract between the SBA, OVBD, and Optimal Solutions Group, LLC, under the contract number SBAHQ-13-D-0003. The total cost is $349,995.64 from September 23, 2014 through March 22, 2016, which encompasses the development of the evaluation architecture, data-collection instruments, the Federal Register listing, and the Office of Management and Budget (OMB) package. Optimal will also provide recommendations on performance metrics, methodology to use for collecting data, and the federal datasets to use (along with client outcome data) for future impact evaluations as well as produce a performance evaluation report and final report based on collected data.

15. Explanation for Program Changes or Adjustments

Explain the reasons for any program changes or adjustments reported in items 13 or 14 of OMB form 83-I.

This is a new data collection; there are no program changes or adjustments.

16. Plans for Tabulation and Publication and Project Time Schedule

For collections of information whose results are planned to be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

This study will use both quantitative (i.e., descriptive and inferential statistical techniques) and qualitative methods for analyzing the data. The descriptive statistics will include frequencies, percentages, averages, standard deviations, and ranges for the data elements. The inferential statistics will include bivariate techniques, such as the t-test and chi-square test, as well as multivariate techniques, such as regressions and analyses of variance. Standard errors and 95% confidence intervals will also be provided for the estimates. The analysis is expected to include tables for the descriptive statistics and visual graphs, such as for frequency distributions. Other possible graphical elements include pie charts and maps in order to visually represent the data. Qualitative analysis will involve grounded theory methods to inductively determine themes across interviews. Quotations and summary tables will used to explain findings.

Data analysis will be driven by the research questions and providing answers to them. Exhibit 1 lists the research topics and the research questions to be addressed by the study. The analysis will determine the populations served by VBOCs; services delivered by VBOCs; VBOC resource capacity; VBOC challenges, lessons learned, and best practices; and preliminary client outcomes.

The surveys will be administered once in November 2015. Qualitative interviews with directors, staff, and clients will occur in December 2015. The data collection will be completed by January 2016. The complete schedule for the project is attached in Appendix A-1 Deliverable Schedule.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

SBA plans to display the OMB expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Explain each exception to the certification statement identified in item 19, "Certification Requirement for Paperwork Reduction Act." of OMB Form 83-I. Part V "Certification Requirement for Paperwork Reduction Act." If Agency is not requesting an exception, the standard statement should be used.

The agency is not requesting an exception to the certification statement.

1The previous VBOC granting period ends in April 2015. OVBD is in the process of choosing new centers, and the new granting period will begin in April 2015. As of now, OVBD is still considering applications and deliberating on the number of VBOCs. The maximum number of VBOCs is 15, so this document is working under the assumption that there will be 15 centers in the new granting period.

2 Clients are defined as individuals who have registered online, have received VBOC services, and are in the WebCATS/Neroserra database, the client management system used by VBOCs.

3 https://www.whitehouse.gov/sites/default/files/omb/memoranda/2013/m-13-17.pdf

4 https://www.whitehouse.gov/sites/default/files/omb/memoranda/2014/m-14-06.pdf

5 Inputs refer to VBOC funding levels, amount of staff, expertise of staff, etc.

6 Outputs refer to types of services offered, amount of services provided, etc.

7 Client outcomes refer to business performance indicators (e.g., firm birth, firm death, revenue, employees, loans) and success in meeting individual client needs.

8 http://www.gpo.gov/fdsys/pkg/FR-2014-12-11/html/2014-29140.htm

9 Directors’ hourly wages were estimated from the Bureau of Labor Statistics Occupational Employment and Wages, May 2013. For 11-9151 Social and Community Service Managers, the mean hourly wage is $31.61 and the mean annual wage is $65,750.

10 Clients’ hourly wage estimations are based on veterans’ average annual salary of $43.370 as cited in Tennant, J. (2012, August). “Disability, Employment, and Income: Are Iraq/ Afghanistan Era U.S. Veterans Unique?” Monthly Labor Review: 3-10.

11 The staff average hourly wage was estimated from the Bureau of Labor Statistics Occupational Employment and Wages, May 2013. For 21-1099 Community and Social Service Specialists, $20.52 is the mean hourly wage and $42,690 is the mean annual wage.

12 http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/15Tables/html/DCB.aspx

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kelsye Turner
File Modified	0000-00-00
File Created	2021-01-24

VBOC Supporting Statement Part A_Final 5-11-15