Evaluation of the Veteran Business Outreach Centers

(OMB Control Number: 3245 - XXXX )

Appendix E-1 Staff Interview Consent Form, Protocol, and Construct Justification

Interview Consent Form for Staff

The U.S. Small Business Administration (SBA)’s Office of Veterans Business Development (OVBD) has contracted Optimal Solutions Group, LLC (Optimal) to conduct a fair and independent performance evaluation of Veterans Business Outreach Centers (VBOCs). This evaluation comprises a landscape and location gap analysis of VBOCs, the development of performance metrics, an assessment of client attitudes about the impact of VBOC services, and the development of a framework that will allow for the implementation of impact evaluations.

As part of this analysis, Optimal will be conducting individual interviews with the directors of the VBOCs and interviews with other VBOC staff. Your participation is completely voluntary. As required by the Paperwork Reduction Act, SBA (OVBD) may not conduct this interview unless it has been approved by the Office of Management and Budget (OMB). SBA has obtained that approval under OMB control #3245-xxxx, Expiration date xx/xx/xxxx. The estimated burden is 65 minutes. The primary purpose of the staff interviews is to collect information on processes used for interacting with clients, collecting and reporting data, providing services to them, and addressing challenges. Other topics covered during the interviews include the types of services offered by this VBOC, how services are delivered, client intake procedures, reporting protocols, networks in your area, and challenges, lessons learned, and best practices of this VBOC.

Confidentiality and Usage

This interview will be documented through written notes and an audio recording, which will be used in part to develop an internal SBA report and a publically available report for SBA at the completion of the project. Any quotations or information “on-the-record” will be attributed to the original source in the reports. The research team will provide the option of going “off-the-record” if requested by the interviewee, in which case the information in reports submitted to SBA will not be attributed to a person or center. Instead, the information will allude to the respondent’s information through phrases such as “a VBOC staff member” or a “VBOC center.”

Storage and Access

The interview notes and audio recording will be stored on a secure server accessible only to the Optimal research team and will not be provided to SBA staff. The interview notes and recording will be retained for no less than 3 years after the completion of the interviews.

Your signature on this form grants the Optimal research team permission to conduct the interview as described above for the above-referenced study. The research team will not use the information provided for any reason other than those stated in this consent form without your permission.

Signature Date

Printed Name

Staff Interview Protocol

General Information about the Interview

Interview date and time: Eastern Standard Time

Interview Questions

The interview questions below are provided for your review in preparation for [VBOC’s] upcoming group staff interview. The questions are asked about you and your organization’s work at the VBOC. Please note that some of these questions may be skipped during the interview because they are not applicable to you or your program, or because additional follow-up questions not listed here may be asked based on the information provided during the interview.

Notation:

• [F] indicates a follow-up question.

VBOC Resources

1. How long have you been working with this VBOC?

[F] What is your current role?

[F] What are your typical responsibilities?

2. What drew you to working at a VBOC?

[F] Do you have any unique skills that make you well-suited for working at a VBOC?

3. Have you received training from this VBOC? Please describe.

What you do

Overall program approach

1. How would you broadly describe the mission of this VBOC?

Program services

1. What are the services that you personally provide to clients?

[F] Do you provide more training sessions or one-on-one counseling?

[F] What is the range of topics you address in training sessions?

[F] What is the range of topics you address in counseling sessions?

[F] Is there a service or topic that you specialize in? How was this service or topic decided upon or developed?

2. Please rank the most common formats of the services that you deliver (e.g., one-on-one in-person, group online training, phone, etc.).

[F- if any virtual interaction] What challenges do you face when trying to keep clients engaged, particularly those who are not in close proximity to the VBOC?

3. Where do you typically deliver your services (e.g., at your VBOC, a resource partner location, an alternative location)?

A typical day

1. How many days per week are you working on-site? Off-site?

2. What is the approximate number of clients that you interact with (by telephone, in-person, or via e-mail) on a typical day?

3. What is the approximate number of clients that you counsel one-on-one (by phone, in-person, or e-mail) on a typical day?

4. What is the approximate number of clients that you train (by telephone, in-person, or online) on a typical day?

5. Are there certain business industries that are common among your clients?

6. What are the three most common needs of clients?

7. Approximately, what percentage of your clients receives multiple types of services (e.g., counseling and training)?

8. Approximately what percentage of your clients receives multiple assistance sessions (e.g., attends multiple trainings, receives multiple counseling sessions, etc.)?

9. How has the client flow and service utilization changed since the start of the grant?

Client intake

1. Please describe any formal or informal screening process used by your VBOC for providing client service.

2. Please describe the client intake procedures.

[F] Do you use the counseling questions (in WebCATS/Neoserra) for each client?

[F] Do you use the training questions (in WebCATS/Neoserra) for each client?

[F] Do you record client-level information for clients (name, address, e-mail, business stage, etc.) attending training sessions?

[F] Do you use any other forms to record data about clients?

[F] How would you rate your consistency in completing these forms?

[F] Briefly describe any challenges that you face in completing these forms.

[F] How could the intake procedure be improved?

3. Now I’d like to discuss the general process you use for assessing a client’s needs and for providing appropriate service. Specifically:

[F] What questions are asked of the client? Is there a standard form or protocol used?

[F] How are decisions made about assistance to provide to the client? Is there a standard decision tree used?

[F] Have you received training on how to assess clients’ needs? Was the training sufficient?

[F] If a client asks for a particular service or kind of assistance but you feel that it is not necessary or sufficient to meet the client’s need, will you recommend a different service?

[F] How often do you re-assess a client’s needs or goals?

Referrals and community connections

1. How often do you refer clients to other resources or service providers that might help clients achieve their goals?

[F] Where do you most commonly refer clients?

[F] If a client is referred to another service provider, is it standard practice to “check in” with the client to see if the client acted on the referral and what outcomes were achieved?

[F] Do your partners refer clients to you?

Clients

Client-tracking and feedback

1. How and when do you document the services or assistance you provide to clients?

[F] Do you record anything outside the WebCATS/Neoserra system?

[F] What is your process for collecting client satisfaction ratings?

[F] What is your process for collecting client outcomes (e.g., business stage, loans received, revenue, employment, contracts awarded, etc.)?

2. Is it common for your clients to initiate follow-up interactions (in-person conversations, phone discussions, or e-mail exchanges)?

[F] What is the most common method for a follow-up meeting: telephone, in-person, or via e-mail?

3. Has the VBOC changed its approach to help clients based on client feedback from the client satisfaction survey or from information gathered in client follow-up interactions with the center?

Moving Forward

1. What are some of the common challenges you face in providing assistance to clients?

2. What are some of the best practices of this center?

3. How could this VBOC improve practices and services?

4. What are some of the lessons learned from assisting clients since the start of the grant?

5. In what areas, if any, would you like more guidance from SBA?

Staff Interview Protocol Construct Justification

Construct	Justification
VBOC Resources	This section asks about each staff member’s skills and responsibilities. In addition, OVBD would like to know if staff members received any formal training from their VBOC. Individual staff members’ experience is important because they directly interact with clients and they often rely on their own business expertise to advise clients.
Overall Program Approach	OVBD would like to know how staff broadly defines the mission. This information will allow OVBD to know if VBOC staff is communicating the same mission, and if VBOC services and clients served are aligned with a VBOC's mission.
Program Services	OVBD would like to know what services each staff member delivers, and where and how they are delivered. While directors are more likely to stay at the VBOC to run the center, staff members and consultants are freer to travel to different sites to serve clients throughout the geographic region.
A Typical Day	This section is meant to address client flow and the tasks that staff members perform in a typical VBOC working day. In addition, these questions address clients’ common needs and overall tendency to return for multiple services. This factor is important for anticipating the staff capacity that will be needed to serve a client population, especially if a VBOC is in the process of expanding.
Client Intake	These questions pertain to how clients' needs are assessed and client information is entered into a database. There is no standard client intake process across VBOCs, so each VBOC may have their own standard operating procedure for assessing clients' needs and entering client information into a database. OVBD wants to know each center’s strategy in order to suggest improvements and share best practices across VBOCs.
Referrals and Community Connections	OVBD would like to have a picture of the resources and ongoing partnerships that VBOCs have developed in order to assess whether or not there is a gap in resources with services offered or with a referral network. In order for VBOCs to succeed, they must have relationships with business assistance programs and SBA resource partners across their geographic span.
Client-Tracking and Feedback	VBOCs are required to submit their WebCATS/Neoserrra data and reports to OVBD on a quarterly basis. Other than that requirement, each VBOC can independently determine how they collect client feedback and track client outcomes. OVBD would like to know what methods are used by VBOCs in order to make recommendations about the type of information that should be collected, as well as methods for reporting accurate and timely information.
Moving Forward	This portion allows for staff to provide OVBD with suggestions for how the VBOC program can be improved moving forward.

8