Evaluation of Field Office Performance 8(a) Focus Groups

Request for Approval under the “Generic Clearance for Formative Data Collections for Evaluation, Research, and Evidence-Building”

(OMB Control Number: 3245-0425)

TITLE OF INFORMATION COLLECTION: Evaluation of Field Office Performance 8(a) Focus Groups

PURPOSE:

This data collection effort is part of the Small Business Administration’s (SBA) field operations performance evaluation. The goal is to enhance the SBA’s understanding of the role of the district offices in the entrepreneurial ecosystem. This focus group data collection effort focuses on firms participating in the 8(a) Business Development Program (8(a) Program). These focus group efforts aim to connect with operators of small businesses that district offices work within the 8(a) Program. These individuals have first-hand experience working with district offices and can provide unique insight into this relationship and the role of district offices in local small business ecosystems. Findings from this effort and the companion 8(a) Program survey will contribute to the Office of Field Operation’s effort to promote continuous improvement, align activities around equitable entrepreneurial ecosystem building, enable strategic decision-making, and support communication and collaboration within the ecosystem.

TYPE OF COLLECTION: (Check one)

[ ] Survey/ questionnaire [ ] Cognitive interview

[X] Focus group/small group discussion [ ] User testing

[ ] Interviews [ ] Observation/field study

[ ] Other: _______________________

CERTIFICATION:

I certify the following to be true:

1. The collections are voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collections are non-controversial and do not raise issues of concern to other Federal agencies.

4. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the near future.

5. Personally identifiable information (PII) is collected only to the extent necessary and is not retained.

6. Information gathered is intended to be used for general service improvement and program management purposes.

7. Summaries and aggregate results may be included in public-facing evaluation or similar report.

8. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

Name: Katherine Aaby

Title: Associate Administrator, OPPCFO

All instruments used to collect information must include:

OMB Control No. 3245-0425

Expiration Date: 10/31/2025

DESCRIPTION OF THIS SPECIFIC COLLECTION

To assist review, please provide answers to the following question:

1. Administration of the Instrument

   1. How will you collect the information? (Check all that apply)

[X ] Web-based or other forms of Social Media

[ ] Telephone or other audio technology

[ ] Video

[ ] In-person

[ ] Mail

[ ] Other _________________________

2. Will interviewers or facilitators be used? [X] Yes [ ] No

2. Description of respondents/participants

Two focus groups of five to eight individuals representing 8(a) Program firms will participate in the virtual focus groups. This data collection does not intend to generalize findings to the broader population. Invitees will be purposively sampled to achieve diversity of geographic region, urban/rural, owner characteristics, business characteristics, and industry mix. The first focus group will consist of individuals whose businesses are in the developmental stage (years one through 4). The second focus group will consist of individuals whose businesses are in the transitional stage (the last five years). Recruited individuals will be identified from the list of 8(a) Program participants maintained by the SBA.

3. Activity timeframe

The focus groups will occur in late Spring and early Summer of 2024.

4. Collection procedures

After developing the purposive sample, the SBA will send an official SBA introduction email introducing the study, followed by a recruitment email inviting participants to participate in a focus group. A follow-up email will be sent to invitees that have yet to respond. Once a time for the focus group has been established, a confirmation email with a calendar invitation Teams link will be sent. A reminder email will be sent the day before the scheduled focus group.

Based on pilot testing of the focus group protocols with two individuals, the focus groups are expected to last 60 minutes. However, 90 minutes will be scheduled to allow all attendees ample time for comments without feeling rushed. An experienced facilitator will conduct each focus using a video conferencing platform. The contractor will request permission from the attendees to audio record the meeting for transcription and analysis.

5. Provide the question list (Submit all instruments, instructions, and scripts with this request).

See Appendix A (Focus Group Correspondence and Communication), B (Focus Group Script and Informed Consent), and C (Focus Group Instrument).

6. Use and dissemination of the results

Aggregated results will be shared in a one-page summary report, evaluation report, and briefings. They will also be used to inform a customer journey map specific to 8(a) participants, the development of performance measure recommendations, and a visualization depicting the role of the SBA with the broader entrepreneurial ecosystem.

PERSONALLY IDENTIFIABLE INFORMATION

1. Is personally identifiable information (PII) collected? [] Yes [X] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No

GIFTS OR PAYMENTS

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [X ] No

If Yes, describe:

BURDEN HOUR COMPUTATION

Category of Respondent	No. of Respondents	Participation Time	Burden Hours
Individuals	16	90 minutes	24
Totals	16	90 minutes	24

BURDEN COST COMPUTATION

Respondent cost estimates are based on the Bureau of Labor Statistics 2023 median weekly earnings for full-time wage and salary workers of $28.63 per hour.

Category of Respondent	No. of Respondents	Hourly Rate	Response Time	Total Estimated Cost
Individuals	16	$28.63	90 minutes	$687.12
Totals	16	$28.63	90 minutes	$687.12

FEDERAL COST: The estimated annual cost to the Federal government is $3,196, which includes 10 hours for a GS-14 evaluator, 5 hours for a GS-15 manager, and $2,400 for an independent third-party contractor.

OTHER SUPPORTING INFORMATION

REQUESTED APPROVAL DATE: April 8, 2024

ICR REQUEST CONTACT:

Name: Shay Meinzer

Title: Lead Program Evaluator/COR

Telephone Number: (202) 539-1429

Email Address: [email protected]

Appendix A

Focus Group Correspondence and Communication

Initial Communication Introducing Study and Upcoming Data Collection Requests

Email Subject Line: Requesting your participation in an important research study highlighting the role of district offices within the SBA

Email To: [8(a) firms]

Cc: [email protected]; [email protected]

Email From: [SBA] 

 

Good morning/afternoon, 

 

The purpose of this email is to inform you of an important research study coming out of the Small Business Administration (SBA). The SBA has contracted Pacific Research and Evaluation (PRE) to study the role of district offices within the SBA and the local areas that they serve. The results of this work will be used by the SBA to better understand the impact of district offices and to identify ways the SBA can continue to serve small businesses such as yours in the years ahead.

In the next week, you will receive an email from PRE asking you to participate in a survey focused on your experience with your local SBA district office. You may also receive an invitation to participate in a focus group. We would like to ask for your help and encourage you to provide your input and participate in these important efforts.

 

If you have any questions, please email Shay Meinzer at [email protected] or Shalyn Stevens at [email protected] who will be managing these efforts.

 

Sincerely, 

[INSERT SIGNATURE BLOCK]

Invitation to Participate in External Focus Groups from PRE

Email Subject Line: Your participation is requested in a focus group for the SBA’s district office research efforts

Email To: [8(a) firms]

Cc: [email protected]

Email From: Shalyn Stevens 

 

Good morning/afternoon, 

 

As [SBA] shared with you on [DATE], the SBA has contracted the team at Pacific Research and Evaluation (PRE) to conduct an important research study on the role of district offices within the SBA and the local areas that they serve. To assist in this effort, we are asking for your participation in a 90-minute virtual focus group about your experience with your local SBA district office.

Please use the following link to provide your availability for the weeks of [DATES] by [DATE]:

 

[INSERT LINK]  

Your conversation with us (the team at PRE) is confidential – your name will not be associated with any comments you provide in our final report to the SBA or shared with anyone outside of the research team. We will be conducting a series focus groups with SBA resource partners, community partners, small businesses, and other stakeholders, and reporting only general themes heard across these sessions. Your participation is greatly appreciated!

  

If you have any questions, please email me at [email protected] or Shay Meinzer at [email protected].

 

Thank you,

Shalyn Stevens, Ph.D.  

Senior Research Associate, Pacific Research and Evaluation 

Follow-Up Invitation to Participate in External Focus Groups from PRE

Email Subject Line: REMINDER: Your participation is requested in a focus group for the SBA’s district office research efforts

Email To: [8(a) firms]

Cc: [email protected]

Email From: Shalyn Stevens 

 

Good morning/afternoon, 

 

This email is a reminder to provide your availability for a 90-minute virtual focus group about your experience with your local SBA district office.

Please use the following link to provide your availability for the weeks of [DATES] by [DATE]:

 

[INSERT LINK]  

If you have any questions, please email me at [email protected] or Shay Meinzer at [email protected].

 

Thank you,

Shalyn Stevens, Ph.D.  

Senior Research Associate, Pacific Research and Evaluation 

Confirming Participation in External Focus Groups from PRE

Calendar Invitation Subject: SBA Focus Group with Pacific Research and Evaluation

Calendar Invitation To: [8(a) firm who provided availability and is being scheduled for a focus group]

Calendar Invitation From: Shalyn Stevens 

 

[Teams Link]

Thank you for your willingness to participate in a 90-minute virtual focus group with Pacific Research and Evaluation. You are scheduled to participate on [DATE/TIME]. Please use the Microsoft Teams link above to access the focus group.

During this session, we will discuss your experience with your local SBA district office. More specific topic areas include:

• Your 8(a) journey.

• Your relationship with your local SBA district office and how being a part of the 8(a) program has impacted your business and meets your needs.

• Your perspective on how your local SBA district office doing in terms of helping your business and other small businesses in your local area succeed.

If you have any questions, please email me at [email protected] or Shay Meinzer at [email protected].

 

Thank you,

Shalyn Stevens, Ph.D.  

Senior Research Associate, Pacific Research and Evaluation 

Reminder to Participate in External Focus Groups from PRE

Email Subject Line: Reminder: You are scheduled to participate in a focus group to support the SBA’s district office research efforts

Email To: [8(a) firms]

Cc: [email protected]

Email From: Shalyn Stevens 

 

Good morning/afternoon, 

 

This email is a reminder that you are scheduled to participate in a 90-minute virtual focus group via Microsoft Teams tomorrow at [TIME]. You should have received a calendar invitation with the details of the focus group. Please use the link in the calendar invitation to access the session.

If you have any questions, please email me at [email protected] or Shay Meinzer at [email protected].

 

Thank you,

Shalyn Stevens, Ph.D.  

Senior Research Associate, Pacific Research and Evaluation 

Appendix B

Focus Group Script and Informed Consent

Focus Group Introduction and Informed Consent:

Thank you so much for joining us today. My name is [NAME], and I am joined by my colleagues [COLLEAGUES]. We are part of a team at Pacific Research and Evaluation that is contracting with the SBA to conduct an evaluation of the role of district offices within the SBA and the local areas that they serve. The results of this work will be used by the SBA to better understand the impact of district offices and to identify ways the SBA can continue to serve small businesses such as yours in the years ahead.

 

We have a set of questions prepared that should take us about 90 minutes to get through, focused on your experience with your local SBA district office. Our hope is for this to be conversational and that you will pick up on each other’s ideas, but please don’t worry if you don’t get to answer every question.

We want to stress the confidentiality of our conversation today – your name will not be associated with any comments you provide in our final evaluation report. We will be conducting focus groups with other 8(a) firms, SBA resource partners, community partners, and other stakeholders, and will be reporting themes that we hear across these sessions into our final report. We encourage you to be as open and honest in your feedback with us today as possible. We understand that people have different experiences and it’s important for us to hear the different perspectives you may have. While we will maintain confidentiality on our end, we also ask that you do the same and not share anything that we talk about today outside of this meeting.

To ensure consistency in the way we analyze data from these and other interviews throughout the evaluation, we would like to take an audio recording of this meeting. The recording will be made with an external audio device and will be transcribed for data analysis, with all identifying information removed. It will be saved in a secure folder on our file system at PRE. Is that okay with you? [WAIT FOR VERBAL CONSENT FROM ALL PARTICIPANTS].

Do you have any questions before we get started?

Please feel free to use the chat throughout our discussion. [MY COLLEAGUE] will be posting the questions in the chat as we go.

Focus Group Closing:

Thank you for your time today. If you have questions or anything else to add later, please feel free to email [FOCUS GROUP FACILITATOR] at [FACILITATOR EMAIL] [PASTE IN CHAT].

Appendix C

Focus Group Instrument

1. Let’s start by having you share a little bit about yourself and your business.

2. How many years has your business been a part of the 8(a) program?

   1. Why was your organization initially interested in participating in the 8(a) program?

      1. What were you hoping to get out of the 8(a) program?

   2. What has been your experience maintaining your certification (if applicable)?

      1. What kind of support do you receive from the SBA or your local SBA district office to maintain your certification?

3. Let’s talk a bit more about your experience with your local SBA district office.

   1. How often do you interact with your local SBA district office? Through what means do you interact?

      1. How is contact generally initiated?

      2. What do you regularly communicate about?

      3. How has your relationship with your local SBA district office or Business Opportunity Specialist grown or changed over time?

      4. How have your needs changed over time? How has the SBA been able to support any change in needs?

      5. What difficulties has your organization experienced, if any, in terms of connecting with your local SBA district office?

   2. What resources or support has your local district office provided your organization since joining the 8(a) program? Are there any examples of particularly impactful support or resources they have provided you?

      1. How does your local SBA district office keep you updated about the services and resources the SBA offers to businesses like yours?

         1. How would you like to be kept updated about these services and resources?

   3. What barriers or bottlenecks, if any, has your organization experienced in the process of working with your local SBA district office to receive resources or support?

4. What changes have you noticed in your business as a result of joining the 8(a) program and working with your local SBA district office? How has this directly or indirectly impacted your business?

   1. How has being involved in the 8(a) program helped prepare you to compete for federal and other contracts?

      1. Upon joining the 8(a) program, how quickly were you expecting to start receiving federal contracts?

         1. How, if at all, did your experience differ from your expectations?

      2. How long after joining the 8(a) program did you receive your first federal contract award?

      3. Approximately how many federal contracts have you competed for?

      4. Approximately how many federal contracts have you been awarded?

      5. What is the approximate value of those contracts?

      6. If you have been awarded a contract since joining the 8(a) program, what challenges, if any, did you experience trying to understand the terms of the contract?

         1. Are there any additional resources you would like to navigate the contract process?

   2. Next, we want to talk about your experiences with other resources and types of support provided by the SBA.

      1. What has been your experience with business development services offered by the SBA?

      2. What about access to training and technical assistance?

      3. Access to mentorship opportunities?

      4. Support receiving federal surplus property on a priority basis?

      5. And lastly, support increasing capacity?

   3. What resources and support systems do you expect will be provided to you once you leave the program?

      1. What else could the SBA be doing to help you succeed after completing the 8(a) program?

5. From your perspective, how is your local SBA district office doing in terms of helping your business and other small businesses in your local area succeed?

   1. What are they doing well?

   2. What could they be doing differently to better support you and other businesses?

6. In general, how well does the 8(a) program meet your needs?

   1. Is there anything you would change about the 8(a) program?

   2. If you joined the 8(a) program prior to the COVID-19 pandemic, how has the pandemic changed your experience in 8(a)?

7. Is there anything else you think we missed or that you would like us to know?

16