Feasibility Study - Responsive

Attachment C: Feasibility Study Interview Protocol

PART A: INTERVIEW DISCUSSION GUIDE

The OMB Control Number for this information collection is 0584-XXXX and the expiration date is XX/XX/XXX.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information is 0584-XXXX. The time required to complete this information collection is estimated to average 2 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.

1. FNS Tribal Feasibility Study: Overview for Interviewers

IMPAQ International, LLC, and its partners Bowman Performance Consulting and WMRA (collectively the IMPAQ team) are conducting a study to ascertain Indian Tribal Organizations’ (ITOs) interest in administering specific United States Department of Agriculture (USDA) Food and Nutrition Service (FNS) programs (National School Lunch Program/School Breakfast Program, Summer Food Service Program, and the Supplemental Nutrition Assistance Program, aka “food stamps”). This study is also gauging the feasibility of ITO administration, including any legislative or regulatory adjustments that may be needed. As part of this research, the IMPAQ team will conduct interviews with Tribal members and employees at up to 16 sites. An overview of the interviews to be conducted is presented below.

1.1 Purpose of the Interviews

Interviews with Tribal/ITO members and staff will be used in combination with information gathered through a survey that is being delivered to all Federally recognized Tribes (566). The survey will be available for completion on the web, by telephone, or in hard copy. The purpose of the interviews is to add depth to the information collected by the survey. With this information we will provide Congress with a report that indicates both the breadth and depth of ITO interest and capacity.

The survey and interviews are considering three dimensions of capacity:¹

• Leadership and Management: The capacity of leaders to prioritize, make decisions, provide directions, innovate, as well as monitor, assess, and respond to internal and external changes

• Operations and Workforce Capacity: The capacity of the ITO to ensure the effective and efficient use of resources

• Infrastructure: The capacity of the ITO to implement key organizational and programmatic functions

1.2 ITO Interviewees

It is expected that 8 ITO leaders/staff will be interviewed at each ITO, on average. This will include 2 leaders and 6 program staff. Program staff may be individuals from existing FNS programs administered by the ITO, or from other programs administered by the ITO (some sites will have large programs like TANF, some may only administer smaller programs).

1.3 Site Selection

We will be visiting a purposive sample of up to 16 sites. Sites will be selected based on several factors, one of which will be ITO experience administering programs. Our goal is to talk to individuals in ITOs with different levels of experience.

2. Interviewer Preparation

2.1 Scheduling

Once ITOs have been selected and agreed to participate, an IMPAQ team member will schedule and coordinate all site visits. This task will involve:

1. Assigning sites to site visit teams;

2. Identifying an appropriate on-site liaison to discuss potential interviewees and to assist in the scheduling of the visit;

3. Confirming basic information about the ITO, such as:

• Correct designation – is this a Tribe, a consortium, etc.,

• Which of the 4 FNS programs the ITO is interested in,

• Whether the ITO currently administers any other FNS programs (WIC, FDPIR, etc.),

• Whether the ITO currently operates any other FNS programs, and

• Whether the ITO currently administers any other benefits programs (such as TANF, Energy Assistance, HUD, etc.); and

4. Providing all site visit details to assigned team members.

2.2 Training and Materials

In-depth training for leader/staff interviews will occur approximately 2-4 weeks prior to the start of site visits. Training of all staff will take place in IMPAQ’s Columbia, MD office and occur over a period of 2 days. Detailed manuals and other relevant materials will be developed for use during training.

In addition to this guide, we will use diagrams to facilitate discussions about the state and local responsibilities involved with the FNS programs a) in general, and b) for each of the 4 specific programs. These diagrams are provided in Section 4.

In the remainder of this guide, italicized text indicates information for interviewers. Text that is not italicized is meant to be shared with respondents. While this guide is not intended to be used as a script, it is important that during a visit, researchers cover all of the information in the guide.

3. Interview Discussion Guide

3.1 Introduction/Purpose of the Study

Good morning/afternoon. My name is [name] and I’m from [company]. My role on the study is [role]. First let me begin by thanking you for letting us be a guest in your community and giving us an opportunity to visit with you today. [Other site visitor to introduce him/herself. Ask respondent to introduce anyone who has not been introduced: Name, role in the Tribe.]

IMPAQ International (Prime Contractor) is working with Bowman Performance Consulting (BPC), a Native American research company from Wisconsin [or WRMA from Washington DC] to find out two things:

1. Whether Tribes are interested in administering Food and Nutrition Service (FNS) programs

2. If interested Tribes have the capacity to do so –that is, whether Tribes have what they would need to meet the program requirements.

The United States Department of Agriculture Food and Nutrition Service (FNS) funded this study in response to Congress, as part of the USDA’s recent reauthorization of the Farm Bill. Information from our study will be provided to Congress in August 2015.

3.2 How We Selected Respondents

We are speaking with you today because you were identified as a good person to talk with about FNS programs and [name of Tribe/ITO]. With your permission, we would like to audio record our interview, to make sure we don’t miss any important information. Is that acceptable to you? [Thank respondent(s) if yes / No problem if no] Would you please tell us your name, your position in the Tribe, and share your background especially related to [FNS/other benefits] programs with the [name of Tribe/ITO]?

3.3 Purposes of Interview

For our interview today, we are asking about [number] FNS programs in particular:

• Supplemental Nutrition Assistance Program (SNAP) --formerly called food stamps or [Placeholder for local name of SNAP]-

• The National School Breakfast Program,

• The National School Lunch Program, and

• The Summer Food Service Program, which provides food to children when the school lunch and breakfast programs are not in operation.

We would like your input about the [Tribe’s/ITO’s²] interest in taking over the administration of these programs. We’re interested in your feedback in several areas, including: which of the program services, functions and activities interest the Tribe, and any challenges (including Federal regulations) you believe could impact the Tribe’s administration of these FNS programs.

Our visits with Tribes are to help us explain to Congress, from the perspective of Tribal members like you, what is needed to administer these programs –including recommendations to current policies and regulations that may hinder Tribal administration.

3.4 Transition & Check-In

We anticipate that it will take about 90 minutes for us to go through our interview questions. We have questions clustered around three main capacity areas: Leadership & Management, Operations & Workforce, and Infrastructure.

At any time please feel free to ask for clarification on the questions we have for you. Do you have any questions about the study or the interview process that we can answer for you now? [If yes, answer.] Are you ready to begin?

Thank you. Before we delve into our specific questions, we have found that learning about a Tribe’s history and how their Tribal governing structure was formed is helpful in gaining an understanding of how a Tribe operates. Could you tell us a bit about your Tribe’s history and how the government was formed?

3.5 General Interest in Administering FNS Nutrition Programs

[Distribute program illustration(s)] We would like to start by talking about the Tribe’s interest in administering these FNS programs. We thought it would be helpful if we all had the same picture in mind of what that administration looks like.

First, it will be important to clarify what we mean by program administration. For the purposes of this study, when we talk about administering specific FNS programs, this involves being responsible for the oversight, management, and monitoring of program activities. Program operation is specific to implementing day-to-day service delivery activities. This is an important distinction as many Tribes/ITOs may already operate some of the FNS programs we are going to discuss today but may not be involved in program administration.

Second, if a Tribe decided to take over administration, it might do so at the state level or at the local level.

For State administration of the nutrition programs³, there is a range of responsibilities for each program but those responsibilities generally fall into one of the following categories:

• Developing and distributing State programmatic policies and operations plans

• Providing guidance and technical assistance to local program administrators

• Maintaining and submitting records to FNS

• Instituting integrity and fraud monitoring efforts

• Overseeing local level program administration

At the local level of administration, responsibilities for the four programs generally fall into one of these categories:

• Processing applications and determining eligibility

• Providing information/education/outreach on the nutrition programs to potential applicants and recipients

• Procuring and/or delivering benefits

• Managing benefit recipient caseload

• Ensuring benefits are delivered according to Federal standards/requirements

• Maintaining program records and reporting to Federal/state entities as required

1. At which level --state or local-- do you see your Tribe implementing the nutrition program[s]?

2. Are there specific responsibilities from either the state and/or local level that you feel your Tribe would be interested in carrying out? If so, which responsibilities and why?

Repeat this discussion using the specific diagrams for the programs of interest to the Tribe.

3.6 Leadership & Management Capacity

Now we would like to talk about your Tribe’s experience planning and implementing programs from a leadership perspective. This is the first of the three capacity areas we mentioned earlier. It includes questions about your Tribe’s existing organizational structures, coordination with agencies within your Tribe, and approaches to programmatic planning

We understand from our phone conversation that the Tribe currently administers [List programs, FNS or otherwise]. We have a few questions about those programs. You can also tell us about any other programs like this that the Tribe administers now – or administered in the past.

1. Are these programs Federally-funded?

2. Are they run by the Tribal Government (as opposed to a TG Board like Tribal Clinics or Tribal Schools)?

3. Does the Tribe receive reimbursement from a Federally-funded source for services, programming resources, staff, and other expenses?

4. Do these programs have external monitoring and reporting requirements for programmatic and fiscal elements?

Regarding the [4] FNS Programs…

5. How has the Tribe discussed any interest in or goals for administering FNS program?

PROBES:

• When did the Tribe start talking about it?

• Who participates in these discussions?

• Can you tell us what some of the main points of the discussions have been?

• Have any major concerns or objections been raised during these discussions? If so, what are these concerns?

6. How do you think your community will respond to the idea of the Tribe administering FNS program(s)?

May we talk about the Tribe’s Strategic Plan?

7. How often is the Tribe's strategic plan updated?

8. Has the plan been modified before to take into account new programming or services? [If so] can you describe the new program or service and this modification process?

9. How is this plan communicated with the Tribe?

10. Is the strategic plan used to help guide daily decisions?

11. How often does the tribal leadership meet to formally discuss progress made on strategic goals?

12. Are their common indicators of success that the Tribe uses from the elected leadership down through program areas? Example: number of new jobs created number of newly trained staff, percent increase in budgets, level of tribal member satisfaction, etc.

13. How do you measure progress of the Tribe in meeting strategic goals? Thinking back to the programs that you told us about administering [discussed earlier], how do you measure progress of departments or programs in meeting strategic goals? Do you require all programs to contribute to the Tribe’s annual report? [If so, request copy.]

14. Can you talk about how administering the FNS programs would fit into the Strategic Plan?

15. How do you think the Tribe would meet the demands of FNS program administration? We are thinking about growing existing infrastructure and resources.

PROBES:

• Can you describe the existing connections and/or relationships that the Tribe has with other State or Federal departments?

• Which State departments do you have experience with in terms of program implementation or administration?

16. Can you discuss the challenges you have experienced in implementing other federal/state programs such as the Food Distribution Program on Indian Reservations (FDPIR) program or Temporary Assistance for Needy Families (TANF), and how you have handled these challenges?

17. How often do levels of Tribal government and the workforce meet to address performance data, or achievement of short and long-term goals?

PROBES:

• What data does the Tribe regularly collect? [E.g., census-type data, school data, resource-use data.]

• Who collects it?

• How frequently?

• How is it shared with other decision-makers?

• [If experienced with FNS programs] Does your organization have experience using data to inform decisions related to FNS programs?

18. Is there anything else that you think we should understand about the Tribe administering FNS program(s)?

Now we have some questions about how your Tribe has identified the need for and developed and adjusted in order to implement new programs.

19. Tell us about the Tribe’s past experience with starting up new programs or expanding an existing one.

PROBES:

• What conversations or processes went into deciding to start or expand a new program?

• How would the Tribe go about investigating if they were ready to take over an FNS program? What would you do, who would you talk to, what would have to be in place?

20. Based on your previous answer, tell us about the details of that planning, development or expansion and implementation process.

PROBES:

• Please provide details on the adoption and implementation process.

• What was the annual program budget (approximately) and how many staff were employed under the program?

• What were some of the strengths of the implementation process for the new (or expanded) program?

• What were lessons learned?

• What would you do different next time? Does your Tribe have the capacity to make those adjustments in the future?

21. How does the Tribe measure and monitor whether programs are being implemented according to institutional requirements?

PROBES:

• Does your Tribe have experience conducting administrative reviews of programs to ensure compliance with institutional and/or Federal programmatic requirements?

• Can you walk us through a recent review that the Tribe has done for one of the programs it administers?

• How often is this kind of review done?

• How are non-compliance issues handled? (For example a Process Improvement Plan?)

22. Do you have internal audit staff? If so, how are Tribal audits performed?

23. When planning across programs or departments, how does the Tribe communicate and share data/information (such as demographics, quarterly outcomes, etc.)? Can you provide an example of how the Tribe has shared information across programs/departments?

PROBES:

• What network or database does the Tribe currently use internally?

• Specifically how does the Tribe share data internally and how often?

24. What are the communication strengths and challenges that the Tribal departments/programs experience when they share information? What resources or capacities would be needed to improve this?

25. Does the Tribe have a shared network or other “common database” where all programs/departments can enter, share, have access to, and retrieve data? If yes, please explain. If not, where are centralized data stored?

26. When new Tribal employees and leaders join the government workforce, how do they learn about the communication and data systems of the Tribe?

27. What are the communication strengths and challenges that the Tribe encounters when they share information externally to other agencies? What resources or capacities would be needed to improve this?

28. When we were talking about Leadership earlier, we discussed how data informs the Tribe’s Strategic Plan. We also would like to know about how the Tribe uses data on a more ongoing basis -- for Tribal initiatives or program planning. When Tribal employees or leadership use data, how do you know it is up to date and accurate? What is the process for ensuring updated information is available?

29. Please share an example of how the Tribe uses data for program planning, decision making, or evaluating whether strategic direction, impact, or intended outcomes have been met?

30. Please tell us how often tribal council, division or department leaders (senior administration/management), and program managers (mid-level) meet formally to discuss the Tribes programs, operations, and initiatives.

PROBES:

• Are there typed notes/agendas?

• Do tasks get assigned & are deadlines set?

• How is information or tasks from one meeting followed up on for the next meeting?

Do you have questions for us so far?

3.7 Operations and Workforce Capacity

Now we have some questions about the how your Tribe/ITO operates and utilizes resources. This is the second of the three capacity areas. These questions will address your organization’s existing management approach, organizational structure, staffing, and other resources, in the context of administering an FNS program. Your responses will help us make sure we’re not overlooking any capacity or assets that the Tribe is already using.

1. In thinking about administering a new program, what are some of the strengths and skills of your Tribe’s current staff?

2. Are there things about your workforce that you would like to improve on?

3. Can you describe the range of positions for which the Tribe hires staff? What efforts do these staff support?

4. Generally, how does the Tribe identify and hire qualified staff members? Can you describe the processes and procedures involved?

5. Does the Tribe currently employ an adequate number of staff with the needed skills and abilities to implement existing policies, programs, services, etc.?

[If yes] How have you worked to staff your Tribe to work at this capacity?

[If not] What has prevented your Tribe from hiring a sufficient number of qualified staff?

6. Can you tell me about how many people the Tribe currently employs to administer any public assistance programs? About how many years’ experience do these people typically have?

PROBES:

• Can you tell us about a recent experience hiring staff?

• Has your Tribe encountered challenges in recruiting and retaining qualified workers to administer current programs?

• Do you have any staff or recruitment incentive programs?

7. Right now, there are rules that require staff performing specific tasks under certain FNS programs to be Federal or state employees or Merit employees. Do you currently have any Federal or state employees supporting programs your ITO administers?

8. What types of IT personnel does your Tribe employ? How many IT employees do you employ?

PROBES:

• What certifications do they have?

• What types of IT systems do they currently work with?

• What types of tasks do they perform daily/on a regular basis?

9. How does your Tribe ensure that IT staff have adequate skills and abilities to perform all required tasks?

PROBE: What types of training/professional development resources are available to your IT staff?

10. Do you see any other operational, capacity, workforce or other resource barriers to the Tribe administering their own FNS program(s)? If so, please explain and provide details about what additional support would be needed.

[For Tribes with existing FNS experience]

11. Thinking about your [existing FNS program], can you tell us about the staff training that it involved?

PROBES:

• Multiple days?

• Onsite/offsite?

• By State people/Contractors/Feds/Staff from Experienced Tribes?

12. What type of feedback did you receive on the trainings?

PROBES:

• What parts do you think were successful?

• Were there areas where that staff wanted more information?

• Do you know if there are refresher trainings available for staff?

13. How do you keep track of who needs training?

[For Tribes currently running other large-scale programs –such as TANF, Energy Assistance, HUD programs, etc.]

We’d like to talk for a minute about how you approach employee reviews/evaluation.

14. Can you describe your Tribe’s employee review process?

PROBES:

• When and how often are these reviews conducted?

• Do you provide any “extra incentives” for good performance, or improvement?

• Do you have a blank employee evaluation form that you could share with us?

• How do professional development trainings fit into this process?

15. Is there anything else you would like to tell me about Tribal personnel?

Next, I have a few questions about the way management communicates.

16. Can you tell me how program management and staff communicate within your existing programs?

PROBES:

• How involved is leadership? What role does your leadership play in this process?

• How often are you updated?

• How are you updated [meetings, reports, 1-1 informal check-ins]?

17. Are there any Tribal policies about communication that are typically put in place at the beginning of a new program? Any program-specific communication policies that are common?

18. Can you provide examples of any Tribal operating manual, or guidelines, or policies related to a service?

19. [If they have an example] Can you tell me about the process of developing and sharing that [guide/policy/manual]?

PROBES:

• How long did it take?

• What people or offices were involved?

• How did you send it out when it was done? (Hard copy, electronically).

• Where does it ‘live’ now –that is, where do people go if they need to see it?

• How often is it reviewed or updated?

3.8 Infrastructure Capacity

Now we would like to talk about your Tribe’s existing assets, policies, and procedures that may help facilitate the daily operations involved in administering FNS nutrition programs. This is the last of the three capacity areas and includes questions about your organization’s existing physical and systemic infrastructure, financial resources, and professional staff. Your responses will help us gain an understanding of what aspects of the FNS nutrition programs your organization could implement at this point in time, as well as what resources your Tribe would require to expand its capacity to implement these programs.

We would like to start out by talking about your organization’s physical infrastructure.

1. Please describe the physical infrastructure of your organization? This can include physical office space, IT equipment (computers, copiers, printers, scanners) and vehicles.

[For sites w/ FNS knowledge/experience]

2. If you were to consider the implementation of the [name FNS program(s)], would you expect that your Tribe could meet the physical infrastructure capacities? If so, please explain. If not what additional physical resources/capacities do you think would be needed?

PROBES:

• Can you describe the physical buildings and/or office space your organization has access to?

• What types of vehicles does your organization have access to?

• Can you describe your organizations IT infrastructure? What are the primary functions of the IT system?

• Are limitations in infrastructure (physical buildings, office space, computers, copiers, printers, vehicles, etc.) often barriers for getting work done successfully?

• In the past how has your organization expanded physical infrastructure capacities (space, vehicles, computers, etc.) when it has been required?

3. Do you operate any programs that require determining whether program applicants are eligible to receive a specific set of benefits?

[If so] PROBES:

• Is this process automated? If yes, how often do you have to update this automation?

• What processes and procedures are used to verify the accuracy of applicant data?

• What processes and procedures are used to determine eligibility?

• What processes and procedures are used to maintain applicant files?

• What processes and procedures are in place for client notification?

• What processes and procedures are in place for recertification?

3. Do you have experience in procuring technology services from a third-party vendor?

[If so] PROBES:

• How did you solicit and obtain these services?

• What has been your experience working with these vendors?

• How do you monitor the quality of the services that they provide?

3. Do you operate a program that requires regular submission of reports to either federal or state agency?

[If so] PROBES:

• What types of reports are prepared and submitted? And to whom?

• What different types of information do these reports provide?

• How are these reports prepared? Who prepares them?

• How has your organization worked to meet the different reporting requirements of different organizations/agencies?

3. Can you describe the current capacity of your organization’s technology or IT systems for collecting data and reporting (internal or external)?

PROBES:

• If you are familiar with the requirements related to administering and implementing [FNS program(s) of interest], do you feel that the Tribe’s technology/IT system for collecting and reporting data meets these requirements? [If yes] Please explain.

• What security systems are in place to protect client data?

• What back-up processes are in place to store client data?

• [If no] What added system upgrades, equipment, software, hardware, or IT staff development might you need?

3. Would you please describe your organization’s financial governance policies? How often do you monitor compliance with these policies?

3. Would you please describe your organization’s IT governance policies? How often do you monitor compliance with these policies?

3. Would you please describe your organization’s processes and procedures for maintaining records and sharing data?

PROBES:

• How does the Tribe maintain administrative and service records? How does your organization plan and prepare for audits?

• How does the Tribe manage the disclosure of data that contains personally identifiable data?

3. Would the [FNS program(s)] pose any issues or strain on your financial or accounting departments with regard to a new reimbursement process?

[If not] Please explain how the Tribe could currently meet the administration requirements related to implementing a new reimbursement process.

[If yes] Please explain what additional resources/capacities you would need to develop.

3. If federal financial support cannot fully fund expansion in operational and human resources, how might the Tribe plan to finance the necessary expansion?

PROBES:

• Does the Tribe have existing funds/resources that could support expansion in these areas? If so, how would these resources be accessed? Are there restrictions in the use of these funds?

• How, if at all, has the Tribe funded (at least in part) the startup or expansion of other programs in the past?

• Would the Tribe be able to leverage activities performed under other programs in order to meet the requirements of FNS programs? For example, your organization may already provide nutrition education programming that includes food delivery inspection, etc.

3.9 Wrap Up

Those are all the questions that we have for you today. Thinking of all the topics we covered, across the different capacity areas, is there anything else you would like to share?

Thank you for taking the time to speak with us. This discussion has been very informative and helpful. If you think of any additional information you would like for us to know, please feel free to contact me directly. [Give business card] If I have follow-up or clarifying questions, may I contact you?

4. Program Diagrams

1 Adapted from the Marguerite Casey Foundation’s Organizational Capacity Assessment Tool. Available at: http://caseygrants.org/resources/org-capacity-assessment/

2 The rest of the guide will refer to Tribes; however, you may be interviewing members of an ITO. In that case you will need to name the ITO/consortium appropriately.

3 These examples are from the general diagram, which should be used to introduce the levels of responsibility to Tribes that are interested in multiple programs. If a Tribe is interested in a single program, interviewers should proceed directly to that diagram.