State Agencies (State QC directors, QC supervisors and QC reviewers)

U.S. Department of Agriculture

Food and Nutrition Service

Enhancing Completion Rates for SNAP

(Supplemental Nutrition Assistance Program) Quality Control Reviews

Request for Clearance

Supporting Statement and

Data Collection Instruments

Attachment A-4A:

State QC Reviewer (SQCR) Semi-Structured Interview

Draft Protocol

Project Officer: Robert Dalrymple

September 26, 2013

OMB Control Number: 0584-XXXX
Expiration Date: XX/XX/XXXX

State QC Reviewer (SQCR) Semi-Structured Interview
Draft Protocol

Introduction: Thank you for taking the time to talk with me today. I’d like to tell you a little bit about why we are doing these interviews. We are talking to State agencies that conduct SNAP Quality Control (QC) reviews to help FNS understand the factors that lead to incomplete SNAP QC cases and recommend ways to improve case completion rates in the future. In particular, we are interested in learning more about the incomplete cases, not the cases deemed Not Subject to Review. I will be asking you some questions to assess the process. We expect this interview to take about an hour to complete. We have a lot to talk about, so I’m going to list for you the different sections of this interview. We’ll talk about your experiences as a QC reviewer, the SNAP QC caseload, the SNAP QC review procedures in your State, and then we’ll go into some details about incomplete cases. After that, we’ll talk about your State’s relationship with the Regional office, tools and materials you use during your reviews, as well as the training you received, and then we’ll talk briefly about challenges and solutions involved in the QC process. We’ll wrap up the interview with a short questionnaire about attitudes related to SNAP QC completion rates. If it’s okay with you, I would like to record the interview so that I don’t miss anything. Is that all right?

Do you have any questions for me before we begin?

A. EXPERIENCES AS A QC REVIEWER

First, I’d like to talk about your experiences working in this position.

• How long have you been a QC reviewer for the SNAP program?

  • Do you perform other SNAP work? If so, what?

• Do you have prior experience in the SNAP program other than SNAP QC? If so, how long?

• Do you conduct QC reviews for any other programs besides SNAP? Please explain. If with other programs, what percent of your time is spent in each program?

B. SNAP QC CASELOAD

Next, I’d like to talk about your typical caseload of SNAP QC cases to review.

Interviewer note: On the occasion that you interview a reviewer who has multiple responsibilities at the QC office (i.e. second level analyst, or other position that is not strictly a reviewer), please indicate that for these questions, we are only interested responses that pertain to SQCR responsibilities.

• Approximately how many SNAP QC cases were assigned to you last month?

• Of these reviews, how many were:

___ [Enter Number] Active

___ [Enter Number] Negative

___ [Enter Number] Other types of reviews (other than SNAP QC)

• Has that number increased over the last 5 years, decreased, stayed the same, or does it change month to month?

• Are you ever assigned additional reviews to assist backlogged reviewers or reviewers on extended leave? If so, are the reviews fairly distributed? How frequently are you assigned additional reviews?

• If not already answered: Do you ever provide a second review of a case already reviewed by a fellow reviewer? Does anyone else review your cases once you’ve completed your review? How often?

• Do you typically receive all of the information you need to conduct the review? (E.g., the full certification case file, verification documents, etc.)

• [If no]: What information are you missing?

• Do you have sufficient time to complete the reviews? Why or why not?

• How close do you live to the SNAP QC cases assigned to you to review?

• In geographic terms, what is the type of population you most often work with (i.e., urban, rural, suburban, mix)?

• How much of your caseload is homeless?

• How many days do you travel each month on SNAP QC reviews that require you to stay away from home?

• Do you conduct SNAP QC reviews in languages other than English? What languages? If not, how do you deal with the need to conduct the QC reviews in languages other than English?

• What times of day and days of the week do you typically work?

C. STATE SNAP QC REVIEW PROCEDURES

Now, I’d like to get some general information about the procedures you use to conduct SNAP QC reviews.

• How many days do you have to review a case and submit it as final?

• What happens if it is not completed in that time?

• Are you given any interim deadlines throughout the process that you are required to meet? If so, what are these deadlines?

• Do you have sufficient time to complete the reviews? Why or why not?

• On average, how long does it take to conduct a SNAP QC review (from the time a case is assigned to you to the time that you are done and no longer responsible for it)?

• Has this time increased, decreased, or stayed the same over time? Why?

• Are the steps you take to conduct SNAP QC reviews the same steps outlined by the FNS 310 Handbook or do they vary? Do you have the flexibility to go beyond the steps outlined in the 310?

• How do your safety concerns play into contact and interview procedures? Do SNAP households express concerns about safety that may affect the review process?

• What problems do you encounter conducting SNAP QC reviews?

Now, I’d like to talk about the procedures you use to locate and contact households.

• What are the general steps you take to locate households selected for review?

• Have these steps changed over time?

• How many attempts do you make to locate a household?

• On average, how long does it take to locate a household?

  • Has this time increased, decreased, or stayed the same since you have been doing SNAP QC reviews? Why?

• What is the most effective strategy for locating households?

• What problems do you have locating households?

• What are the general steps you take to contact households once they’ve been located?

• Have these steps changed over time?

• On average, how long does it take to contact a household?

  • Has this time increased, decreased, or stayed the same since you have been doing SNAP QC reviews? Why?

• What percentage of QC review cases requires 1-2 contact attempts to get the information you need to complete the review? What percentage requires 3-4 contact attempts? 5-6 contact attempts. 7 or more contact attempts?

  • What is the minimum number of contact attempts that are required? Why?

  • What is the maximum number of contact attempts that are required? Why?

• What is the most effective strategy for contacting households?

• What problems do you have contacting households?

• Are there any other steps that you take (outside of the ones just mentioned) that you use to locate and contact households?

Now, I’d like to get some general information about how you ensure that households cooperate.

• What strategies are used to convince households to cooperate with the review? What is the most effective strategy? (Probe if necessary: Offer flexible times to meet, including evenings and weekends; offer alternative locations to meet; notify the household that failure to cooperate could affect their benefits; enlist the help of the caseworker; offer to help with childcare)

• What do you think are the reasons why households fail to cooperate? Just to clarify, I’m specifically interested in households that fail to cooperate, not those that refuse to cooperate.

• Has the number who fails to cooperate increased, decreased, or stayed pretty constant since you’ve been doing SNAP QC reviews? [If increased, why do you think that is the case?]

• What about the cases that refuse to cooperate—have they increased, decreased, or stayed pretty constant since you’ve been doing SNAP QC reviews? [If increased, why do you think that is the case?]

• What steps do you take if a client fails to keep an interview appointment?

• How often are you successful in turning missed appointments into completed reviews?

Now, I’d like to get some general information about the field interview procedures.

• Once the household agrees to an interview, what are the general steps taken to conduct the field interview? (Probe if necessary: Where do you typically meet? How long does it take? What questions do you ask? What documentation do you request? Any other steps?)

• [If not already answered:] Where does the interview typically take place? Is the location ever changed at the request of the household?

• Are the same steps taken for every case?

• Have these steps changed since you’ve been a QC reviewer? How? Why?

• Are there certain times of day that are better than others for making review appointments with clients? What about days of the week?

• Are there part(s) of the review that are more challenging than others to complete? Why?

D. INCOMPLETE CASES

Now, I’d like to get some information about ‘incomplete’ SNAP QC cases. For these questions, we’re interested in only the incompletes and not those disposed of as Not Subject to Review.

• On average, how many SNAP QC cases do you code each month as incomplete?

• Has that number increased, decreased, or stayed the same since you have been doing SNAP QC reviews? Why?

• What are the main reasons that cases are coded as incomplete? (Probe if necessary: Case file record could not be found? Household cannot be located? Failure to cooperate? Refusal to cooperate? Unable to arrive at likely conclusion? Other?)

• Approximately how many incomplete cases fit each of those reasons?

• Have the reasons changed over time? If so, how?

• How often do you use likely conclusion to complete a case?

• What are the general steps taken before designating a case as “incomplete”? (Probe if necessary: How many contact attempts are made? How many collateral contacts are pursued? How is the decision documented? Use “likely conclusion”? Other steps?)

• Have these steps changed over time? How? Why?

• [If not already answered:] How many attempts do you typically make to contact the household in-person or by phone, mail, or email, before it is classified as incomplete?

• Has this changed over time? How? Why?

• What are the most effective methods in tracking down potential incomplete cases? What about encouraging those individuals that refuse an interview?

• Are you given any incentives to have high completion rates? Any penalties for low completion rates?

E. RELATIONSHIP WITH REGION

Now, I’d like to get some additional information about your office’s interaction with the regional office that reviews your State’s SNAP QC reviews.

• How would you describe the working relationship between the Federal and State QC review offices?

• Who in the regional office is the point of contact with your State? Who is the point of contact in your office? (E.g. Director only? Supervisor?)

• How much interaction do you have with the regional office that reviews your State’s QC reviews? What’s the nature of that interaction?

• What sort of feedback do you receive on your cases, particularly the incomplete cases?

• How useful is this feedback?

• Have you made changes to your review process in response to feedback from Federal QC?

• Is there anything else you’d like to tell us about your office’s interaction with the regional office?

F. TOOLS AND MATERIALS

Next, I have a few questions on the tools and materials you use when conducting SNAP QC reviews.

• Do you use an electronic or automated 380? Electronic or automated comp sheet? If so, what effect, if any, has it had on your ability to conduct and complete SNAP QC reviews?

• Do you use the FNS 310 Handbook? Why or why not?

• What do you use it for?

• How effective and useful is it? Why or why not?

• What could be done to improve it?

• Do you use any State-specific materials or other supplements to the FNS 310 Handbook, such as guidance on State policy? Why or why not?

• What do you use them for?

• How effective and useful are these materials? Why or why not?

• What could be done to improve them?

• [If not already answered:] What guidance is offered on State policies regarding eligibility and allotment determination?

• What other tools do you use to complete cases (i.e., forms, databases, computer systems, other materials, etc.)?

• Do all SNAP QC reviewers in your State use these materials?

• Are there any additional tools or materials that would help you to be more effective?

G. TRAINING

Next, I’d like to ask you about any training you have received for doing SNAP QC reviews. Please limit your responses to just the training you have received for SNAP.

• What training do you receive to conduct a SNAP QC review?

• What topics are covered in this training? Certification? How to locate and contact a household? How to locate and work with collateral contacts? How to conduct the field interview? How and when to code a case as incomplete? Use of “likely conclusion”? Something else?

• Has that changed since you’ve been doing SNAP QC reviews?

• How often do you receive training? Do you receive regular refresher training? Annually?

• Has that changed since you’ve been doing SNAP QC reviews?

• Who conducts the training? (E.g. Senior Staff? Designated trainer? Contractor? Regional SNAP office?)

• What is the format of the training? (E.g. Online? Instructor-led? Both?)

• [If not already answered:] How do you learn about changes in Federal or State policy that affect eligibility or allotment determination in the cases you review?

• Do you think your SNAP QC training is effective? Is there anything that could be done to improve the training?

• From the following list, please tell me whether additional training on each topic would be helpful.

• SNAP eligibility

• Procedural aspects of QC review

• Interview techniques

• Household location techniques

• State-specific policy, including options and waivers

• “Likely conclusion” as a means of case completion

• Other, please explain

H. CHALLENGES AND SOLUTIONS

Finally, I’d like to ask your opinion on some things related to SNAP QC reviews.

• What are the most common types of problems that you face when conducting SNAP QC reviews?

• How can these problems be minimized or solved?

• What strategies, tools, or techniques do you use in your State that have been effective in ensuring that as many reviews as possible are complete?

• [For MS, OK] Why do you think your State’s completion rates are higher than some other States’? Do you have recommendations for how those States might improve their completion rates?

• [For KY] Your completion rates have risen substantially in recent years. What has contributed to this increase? Do you have recommendations for other States who are trying to improve their rates?

• [For IA, MI, PA] Why do you think your State’s completion rates are lower than some other States’? What challenges has your State faced in attaining higher completion rates?

• Do you have any suggestions on how your current procedures could be adjusted to improve completion rates?

• Could FNS or the Regional Office assist in making any changes?

• What additional tools might help boost completion rates? (Probe if needed: Skip tracing, Background checks)

• Are there changes that could be made to improve the efficiency of the QC review process? (E.g. things that could be automated?)

• Is there anything else that you’d like to share about the SNAP QC review process?

• Is there anything else that you’d like to share about factors contributing to incomplete SNAP QC reviews?

I. ATTITUDES

That concludes our interview. However, we have a very brief questionnaire about attitudes related to SNAP QC review completion rates that should take only 1-3 minutes to complete. Would you be willing to complete this questionnaire and return it to us? (Provide paper or email questionnaire.)

Please circle the number below that best represents you opinion on each statement, where 1 = strongly agree, 2 = agree, 3 = disagree, and 4 = strongly disagree.

I am personally held accountable when I have low completion rates for SNAP QC reviews.	1	2	3	4
The reputation of my office is negatively affected by low completion rates for SNAP QC reviews.	1	2	3	4
Obtaining high completion rates for SNAP QC reviews is a priority for my office.	1	2	3	4
Determining over- or under- issuance of benefits is a priority for my office.	1	2	3	4
I go the extra mile to complete SNAP QC reviews.	1	2	3	4
During the SNAP QC review process, I make concerted efforts to speak with collateral contacts of a client who is uncooperative or hard to locate.	1	2	3	4
During the SNAP QC review process, I come up with creative solutions to locate hard-to-reach clients.	1	2	3	4
During the SNAP QC review process, I come up with creative solutions to convert uncooperative clients.	1	2	3	4
My office environment and managers encourage sharing information about new approaches with other SNAP QC reviewers.	1	2	3	4
Most clients interviewed during the SNAP QC process are honest with information needed for their case review.	1	2	3	4
I enjoy my work.	1	2	3	4
I work hard.	1	2	3	4
I feel supported by upper management.	1	2	3	4
There is a high rate of turnover among SNAP QC staff.	1	2	3	4

Thank you for your time today.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB number. The valid OMB control number for this information collection is 0584-XXXX. The time required to complete this information collection is estimated to average 1 hour per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.