State Agencies (State QC directors, QC supervisors and QC reviewers)

U.S. Department of Agriculture

Food and Nutrition Service

Enhancing Completion Rates for SNAP

(Supplemental Nutrition Assistance Program) Quality Control Reviews

Request for Clearance

Supporting Statement and

Data Collection Instruments

Attachment A-3B:

State QC Director and State QC Supervisor

Web and Telephone Survey

Project Officer: Robert Dalrymple

September 26, 2013

OMB Control Number: 0584-XXXX
Expiration Date: XX/XX/XXXX

State QC Director and State QC Supervisor

Web and Telephone Survey

INTRO 1: Hello, my name is [NAME]. I’m calling on behalf of the Food and Nutrition Service of the US Department of Agriculture. May I please speak with [INSERT NAME FROM SAMPLE]?

• 01 – Yes, selected respondent available (Skip to Intro2)

• 02 – No, selected respondent not available (Continue to Exit)

EXIT: Thank you, I will call back later. When would be a good time to reach _______?

INTRO 2: FNS is conducting a study of State agencies that conduct the SNAP Quality Control (QC) reviews to help understand the factors that lead to incomplete SNAP QC cases and recommend ways to improve completion rates in the future. In particular, this study focuses on incomplete cases, not cases deemed Not Subject to Review. We understand you are a SNAP QC [INSERT “Director” or “Supervisor”] and would like to ask you some questions about the SNAP QC review process. This survey will take about 30 minutes to complete. The survey format includes multiple choice and short answer questions to keep the survey as brief as possible. There is an opportunity at the end of the survey to expand on responses that cannot be summarized through the multiple-choice options. Everything we talk about will be private, although this call may be monitored and recorded for quality assurance and transcription purposes. Your responses, which will be reported only in summary form in combination with those of other SNAP QC [INSERT “Directors” or “Supervisors”], will help FNS improve the SNAP QC process.

If you have any questions about the survey, you may contact the Project Officer, Bob Dalrymple, at FNS. Call 703-305-2122 or email [email protected].

• 01 – Continue

• 02 – Not a SNAP QC Director or Supervisor [Terminate]

• 99 – Refused [Terminate]

TERMINATE: Thank you for your time.

A. EXPERIENCES AS A QC DIRECTOR/SUPERVISOR

Thank you for taking the time to talk with me. First, I have some questions about your experiences as a QC [INSERT “Director” OR “Supervisor”].

1. How long have you worked in Quality Control for the SNAP program?

Years Months

2. Do you have prior experience in the SNAP program other than SNAP QC?

• No (Skip to Q3)

• Yes

2a. How long have you worked in other areas of the SNAP program?

Years Months

3. Do you work entirely with the SNAP QC program or do you also work with other programs besides SNAP?

• I only work with SNAP QC (Skip to Q4)

• I work with other programs as well

  • What percent of your time is spent in SNAP QC?

% [Enter Percent]

B. SNAP QC CASELOAD

Next, I’d like to talk about how you assign SNAP QC cases to reviewers.

4. How many SNAP QC reviewers do you have working during any given month?

[Enter Number]

5. Has that number changed over the last 5 years? Would you say ?

• No, it has not changed. (Skip to Q6)

• Yes, it has increased.

• Yes, it has decreased.

5a. Why has the number of SNAP QC reviewers changed? Please select all that apply from the following list.

• We saw an increase in reviewers due to more funding for staffing.

• We saw a decrease in reviewers due to less funding for staffing.

• We saw a decrease in reviewers due to high staff turnover and therefore fewer trained people were available to work.

• We saw an increase in reviewers because we were able to use QC reviewers from other programs to complete SNAP QC reviews.

• Other

6. Approximately, how many SNAP QC cases were assigned to any one reviewer last month?

___ [Enter Number]

6a. Of those cases, how many were:

[Enter Number] Active

[Enter Number] Negative

[Enter Number] Other types of reviews (other than SNAP QC)

7. Do you ever assign additional SNAP QC reviews to your staff to assist backlogged reviewers or reviewers on extended leave?

• No

• Yes

8. Do you have reviewers who are specialized in SNAP cases with certain characteristics like earnings, large household sizes, languages, or certain geographic areas?

• No (Skip to Q9)

• Yes

8a. What are those specialty areas? Select all that apply.

• Earnings

• Large household sizes

• Non-English language

• Certain geographic areas

• Other

For the following question, please tell me how much you agree or disagree with the following statement:

9. There is currently enough staff to maintain QC quality, e.g. unbiased reviews, accuracy, thoroughness of procedures…. Would you say that you___?

• Strongly agree

• Agree

• Disagree

• Strongly disagree

C. TRAINING AND TOOLS

C1. Training

Next, I’d like to ask some questions about the training and tools that your QC reviewers receive.

10. Does your staff receive any training on how to conduct a SNAP QC review?

• No (Skip to Q11)

• Yes

10a. From the following list, please tell me whether the following topics are covered during training:

• SNAP eligibility (Y/N)

• Procedural aspects of QC review (Y/N)

• Interview techniques (Y/N)

• Household location techniques (Y/N)

• State-specific policy, including options and waivers (Y/N)

• Likely conclusion as means of case completion (Y/N)

• Other (Y/N)

10b. From the following list, please tell me the format of the training.

• Formal in-person training (Y/N)

• Online independent tutorial or module (Y/N)

• Online group webinar (Y/N)

• Informal meetings, such as a staff meeting (Y/N)

• Peer mentoring (Y/N)

• Written materials for individual study (Y/N)

• Other (Y/N)

11. Who leads the instructor-led trainings?

• A senior staff member from the State QC office (Y/N)

• A peer QC reviewer from the State office (Y/N)

• A QC reviewer from the FNS Regional office (Y/N)

• A QC director or supervisor from the FNS Regional office (Y/N)

• A Contractor (Y/N)

• No instructor-led training

12. Who is responsible for planning training?

• I am

• A designated person from the State office

• A designated person from the FNS Regional office

• Other

13. How frequently do SNAP QC reviewers receive training? Would you say___?

• Training happens once, when they start the job

• Training is ongoing and is conducted weekly, monthly or semi-monthly

• Training is ongoing and is conducted twice a year

• Training is ongoing and is conducted annually

• Training is conducted on an as-needed basis with no set schedule

14. Has the amount of training changed over time?

• No

• Yes, it has increased

• Yes, it has decreased

15. Have you noticed a change in the quantity or quality of training as a result of budgeting cutbacks?

• Yes. There has been a decrease in the quality of training.

• Yes. There has been a decrease in the quantity of training.

• Yes. There has been a decrease in both the quantity and quality of training.

• No. The quality and quantity of training has not changed as a result of budget cutbacks.

16. From the following list, please tell me how SNAP QC reviewers are trained or alerted when there is a change in Federal or State policy that affects SNAP eligibility or allotment determination? Please select all that apply.

• Email alert (Y/N)

• State manual page change (Y/N)

• Memo (Y/N)

• Formal training/meeting with staff (Y/N)

• Conference call (Y/N)

• Other (Y/N)

• They are not notified

17. Who is responsible for notifying SNAP QC staff of changes in Federal or State policy that affect eligibility or allotment determination?

• I am

• The State QC Director

• A State QC supervisor

• A State worker from certification policy

• A representative from the FNS Regional office

• No one

18. Do you compare your State regulations with Federal regulations to ensure the State policy is correct, taking into account the options and waivers your State has chosen?

• No

• Yes

C2. Tools

Now I’m going to ask you about the tools and materials reviewers use when conducting SNAP QC reviews.

19. Do all your SNAP QC reviewers have access to a copy of the most recent version of the FNS 310 Handbook?

• No (Skip to Q20)

• Yes

19a. Is that copy of the 310 paper, electronic, or both?

• Paper

• Electronic

• Both

20. Does your State have its own QC review materials or other supplement to the FNS 310 Handbook, such as guidance on State policies?

• No (Skip to Q21)

• Yes

20a. Do all your reviewers have a copy of these materials?

• No

• Yes

20b. Are those materials paper, electronic, or both?

• Paper

• Electronic

• Both

21. Are there additional tools or materials (either paper or electronic) available to SNAP QC reviewers when they are conducting the reviews?

• No

• Yes

22. Does your staff use an automated or electronic 380?

• No (Skip to Q23)

• Yes

22a. How has the automated 380 affected your staff’s ability to complete reviews?

• The process is easier

• The process is about the same

• The process is more difficult

D. STATE LEVEL QC REVIEW PROCEDURES

D1. Ease of Review

Next, I’d like to ask some questions about the QC review experiences of your reviewers. From this point on, we are interested only in ACTIVE cases, not negative QC cases.

For the following statements, please tell me how much you agree or disagree with the following statements.

23. My SNAP QC staff has sufficient time to complete the reviews assigned to them. Would you say you ___?

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

24. My SNAP QC staff typically receives from the SNAP office all the information they need to conduct their reviews, including all of the case records and certification information. Would you say you___?

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

25. My SNAP QC reviewers have sufficient resources to conduct reviews in languages other than English. Would you say you___?

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

D2. PROCEDURES

Next, I’d like to get some general information about the procedures your SNAP QC staff uses to locate households and review cases.

26. On average, how long does it take to locate and contact a household?

Days

27. On average, how long does it take to complete a household interview once the client agrees to participate?

Days

28. On average, how long does it take to conduct an entire QC review from the time the case is assigned to the time of disposition?

Days

29. Do all reviewers follow the same procedures to conduct SNAP QC reviews?

• Yes

• No

Now I’d like to talk about strategies reviewers may use to ensure households cooperate with the review.

30. What is the most effective strategy for ensuring households cooperate with the review?

• Offering flexible times to meet with households, including evenings and weekends

• Offering alternative locations to meet with households

• Sending notifications to the households that failure to cooperate could affect their benefits

• Offering to help with childcare during an interview

• Something else

30a. For what percentage of cases is this strategy successful?

%

31. What is the second most effective strategy for ensuring households cooperate with the review?

• Offering flexible times to meet with households, including evenings and weekends

• Offering alternative locations to meet with households

• Sending notifications to the households that failure to cooperate could affect their benefits

• Offering to help with childcare during an interview

• Something else

31a. For what percentage of cases is this strategy successful?

%

32. What percentage of SNAP QC cases refuses to cooperate? (Note that we are talking specifically about refusals, not failure to cooperate.)

%

32a. Has that percentage increased since you’ve been doing QC reviews, decreased, or stayed about the same?

• Increased

• Decreased

• Stayed about the same

33. What percentage of cases fails to cooperate? (Note that we are not talking about refusals.)

%

33a. Has that percentage increased over time, decreased, or stayed about the same?

• Increased

• Decreased

• Stayed about the same

34. Has your State ever engaged an outside party (e.g. consultant) to review QC policies and procedures and recommend changes?

• No

• Yes

34a. How effective was this?

• Very effective

• Somewhat effective

• Only a little effective

• Not at all effective

E. INCOMPLETE CASES

Now I’d like to ask a few questions about incomplete SNAP QC cases. For these questions, we’re interested in only the incompletes and not those disposed of as Not Subject to Review.

35. What percentage of your office’s SNAP QC cases each month are coded as incomplete?

% [Enter Percent]

35a. Has that number increased, decreased, or stayed the same over time?

• Increased

• Decreased

• Stayed the same

36. What is the most common reason that cases are coded as incomplete?

• The case file record could not be found

• The household could not be located

• Failure to cooperate (e.g. the client made an initial effort but was unable to coordinate a meeting with you.)

• Refusal to cooperate

• The reviewer was unable to arrive at a likely conclusion

36a. What percentage of incompletes fits this reason?

%

37. What is the second most common reason that cases are coded as incomplete?

• The case file record could not be found

• The household could not be located

• Failure to cooperate (e.g. the client made an initial effort but was unable to coordinate a meeting with you.)

• Refusal to cooperate

• The reviewer was unable to arrive at a likely conclusion

37a. What percentage of incompletes fits this reason?

%

For the next few questions, please indicate if you strongly agree, agree, disagree, or strongly disagree.

38. Overall, my SNAP QC staff has the tools and knowledge they need to correctly determine if a case is complete or incomplete. Would you say that you ?

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

39. Overall, my SNAP QC staff understands the steps they should take if the client in their case fails to cooperate. Would you say that you ?

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

Now I’d like to talk about reasons reviewers cannot locate households.

40. What percentage of households selected for SNAP QC review cannot be located?

%

40a. Has that percentage increased over time, decreased over time, or stayed pretty constant?

• Increased over time

• Decreased over time

• Stayed pretty constant

41. The FNS 310 Handbook mentions that reviewers must make a minimum of two follow-up attempts to contact a household. To what extent are QC reviewers in your State encouraged to go beyond that two-step minimum?

• Additional steps are strongly encouraged (Skip to Q42)

• Additional steps are mildly encouraged (Skip to Q42)

• Additional steps are not encouraged

41a. You said that QC reviewers are not encouraged to take additional time to locate households. Please indicate the main reason why:

• There is not enough time

• There are not enough resources

• Completion of all reviews is not a top priority for our office

• Other, please specify: ______________. [Enter text]

F. ATTITUDES RELATED TO QC REVIEW COMPLETION RATES

F1. ATTITUDES TOWARD COMPLETION RATES

42. What do you think is an achievable target for your office’s SNAP QC completion rate?

Enter %___

[IF R GIVES A RANGE, probe:] We understand why you provided a range. Please pick a single percentage that best captures how you feel; perhaps somewhere in the middle of that range.

43. I am currently satisfied with the completion rate for SNAP QC reviews among my staff. Would you say you…

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F2. ACCOUNTABILITY

44. Is there a process to track the completion rates by reviewer?

• Yes

• No

45. Do you actively monitor the completion rates by reviewer?

• Yes

• No

Now, I’m going to read you some statements about the SNAP QC process. For each indicate if you strongly agree, agree, disagree, or strongly disagree.

46. I am personally held accountable when the employees I supervise have low completion rates for their reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

47. The reputation of my office is negatively affected by low completion rates for SNAP QC reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

48. I am personally held accountable for the integrity of the SNAP QC process.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F3. OFFICE PRIORITIES

49. Obtaining high completion rates for SNAP QC reviews is a priority for my office.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

50. Determining over-or under- issuance of benefits is a priority for my office.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F4. PERCEPTIONS OF STAFF EFFORT

51. My staff goes the extra mile to complete SNAP QC reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

52. During the SNAP QC review process, my staff makes concerted efforts to speak with collateral contacts of a client who is uncooperative or hard to locate.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

53. During the SNAP QC review process, my staff comes up with creative solutions to locate hard-to-reach clients.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

54. My staff routinely shares information about new approaches with each other to expand their tools for completing case reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F5. PERCEPTION OF CHALLENGE

55. Completing SNAP QC review cases in my State is more challenging than in most other States.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

56. There are more clients in my State who cannot be found during the SNAP QC process than in most other States.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

57. Most clients interviewed during the SNAP QC process are honest with information needed for their case review.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F6. Perceptions of Staff Satisfaction

58. The SNAP QC reviewers in my office enjoy their work.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

59. The SNAP QC reviewers in my office are overworked.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

60. The SNAP QC reviewers in my office feel supported by upper management.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

61. There is a high rate of turnover among SNAP QC staff.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

G. SUPERVISORS

Next, I have some questions about supervisors. Again, would you strongly agree, agree, disagree, or strongly disagree?

62. There are an adequate number of supervisors to manage the SNAP QC staff.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

63. Supervisors have the support and tools they need to do their jobs effectively.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

64. Does anyone review SNAP QC staff’s decisions on cases, particularly in the designation of cases as incomplete? Would you say…

• Yes, I review SNAP QC staff’s decisions on cases by myself.

• Yes, I review SNAP QC staff’s decisions on cases with another staff member or members.

• Yes, there is a designated person who reviews SNAP QC staff’s decisions on cases, but it is not me.

• No, there is no one who reviews SNAP QC staff’s decision on cases.

64a. What percentage of SNAP QC staff’s cases are reviewed?

%

65. Finally, is there anything else you’d like to tell me about the SNAP QC review process or factors contributing to SNAP QC review completion rates?

[Enter text response.]

Thank you for your time today.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB number. The valid OMB control number for this information collection is 0584-XXXX. The time required to complete this information collection is estimated to average 30 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.