State Agencies (State QC directors, QC supervisors and QC reviewers)

U.S. Department of Agriculture

Food and Nutrition Service

Enhancing Completion Rates for SNAP

(Supplemental Nutrition Assistance Program) Quality Control Reviews

Request for Clearance

Supporting Statement and

Data Collection Instruments

Attachment A-4B:

State QC Reviewer (SQCR)

Web and Telephone Survey

Project Officer: Robert Dalrymple

September 26, 2013

OMB Control Number: 0584-XXXX
Expiration Date: XX/XX/XXXX

State QC Reviewer (SQCR)

Web and Telephone Survey

INTRO 1: Hello, my name is [NAME]. I’m calling on behalf of the Food and Nutrition Service of the US Department of Agriculture. May I please speak with [INSERT NAME FROM SAMPLE]?

• 01 – Yes, selected respondent available (Skip to Intro 2)

• 02 – No, selected respondent not available (Continue to Exit)

EXIT: Thank you, I will call back later. When would be a good time to reach _______?

INTRO 2: FNS is conducting a study of State agencies that conduct the SNAP Quality Control (QC) reviews to help understand the factors that lead to incomplete SNAP QC cases and recommend ways to improve completion rates in the future. In particular, this study focuses on incomplete cases, not cases deemed Not Subject to Review. We understand you are a SNAP QC Reviewer and would like to ask you some questions about the SNAP QC review process. This survey will take about 30 minutes to complete. The survey format includes multiple choice and short answer questions to keep the survey as brief as possible. There is an opportunity at the end of the survey to expand on responses that cannot be summarized through the multiple choice options. Everything we talk about will be private, although this call may be monitored and recorded for quality assurance and transcription purposes. Your responses, which will be reported in summary form in combination with those of other reviewers, will help FNS improve the SNAP QC process.

If you have any questions about the survey, you may contact the Project Officer, Bob Dalrymple, at FNS. Call 703-305-2122 or email [email protected].

• 01 – Continue

• 02 – Not a SNAP QC reviewer [Terminate]

• 99 – Refused [Terminate]

TERMINATE: Thank you for your time.

1. EXPERIENCES AS A QC REVIEWER

Thank you for taking the time to talk with me today. First, I’d like to talk about your experiences working in this position.

1. How long have you been a QC reviewer for the SNAP program?

Years Months

2. Do you have prior experience in the SNAP program other than SNAP QC?

• No (Skip to Q3)

• Yes

2a. How long have you worked in other areas of the SNAP program?

Years Months

3. Do you do anything else besides SNAP QC reviews?

• No, my only task is to conduct SNAP QC reviews (Skip to Q4)

• Yes, I have other tasks besides SNAP QC Reviews.

3a. Do you conduct QC reviews for any other program: FNS, other Federal, or State?

• No (Skip to Q4)

• Yes

3b. Please indicate which of the following you also conduct QC reviews for.

• Other FNS programs

• Other Federal programs (not FNS)

• Other State programs

3c. What percentage of your time is dedicated to SNAP QC?

% [Enter Percent]

B. SNAP QC CASELOAD

Next, I’d like to talk about your SNAP QC caseload.

Interviewer note: On the occasion that you interview a reviewer who has multiple responsibilities at the QC office (i.e. second level analyst, or other position that is not strictly a reviewer), please indicate that for these questions, we are only interested responses that pertain to SQCR responsibilities.

4. Approximately how many SNAP QC reviews were assigned to you last month?

[Enter number]

4a. Of these reviews, how many are:

[Enter Number] Active

[Enter Number] Negative

[Enter Number] Other types of reviews (other than SNAP QC)

5. How has the number of SNAP QC reviews assigned to you changed in the last month? Has it generally:

• Increased

• Decreased

• Stayed about the same

6. Are you ever assigned additional reviews to assist backlogged reviewers or reviewers on extended leave?

• No (Skip to Q8)

• Yes

6a. Are the additional reviews fairly distributed among reviewers?

• No

• Yes

6b. How frequently are you assigned additional reviews?

• Every month

• Most months, but not all

• About half of the year

• A little more than once a year

• About once a year

• More than once a year

7. In geographic terms, what is the primary type of population you most often work with?

• Urban

• Rural

• Suburban

• A mix of urban and suburban

• A mix of rural and suburban

• All of the above

8. In a typical month, how many of your households are homeless?

Enter number

9. How many days do you travel each month on SNAP QC reviews that require you to stay away from home?

• No days

• 1-5 days

• 6-9 days

• 10 days or more

10. Which of the following best describes your typical work schedule when you are reviewing SNAP QC cases?

• Weekdays only

• Mostly weekdays plus occasional weekends as well

• Both weekdays and weekends

11. Do your work hours vary outside of the standard 9-5 workday? By this I mean, are you expected to work early mornings or late in the evening to accommodate households who are working or otherwise not available during the normal 9-5 day?

• Yes, I typically work in the evenings or early mornings.

• Yes. This is an option, but I typically work within a 9-5 schedule.

• No. I do not vary my schedule outside of a typical 9-5 schedule.

12. How many days are you given to complete SNAP QC reviews and submit them as final?

• <60

• 60-75

• 75-95

• 95+

13. Are you given interim deadlines throughout the process which you are required to meet?

• No

• Yes

For the following questions, please tell me how much you agree or disagree with the following statements.

14. I have sufficient time to complete the SNAP QC reviews assigned to me. Would you say that you ?

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

15. The case files I receive from the SNAP office typically have all of the case records and certification information I need to conduct the review.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

C. STATE LEVEL QC REVIEW PROCEDURES

C1. OVERVIEW OF QC ACTIVE CASE REVIEW PROCEDURES

Now, I’d like to get some general information about the process of conducting the SNAP QC reviews. From this point on, we are interested only in ACTIVE cases, not negative QC cases.

16. How often do concerns for your safety affect your ability to contact or locate a household when conducting a review?

• Often

• Sometimes

• Rarely

• Never

17. How often do SNAP clients express safety concerns that hinder the ability to complete a review?

• Often

• Sometimes

• Rarely

• Never

C2. LOCATING AND CONTACTING HOUSEHOLDS

Next, I’d like to get some information about contacting households.

18. How many attempts do you make to contact the households?

Enter number

19. What percent of cases do you successfully contact on your first attempt?

• <50%

• 50-70-%

• 70-90%

• 90+%

20. If the first contact is not successful, how likely is it that you can contact the household during subsequent attempts?

• Unlikely

• Likely, usually during the second or third attempt

• Likely, but usually after four or more attempts to contact

20a. How often are you able to complete cases that you contact only after multiple attempts?

• <70% of the time

• 70%-79% of the time

• 80%-89% of the time

• 90+% of the time

21. During a review period, how many households selected for QC review cannot be located?

___

21a. Has that number increased over time, decreased over time, or stayed the same?

• Increased over time

• Decreased over time

• Stayed the same

• Never been unable to locate a household

22. The FNS 310 Handbook mentions that reviewers must make a minimum of two follow-up attempts to contact a household. To what extent are QC reviewers in your State encouraged to go beyond that two step minimum?

• Additional steps are strongly encouraged (Skip to Q23)

• Additional steps are mildly encouraged (Skip to Q23)

• Additional steps are not encouraged

22a. You said that QC reviewers are not encouraged to take additional time to locate households. Please indicate the main reason why:

• There is not enough time

• There are not enough resources

• Completion of all reviews is not a top priority for our office

• Other, please specify: __________. [Enter text]

23. Has the number of contact attempts necessary to complete a case increased, decreased, or stayed the same since you’ve been doing SNAP QC reviews?

• Increased

• Decreased

• Stayed about the same

Now I am going to read you a list of possible ways to contact a household based on conversations we’ve had with other State reviewers. Please answer “yes” if this is a method you use during case review and “no” if this is a method that you do not use. Please remember that this is a reflection of your own experiences and there is no right or wrong answer.

24. When locating a household, I send a letter to the home address that I have on file.

• Yes (Skip to Q23a)

• No (Skip to Q23b)

24a. Do you send letters through certified mail?

• Yes (Skip to Q23ai)

• No (Skip to Q24)

24ai. When do you send letters through certified mail?

• First contact attempt

• Second contact attempt

• After more than two unsuccessful attempts at contacting the household

(Skip to Q25)

24b. Please indicate why you do not send letters to the home address on file. (Please select all that apply.)

• My manager discourages this method

• This method is too time-consuming

• This method requires too many resources

• Letters are often returned/addresses are not current

• Most clients do not respond to or acknowledge letter

• Other reason

25. When locating a household, I call the client.

• Yes (Skip to Q26)

• No

25a. Please indicate why you do not call the client. (Please select all that apply.)

• My manager discourages this method

• This method is too time-consuming

• This method requires too many resources

• Clients may not have working telephones/change their phone numbers

• Clients do not answer calls from unknown number

• Other reason

26. When locating a household, I use text messaging.

• Yes

• No (Skip to Q26)

26a. Please indicate why you do not use text messaging. (Please select all that apply.)

• My manager discourages this method

• This method is too time-consuming

• This method requires too many resources

• Clients may not be able to receive text messages

• This is not an approved contact method according to State or Federal procedures

• I don’t have or want to use a cell phone.

• Other reason

27. When locating a household, I reach out to neighbors.

• Yes (Skip to Q28)

• No

27a. Please indicate why you do not reach out to neighbors. (Please select all that apply.)

• My manager discourages this method

• This method is too time-consuming

• This method requires too many resources

• Most neighbors are not willing to share information

• Most neighbors do not know enough to make this worthwhile

• Safety concerns

• Other reason

28. When locating a household, I reach out to additional collateral contacts like the U.S. Post Office, DMV, property manager, etc.

• Yes (Skip to Q29)

• No

28a. Please indicate why you do not reach out to collateral contacts. (Please select all that apply.)

• My manager discourages this method

• This method is too time-consuming

• This method requires too many resources

• Most collateral contacts are not willing to share information

• Other reason

29. When locating a household, I use the internet as a means to track down a client.

• Yes (Skip to Q30)

• No

29a. Please indicate why you do not use the internet as a means to track down a client. (Please select all that apply.)

• My manager discourages this method

• This method is too time-consuming

• This method requires too many resources

• This method is hit or miss/may not yield accurate information

• This is not an approved contact method according to State or Federal procedures

• Other reason

C3. ENSURING HOUSEHOLD COOPERATION

Now I’d like to talk about strategies you may use to ensure households cooperate with the review.

30. What is the most effective strategy for ensuring households cooperate with the review?

• Offering flexible times to meet with households, including evenings and weekends

• Offering alternative locations to meet with households

• Sending notifications to the households that failure to cooperate could affect their benefits (or having the caseworker notify the household)

• Enlisting the help of the caseworker/ getting the caseworker to talk to the household

• Offering to help with childcare during an interview

• Something else

30a. For what percentage of cases is this strategy successful?

___%

31. What is the second most effective strategy for ensuring households cooperate with the review?

• Offering flexible times to meet with households, including evenings and weekends

• Offering alternative locations to meet with households

• Sending notifications to the households that failure to cooperate could affect their benefits (or having the caseworker notify the household)

• Enlisting the help of the caseworker/ getting the caseworker to talk to the household

• Offering to help with childcare during an interview

• Something else

31a. For what percentage of cases is this strategy successful?

___%

Now I’d like to talk about households that fail or refuse to cooperate with the SNAP QC review.

32. Last month, how many of your SNAP QC cases refused to cooperate with the review? (Note that we are talking specifically about refusals, not failure to cooperate.)

[Enter Number]

32a. Since you’ve been doing SNAP QC reviews, has that number increased, decreased, or stayed about the same?

• Increased

• Decreased

• Stayed about the same

• Never had a “refusal to cooperate”

33. Last month, how many of your SNAP QC cases failed to cooperate? (Note we are not talking about refusals here.)

___

33a. Since you’ve been doing SNAP QC reviews, has that number increased, decreased, or stayed about the same?

• Increased

• Decreased

• Stayed about the same

• Never had a “failure to cooperate”

34. How often do you use “likely conclusion” to complete a case?

• Never

• 1-5% of the time

• 6-10

• 11-20

• 21-30

• >30% of the time

D. INCOMPLETE CASES

Now I’d like to ask a few questions about incomplete SNAP QC cases. For these questions, we’re interested in only the incompletes and not those disposed as Not Subject to Review.

35. Last month, how many of your SNAP QC cases did you code as incomplete?

____

35a. Has that number increased, decreased, or stayed the same since you have been doing SNAP QC reviews?

• Increased

• Decreased

• Stayed the same

• Never had an incomplete case

36. How often are you successful in turning missed appointments into completed reviews?

• Most of the time

• Some of the time

• Rarely

• Never

37. What was the reason that your most recent incomplete case was incomplete?

• The case file record could not be found

• The household could not be located

• Failure to cooperate (e.g. the client made an initial effort but was unable to coordinate a meeting with you.)

• Refusal to cooperate

• The reviewer was unable to arrive at a likely conclusion

• Has never had an incomplete case.

38. Have the reasons for classifying cases as incomplete changed since you’ve been doing SNAP QC reviews?

• Yes

• No (Skip to Q39)

38a. Please indicate the reason or reasons for a change in the classification of incompletes. Please select all that apply.

• Clients are less cooperative with QC than they used to be (Skip to Q39)

• Clients are more cooperative with QC than they used to be (Skip to Q39)

• Clients are harder to locate than they used to be (Answer Q38ai)

• Clients are easier to locate than they used to be (Answer Q38aii)

38ai. You indicated that clients are harder to locate than they used to be. Why is this? (Please select all that apply.) Clients are harder to locate than they used to be because…

• I have more homeless clients on my caseload.

• I encounter more disconnected phones than I used to.

• I encounter more fake/ incorrect addresses than I used to.

• I receive more returned mail than I used to.

• Collateral contacts are less cooperative in sharing information than they used to be.

• Neighbors and community members are less likely to know information than they used to be.

• Some other reason.

38aii. You indicated that clients are easier to locate than they used to be. Why is this? (Please select all that apply.) Clients are easier to locate than they used to be because…

• I have fewer homeless clients on my caseload.

• I encounter more working phone numbers than I used to.

• I encounter fewer fake/ incorrect addresses than I used to.

• Collateral contacts are more cooperative in sharing information than they used to be.

• Neighbors and community members are more likely to know information than they used to be.

• I utilize technology such as email, social media or texting to contact people.

• Some other reason.

E. TRAINING AND TOOLS

E1. TRAINING

Next, I’d like to ask you about any training and materials you have received for doing QC reviews for SNAP. Please limit your responses to just the training and materials you have received for SNAP.

39. Do you receive any training on how to conduct a SNAP QC review?

• Yes

• No (Skip to Q47)

40. How frequently do you receive training? Would you say___?

• Training happened once, when I started the job

• Training is ongoing and is conducted weekly, monthly or semi-monthly

• Training is ongoing and is conducted twice a year

• Training is ongoing and is conducted annually

• Training is conducted on an as-needed basis with no set schedule

41. From the following list, please tell me whether the following topics are covered during training.

• SNAP eligibility (Y/N)

• Procedural aspects of QC review (Y/N)

• Interview techniques (Y/N)

• Household location techniques (Y/N)

• State-specific policy, including options and waivers (Y/N)

• Likely conclusion as means of case completion (Y/N)

• Other (Y/N)

42. From the following list, please tell me the format of the training.

• Formal in-person training (Y/N)

• Online independent tutorial or module (Y/N)

• Online group webinar (Y/N)

• Conference call (Y/N)

• Informal meetings, such a staff meeting (Y/N)

• Peer mentoring (Y/N)

• Written materials for individual study (Y/N)

• Other(Y/N)

43. Who leads the instructor-led trainings?

• A senior staff member from the State QC office

• A peer QC reviewer from the State office

• A staff member from the Regional office

• A Contractor

• No instructor-led training

44. From the following list, please tell me how reviewers are trained or alerted when there is a change in Federal or State policy that affects SNAP eligibility or allotment determination? Select all that apply.

• Email alert (Y/N)

• State manual page change (Y/N)

• Memo (Y/N)

• Formal training/meeting with staff (Y/N)

• Conference call (Y/N)

• Other (Y/N)

• We are not notified (Y/N)

45. Has the amount of training changed since you’ve been a SNAP QC reviewer?

• No

• Yes, it has increased

• Yes, it has decreased

For the following statement, please tell me whether you strongly agree, agree, disagree, or strongly disagree.

46. The training I received provided me with the tools and knowledge I need to effectively complete a SNAP QC review.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

47. Please tell me whether additional training on each of the following topics would be helpful.

• SNAP eligibility (Y/N)

• Procedural aspects of QC review (Y/N)

• Interview techniques (Y/N)

• Household location techniques (Y/N)

• State-specific policy, including options and waivers (Y/N)

• Likely conclusion as means of case completion (Y/N)

• Other (Y/N)

E2. TOOLS

Now I’m going to ask you about the tools and materials you use when conducting a SNAP QC review.

48. Do you use the FNS 310 Handbook when conducting reviews?

• Yes, I regularly use it as a resource

• Yes, I sometimes use it as a resource but not as often as I used to

• No, I rarely use it now, but I used to use it

• No, I rarely use it now, and I rarely used it in the past

49. Do you have access to a copy of the most recent version of the FNS 310 Handbook?

• Yes

• No (Skip to Q49)

49a. Is that copy of the 310 paper, electronic, or both?

• Paper

• Electronic

• Both

50. Does your State have its own QC review materials or other supplements to the 310 Handbook, such as guidance on State policies?

• Yes

• No (Skip to Q51)

50a. Do you have a copy of the most recent version of those materials?

• Yes

• No

50b. Are those materials paper, electronic, or both?

• Paper

• Electronic

• Both

50c. Do you use the State materials when conducting reviews?

• Yes, I regularly use them as a resource

• Yes, I sometimes use them as a resource but not as often as I used to

• No, I rarely use them now, but used to use them

• No, I rarely use them now and rarely used them in the past

51. Do you use an automated or electronic 380?

• No (Skip to Q52)

• Yes

51a. How frequently do you use the automated or electronic 380?

• Always

• Often

• Sometimes

• Rarely

• Never (Skip to Q52)

• Not available yet (Skip to Q52)

51b. How has the automated or electronic 380 affected your ability to complete SNAP QC reviews?

• The process is easier

• The process is about the same

• The process is more difficult

52. Are there additional tools or materials (either paper or electronic) available to you when you are conducting SNAP QC reviews?

• Yes

• No

F. ATTITUDES RELATED TO QC REVIEW COMPLETION RATES

F1. ACCOUNTABILITY

Next, I’m going to ask you a few questions about your attitudes and opinions about doing SNAP QC reviews. For the each of the following statements, please tell me whether you strongly agree, agree, disagree, or strongly disagree.

53. I am personally held accountable when I have low completion rates for SNAP QC reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

54. The reputation of my office is negatively affected by low completion rates for SNAP QC reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F2. OFFICE PRIORITIES

55. Obtaining high completion rates for SNAP QC reviews is a priority for my office.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

56. Determining over- or under- issuance of benefits is a priority for my office.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F3. PERCEPTIONS OF EFFORT

57. I go the extra mile to complete SNAP QC reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

58. During the SNAP QC review process, I make concerted efforts to speak with collateral contacts of a client who is uncooperative or hard to locate.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

59. During the SNAP QC review process, I come up with creative solutions to locate hard-to-reach clients.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

60. During the SNAP QC review process, I come up with creative solutions to convert uncooperative clients.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

61. My office environment and managers encourage sharing information about new approaches with other SNAP QC reviewers to expand their tools for completing SNAP QC reviews.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F4. PERCEPTION OF CHALLENGE

62. Most clients interviewed during the SNAP QC process are honest with information needed for their case review.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

F5. PERCEPTION OF JOB SATISFACTION

Finally, I am going to ask you a few questions about your job satisfaction. Your answers to these questions will be kept private and you do not need to answer them if you do not want to.

63. I enjoy my work.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

64. I work hard.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

65. I feel supported by management.

• Strongly Agree

• Agree

• Disagree

• Strongly Disagree

66. Finally, is there anything else you’d like to tell me about the SNAP QC review process or factors contributing to SNAP QC review completion rates?

[Enter text response.]

Thank you for your time today.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB number. The valid OMB control number for this information collection is 0584-XXXX. The time required to complete this information collection is estimated to average 30 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.