SS Part A PMS-SNAP Mod 5-17-2010

Supporting Statement For

Performance Standards and Reporting for Supplemental Nutrition Assistance Program Modernization Initiatives

Rosemarie Downer

Office of Research & Analysis

Food and Nutrition Service/USDA

Park Center Building, Room 1014

3101 Park Center Drive

Alexandria, VA 22302

703-305-2129

703-305-2576

[email protected]

CONTENTS

Part Page

A. JUSTIFICATION 1

A.1. Circumstances Making the Collection of Information Necessary 1

A.2. Purpose and Use of the Information 2

A.3. Use of Information Technology and Burden Reduction 3

A.4. Efforts to Identify Duplication and Use of Similar Information 4

A.5. Impacts on Small Businesses or Other Small Entities 5

A.6. Consequences of Collecting the Information Less Frequently 6

A.7. Special Circumstances Relating to the Guideline of 5 CFR 1320.5 6

A.8. Comments in Response to the Federal Register Notice and Efforts

to Consult Outside Agency 6

A.9. Explanation of Any Payment or Gift to Respondents 7

A.10. Assurance of Confidentiality Provided to Respondents 7

A.11. Justifications for Sensitive Questions 7

A.12. Estimates of Hour Burden Including Annualized Hourly Costs 7

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers 11

A.14. Annualized Cost to Federal Government 11

A.15. Explanation for Program Changes or Adjustments 12

A.16. Plans for Tabulation and Publication and Project Schedule 12

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 14

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 14

APENDIX A: PUBLIC COMMENTS

APPENDIX B: respondent invitation letters

APPENDIX C: project description

APPENDIX D: frequently asked questions

APPENDIX e: Request for performance data

APPENDIX f: survey access instructions

APPENDIX g: electronic survey

APPENDIX h: survey reminder call script

APPENDIX i: Phone interview scheduling call script

APPENDIX j: performance data reminder call

APPENDIX k: snap staff phone interview protocol

APPENDIX l: snap partner phone interview protocol

APPENDIX m: site visit interview topics

LIST OF TABLES

Table A.12.1 Annual Burden Estimate 10

Table A.12.2 Annual Cost to Respondents 11

Table A.16.1 Data Collection, Analysis, and Reporting Schedule 14

A. JUSTIFICATION

A.1. Circumstances Making the Collection of Information Necessary

This is a new information collection. Efforts are underway in many States to modernize the Supplemental Nutrition Assistance Program (SNAP) to improve access among eligible households and increase operational efficiency, while maintaining payment accuracy. While modernization may achieve these goals, it also poses potential threats to smooth operations. States report some required measures of access, accuracy, and efficiency to the U.S. Department of Agriculture (USDA) Food and Nutrition Service (FNS), but there are currently no set performance requirements for modernization initiatives or for their measurement. States vary in the modernization features they implement and in whether, what, and how they measure additional aspects of performance.

The Food, Conservation, and Energy Act of 2008 (Public Law 110-234), amended Section 11 of the Food and Nutrition Act of 2008, 7 USC 2020, to include a provision for FNS to develop standards for identifying major operational changes and for States to provide any information required by the USDA. To inform USDA’s development of such standards, this data collection will gather information from all 50 States and the District of Columbia¹ in order to better understand how States are assessing performance of their modernization initiatives. Prior collections, 0584-0537 Study of the Modernization of the FSP in Florida, expiration date 07/31/2009; and 0584-0547 Enhancing Food Stamp Certification: Food Stamp Modernization Efforts, expiration date 4/30/2011, together gathered information from all States about their modernization efforts. This new collection effort will focus on States’ approaches to modernization performance measurement and will update FNS’ information on State modernizations to provide context for understanding performance measurement procedures, drawing on information prior studies provided about the range of modernization activities in which states may be engaged.

A.2. Purpose and Use of the Information

Information for the Performance Standards and Reporting for Supplemental Nutrition Assistance Program Modernization Initiatives will be collected by Mathematica Policy Research. To obtain a broad view of SNAP performance measures, staff from State and local SNAP offices and partner organizations will be asked to respond. Responses collected from 150 county and local SNAP offices and 122 partner organizations will identify the extent to which performance measurement activity is being performed at local levels. Data will be collected via surveys of SNAP staff, telephone and in-person interviews of SNAP and partner staff, and aggregated administrative records collection from SNAP and partner agencies.

The electronic survey (Appendix G), delivered to respondents on a secure file exchange site (or by CD-ROM if requested) will ask about the performance measures and data SNAP agencies collect and can be returned by uploading responses, email, or faxing or mailing paper copies. The file exchange site uses a Secure Sockets Layer (SSL) certificate to encrypt the data transmission that respondents will authenticate with a username/password combination provided by Mathematica.

Telephone interviews (Appendices K and L) will include questions about how respondents define and collect their performance measures, the context for implementing them, the rationale behind choices to implement some measures and not others, and their success with using the measures. Telephone interviews, requiring an average of 90 minutes or less to complete, capture the detailed national information required to meet the goals of the study. Participating SNAP agencies and partner organizations will be asked to submit 12 months of performance data for each of the measures they calculate. Finally, in-person interviews (Appendix M) of State and local SNAP staff and partner organization staff in 10 States will provide additional in-depth details about performance measurement in States where the most intensive efforts are underway.

This collection includes one wave of each type of data collection, to be completed within approximately four months of receiving Office of Management and Budget (OMB) approval (it is not an ongoing collection). Information obtained from this collection will only be shared with other organizations in aggregate form, without any personally identifying information.

The four-stage data collection approach outlined in this Supporting Statement will systematically collect detailed information about performance measurement activities related to specific SNAP modernization efforts from all States and a number of county and local SNAP offices and program partners. The four stages are synchronized such that data collected in the later stages help to ensure the accuracy of data collected in earlier stages. In addition, each successive stage in the process gathers a greater level of detail about the implementation of SNAP performance measurement from the SNAP agencies and program partners.

A.3. Use of Information Technology and Burden Reduction

In conjunction with the E-Government Act of 2002 (E-GOV), this collection will employ technology in administering the survey (as an Access Runtime executable file) and in streamlining interviews and accepting and analyzing varied formats of administrative data (in a project database that produces tailored protocols and calculates uniform performance measures from diverse sources). The electronic, downloadable survey has three advantages for the user. First, it requires no special software to run on a respondent’s computer; everything the respondent needs will be available via download or on the CD-ROM submitted to them. Second, it includes a user-friendly survey interface that uses skip patterns to help ensure that a respondent will only see questions that are related to the modernization initiatives that have been implemented in their State. And finally, it prevents respondents from accessing underlying data, which ensures that survey questions or formatting are not mistakenly deleted or otherwise altered. Mathematica will then use survey responses to streamline telephone interviews and on-site interviews, using the project database to execute skip patterns in the interview protocol that identify the questions most relevant to a respondent’s situation. This approach reduces respondent burden by enabling the research team to most efficiently use the respondent’s telephone interview time. Finally, in requesting 12 months of performance data Mathematica will minimize the burden on respondents by accepting those data in the format—whether it is actual performance data or summary reports—and delivery method that is most convenient for respondents (such as use of the secure file exchange site, email, fax, or hard copy delivery). The project database will render these varied formats comparable by arraying them in a common database, and calculating them uniformly across States, eliminating the need for respondents to submit data in a specified format that they may not routinely use. It is estimated that well over half the states will submit the required data electronically at www.SNAP.mathematica-mpr.com.

A.4. Efforts to Identify Duplication and Use of Similar Information

There are similar data collection efforts available, but those do not meet our needs. Based on information from State and regional offices, there is no existing, uniform, national information available on the use of performance measures to gauge SNAP modernization activities. Some respondents described their modernization activities and limited measurement information in the recent collections for Enhancing Food Stamp Certification: Food Stamp Modernization Efforts and Study of the Modernization of the FSP in Florida. That information was used to formulate questions and select the modernization initiatives for the current study. This new survey will include a few focused questions about current modernization activities so that interviewers and the survey tool can filter out subsequent irrelevant questions, but most of the questions will collect new performance measure information. For example, the survey will ask respondents if their State or locality has an online application and, if they answer no, will skip asking questions about performance measures for online applications; phone interviewers will similarly skip more detailed questions about calculating such performance measures when talking with the respondent.

A.5. Impacts on Small Businesses or Other Small Entities

We will request participation from 115 small organizations that may, as SNAP partners, participate in this collection² and will minimize burden on these small organizations by asking them only to participate in telephone and in-person interviews and to submit copies of existing reports on 12 months of performance data. We will not ask them to complete the survey because we will have sufficient information from the SNAP office about the partner’s modernization activities to proceed with the interview. Additionally, we will tailor interview questions to ask staff only the questions that pertain to their work with the state or county or local SNAP office and to exclude references to modernization initiatives in which the partner agency is not involved. We estimate these interviews will be 60 minutes instead of the 90 minutes needed for the SNAP staff interviews.

A.6. Consequences of Collecting the Information Less Frequently

This is a one-time collection effort. The data collection plan described in this submission is necessary for conducting the Study of Performance Standards and Reporting for SNAP Modernization Initiatives Data Collection and will help FNS understand how performance standards and reporting are occurring in the States with respect to SNAP modernization. Respondents may respond more than once during the study. The four stages allow data collected in the later stages to help ensure the accuracy of data collected earlier. Each successive stage gathers more detail about the implementation of SNAP performance measurement. Without this data collection, FNS would not be able to effectively develop, monitor or assess standards for identifying operational changes.

A.7. Special Circumstances Relating to the Guideline of 5 CFR 1320.5

In order to comply with FNS’ study timeline, we will request that respondents return the survey within three weeks. Doing so permits the research team to tailor its telephone and on-site interview instruments to further minimize the burden on respondents, asking only questions that are relevant to the situation of their state or locality. Waiting longer to receive the survey responses will not permit tailored interviews to be completed in a timely fashion.

There are no other special circumstances. The information is collected in a manner consistent with the guidelines in 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

A notice of the proposed information collection and an invitation for public comment was published in the Federal Register, September 24, 2009, Volume 74, Number 184, Pages 48709-48710. Three comments were received (included in this package) from New York, Louisiana, and the American Public Human Services Association (APHSA). As requested by New York, FNS provided the draft instruments. In response to the other comments, the instruments have been streamlined as much as possible, eliminating several questions that are not directly relevant to the project objectives and research questions. As suggested, as much data as possible will be collected electronically, with phone interviews used only for clarification purposes and to collect data that are not amenable to collection through electronic surveys.

In addition to soliciting comments from the public and from National Agricultural Statistics Service (NASS), FNS consulted four Mathematica senior technical staff about the availability of data, design, level of burden, and clarity of instructions for this collection:

Laura Castner: Senior Researcher and Project Director 202-484-3282

William Borden: Senior Fellow and Technical Reviewer 609-275-2321

Scott Cody: Associate Director of Research, Technical Reviewer 202-484-4523

Rhoda Cohen: Senior Survey Researcher and Technical Reviewer 609-275-2324

No other federal agency representatives commented on this information collection.

A.9. Explanation of Any Payment or Gift to Respondents

There will be no payments or gifts to respondents.

A.10. Assurance of Confidentiality Provided to Respondents

This collection is not gathering any confidential data. All data on State and local SNAP modernization efforts will be requested and reported in aggregate form. During the collection, the study’s database will retain contact information for each respondent and the details of their responses about State, county, local, or partner efforts towards performance measurement, but this contact information will not be released. In accordance with the FNS Privacy Act, Mathematica will safeguard all data, and only authorized users will have access to them.

A.11. Justifications for Sensitive Questions

No sensitive questions will be asked.

A.12. Estimates of Hour Burden Including Annualized Hourly Costs

Respondent groups for this collection will include staff from State and local SNAP offices and SNAP community business and nonprofit partners: an estimated total of 786 respondents. This includes: 51 State SNAP office staff from each state SNAP office who will respond approximately 3.5 times, and convenience samples of 400 county/ local SNAP office staff and 335 SNAP partners at local community organizations and businesses.

State	Predicting a 100 percent response rate, 51 State SNAP offices will complete the survey approximately 3.5 times for a total of 178 annual responses, 51 State SNAP office will complete the telephone interview approximately 2.5 times for a total annual response of 127, and 51 will complete the in-person interview only one time per State for a total of 51 annual responses. Records will be collected from all State SNAP office approximately 3 times per respondent for a total of 153 responses. Survey and telephone pretests were conducted with one state. The total burden hours required for data collection from the States is 734.57. Table A.12.1 provides the burden hours required for each respondent source.
County/Local SNAP Office	Predicting a 83 percent response rate, of 400 county and local SNAP office staff, 332 will complete the survey once (68 will be attempted) for a total of 332 annual responses, 332 will complete the telephone interview once for a total of 332 annual responses, and 90 will complete the in-person interview once for a total of 90 annual responses. Records will be collected from 332 respondents once for a total of 332 responses. The total burden hours required for data collection from the county and local offices is 1,943.67. Table A.12.1 provides the burden hours required for each respondent source.
Partner	Predicting a 73 percent response rate, of 335 SNAP partners, 244 will complete the telephone interview once (91 will be attempted) for a total of 244 annual responses and 30 will complete the in-person interview once for a total of 30 annual responses. Records will be collected from 244 respondents once for a total of 244 responses. Telephone interview pretest was conducted with one partner. The total burden hours required for data collection from the partners is 794.12. Table A.12.1 provides the burden hours required for each respondent source.

For in-person and telephone interviews, burden estimates reflect the flexible nature of the interviews. The interview guides provided in Appendices K, L, and M reflect a comprehensive list of potential questions, follow-up probes, and interview topics that researchers might pursue during the interviews. The questions that will be asked of each respondent will be determined by the survey responses provided by SNAP staff in their State.

The estimated time of response varies from 10 minutes (0.167 hours) for nonresponders to 4 hours total (between one and two hours each) for two to three responders to collaborate on compiling 12 month performance data or for a single responder to participate in multiple parts of the collection. The estimated total annual burden on respondents is 3472.54 hours.

The total cost to respondents for their time in this collection is $62,354.34 (Table A.12.2). To calculate the annualized cost to respondents, we used the mean hourly wage rate categories as determined by the Bureau of Labor Statistics, May 2008, National Industry-Specific Occupational Employment and Wage Estimates. Estimates for the total cost to respondents for this collection is based on three respondent categories: State government staff, local government staff, and staff providing community food and housing, emergency and other relief services. The average time per response incorporates all respondent time to read instructions and make logistical arrangements for interviews, including time incurred by nonrespondents.

Table A.12.1 Annual Burden Estimate

Respondent		Estimated Respondents	Responses Annually per Respondent	Total Annual Responses	Estimated Avg. Hours Per Response	Estimated Total Hours
State
Survey	Completed	51	3.5	178	1.37	245
Survey - pretest	Completed	1	4	4	1.50	6
Telephone interview	Completed	51	2.5	127	1.54	196
Telephone interview - pretest	Completed	1	1	1	1.75	1.75
In-person interview	Completed	51	1	51	1.55	79
Record collection	Completed	51	3	153	1.35	207
County/Local SNAP Office
Survey	Completed	332	1	332	2.06	684
	Attempted	68	1	68	0.25	17
Telephone interview	Completed	332	1	332	1.55	515
In-person interview	Completed	90	1	90	0.63	56.67
Record collection	Completed	332	1	332	2.02	671
Partner
Telephone interview	Completed	244	1	244	1.07	261
	Attempted	91	1	91	0.08	7
	Pretest	1	1	1	1.07	1.07
In-person interview	Completed	30	1	30	1.04	31.34
Record collection	Completed	244	1	244	2.02	493.71
Total burden						3472.54

Note: We selected Pennsylvania for the pretest because the state is operating numerous modernization initiatives. This allowed us to pretest nearly all components of the survey and interview guide, but also required more than the average respondent time we expect this collection to require. The survey and interview are identical for state and county/local offices, so we pretested in the state office only. The state office will not be asked to respond to the survey or telephone interview again when data are collected from remaining states.

Table A.12.2 Annual Cost to Respondents

Respondent Type	Instrument Type	Average Time per Response	Number of Respondents	Frequency of Response		Median Hourly Wage Rate	Respondent Cost
State SNAP Staffa	Survey	1.37	51	3.5		$20.73	$5,069.42
	Survey - pretest	1.50	1	4		$20.73	$124.38
	Telephone interview	1.54	51	2.5		$20.73	$4,070.34
	Telephone interview - pretest	1.75	1	1		$20.73	$36.28
	In-person interview	1.55	51	1		$20.73	$1,638.71
	Records collection	1.35	51	3		$20.73	$4,281.78
County/Local SNAP Staffb	Survey - completed	2.06	332		1	$18.78	$12,844.12
	Survey – non-respondents	0.25	68		1	$18.78	$319.26
	Telephone interview	1.55	332		1	$18.78	$9,664.19
	In-Person interview	0.63	90		1	$18.78	$1,064.83
	Records collection	2.03	332		1	$18.78	$12,256.97
SNAP Partners	Telephone interview - completed	1.07	244		1	$13.80	$3,602.90
	Telephone interview – non-respondents	0.08	91		1	$13.80	$100.46
	Telephone interview - pretest	1.07	1		1	$13.80	$14.72
	In-Person interview	1.04	30		1	$13.80	$430.56
	Records collection	2.03	244		1	$13.80	$6,835.42
Total							$62,354.34

^a NAICS 999200: State Government

^b NAICS 999300: Local Government

^c NAICS 624200: Community Food and Housing, Emergency and Other Relief Services

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

No capital/start-up or ongoing operation/maintenance costs are associated with this information collection.

A.14. Annualized Cost to Federal Government

The total costs of this study include a firm fixed price contract with Mathematica and time spent by the federal project officer to manage data collection ($6,600). The budget for this contract estimates total costs to be $1,498,508, for a total of $1,505,108 including project officer costs. Contract costs include design of the study and development of data collection instruments, data collection, analysis, and report writing. Annual contract costs for the study are as follows:

Year 1: $445,387 (September 2008-August 2009)

Year 2: $629,573 (September 2009-August 2010)

Year 3: $423,548 (September 2010-August 2011)

A.15. Explanation for Program Changes or Adjustments

This is a new collection of information resulting of program changes 3,473 burden hours.

A.16. Plans for Tabulation and Publication and Project Schedule

Products resulting from information obtained in this data collection will provide FNS with an inventory of modernization-related performance measures, an analysis of the characteristics of existing performance measures, and an assessment of the options for moving performance measurement forward. Mathematica will integrate the information obtained during data collection and will perform a thorough analysis of existing performance measurements for SNAP program access, accuracy and integrity, efficiency, and customer service. Along with describing what States are measuring and the standards they have set, the analysis will highlight key gaps in measurement and the context in which those gaps exist. It will also seek to identify the importance of the measures to the State and local agency staff, the rationale behind the implementation decisions, and discuss changes in the measures over time. In addition, the analysis will provide FNS, States, county and local agencies, and community partners with feedback on how their performance measures compare with others conducting similar activities. Where possible, Mathematica will make comparisons across States and within States, discussing factors that might lead to any observed variation.

Mathematica will deliver analyses derived from this data collection to FNS in a series of deliverables (final reports, briefings, and ad hoc memoranda) that will present key findings of the study in clear, nontechnical language that makes them accessible to a wide audience of policy makers, researchers, and program staff. Table A.16.1 presents the schedule for delivering these products to FNS. Mathematica will qualitatively analyze the information from all four modes of data collection to prepare these deliverables. No complex quantitative analytical techniques will be employed in this project. The reports, to be published on the FNS website, will include a stand-alone summary of the purpose, methodology, key findings, and policy implications, as well as a short executive summary attached to the full report. Tables, figures, and graphs will help the reader quickly absorb the findings and easily compare and contrast information across States.

Final Reports. The first report will provide a menu of possible performance measures and standards for each modernization initiative. The report will also discuss the choices States have made with respect to how specific measures and standards relate to FNS’ program goals—program access, accuracy and integrity, efficiency, and customer service. The analysis will also identify States’ consistent use of performance measures and establishment of standards where such measures and standards are in place, and also where gaps exist. The second report will create performance profiles for each State by documenting the actual performance measures and standards used in each State. The report will separate measures by modernization initiative and by how they relate to FNS’ four program goals across key phases of the program—from application, to eligibility determination, through recertification. In this report, Mathematica will include the 12 months of actual performance data that States have submitted. These reports will be published on FNS’ intranet at www.insert-url-here.gov.

Table A.16.1 Data Collection, Analysis, and Reporting Schedule

Data collectiona	3/1/2010 - 7/06/10
Data analysis; preliminary analysis report and briefing
Final table shells	7/20/10
Briefing Complete data analysis	11/16/10 2/14/11
Reports
Draft report 1	5/10/11
Draft report 2	7/19/11
Revised report 1	8/02/11
Revised report 2	8/30/11
Final report 1	9/27/11
Final report 2	10/25/11
Final briefing	9/27/11
Ad hoc memoranda
Memorandum #1	Upon request
Memorandum #2
Memorandum #3
Memorandum #4
Analysis file
File documentation	11/22/11
Revised file documentation and data files	12/13/11

Briefings. Mathematica will brief FNS staff twice. The first briefing will present preliminary findings prior to writing each of the two reports. The second briefing will focus on the results and the key findings from the two major reports. The briefings will be written and presented in a manner that is appropriate for a nontechnical or policy audience from FNS.

Ad Hoc Memoranda. Mathematica will also prepare four ad hoc memoranda to address specific issues related to SNAP performance measurement, including implications for selecting performance measures and setting performance standards.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

The agency will display the OMB expiration date on all instruments.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

1 For ease of reference, in the remainder of this supporting statement we use the term States to refer to the 50 States as well as the District of Columbia.

2 We expect that approximately seven of the 122 interviews with partner organizations will be with large companies that provide services under contract to state or local SNAP agencies.