U.S. Department of Agriculture
Food and Nutrition Service
COMMUNITY PARTNER INTERVIEW DEMONSTRATION PROJECT
Project Officer: Rosemarie Downer
Part B
March 17, 2016
TABLE OF CONTENTS
part
B. Collection of Information Employing Statistical
Methods 1
B.1. Respondent Universe and Sampling Methods 1
B.2. Procedures for the Collection of Information 4
B.3. Methods
to Maximize Response Rates and the Issue of
Nonresponse 5
B.4. Tests of Procedures 5
B.5. Consultants 6
APPENDIX A: SNAP Local AGENCY Director email
APPENDIX B: client satisfaction survey - English
APPENDIX C: client satisfaction survey - spanish
APPENDIX D: INSTRUCTIONS FOR SUBMITTING DATA FILES AND LIST OF REQUESTED
VARIABLES
APPENDIX E: DATA collection SPREADSHEET
APPENDIX F: Nass comments
APPENDIX G: PUBLIC COMMENT
APPENDIX H: FNS Response to PUBLIC COMMENT
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
The purpose of this section is to document the statistical procedures to be used for the Supplemental Nutrition Assistance Program (SNAP) Community Partner Interview (CPI) Client Satisfaction Survey (Appendix B and C). The satisfaction survey will be based on a sample of Individual/Household, SNAP participants who, when applying for benefits, were interviewed by a community-based organization’s (CBO) staff person. Convenience sampling will be used to create the sample. Convenience sampling is a statistical method of drawing representative data by selecting people because of the ease of their volunteering or selecting units because of their availability or easy access. The advantages of this type of sampling are the availability and the quickness with which data can be gathered. The reference period for SNAP participants who were interviewed by a CBO is May 17, 2016 through March 31, 2020.
Key objectives of the CPI Satisfaction Survey (Appendix B and C) include the following:
Objective #1: Assess the levels of satisfaction that SNAP participants report about the quality of services they receive from CBOs (i.e., staff knowledge about SNAP application processes, customer service, wait times, etc.).
Objective #2: Determine the factors that contribute to SNAP participants' choice or decision to apply for benefits at a CBO instead of a SNAP office.
Objective #3: Assess how SNAP participants describe their experience with the CPIs compared to their most recent experience applying for SNAP benefits.
Key results will be tabulated using the satisfaction score, which will be measured in two ways. One measure of satisfaction will be an average score. The other measure will be based on the percentage of SNAP participants who are mostly, or completely, satisfied with the CPI. We aim to complete 3,384 surveys (approximately 676 SNAP participants in each of the 5 States). In summary, the sample for the study was designed to achieve the following goals:
Across-State development of CPI customer satisfaction ratings with 95-percent, two-tailed confidence intervals of between 1.4 and 3.1 percentage points across all 5 States1
Within-State development of CPI customer satisfaction ratings with 95-percent, two-tailed confidence intervals of between 2.7 and 6.2 percentage points for each State
To minimize recall bias, we will eliminate the period between application submission and data collection. Potential respondents will be asked to voluntarily complete the brief survey at the time they are interviewed by a CBO staff person. To maintain privacy, the respondent will be instructed to deposit the survey in a secure drop box located in each CBO office. Steps involved in the sample design are briefly described below.
B.1.1 Target population. The target population for this survey includes individuals who apply for SNAP in selected demonstration counties between May 2016 and March 2020.
B.1.2 Survey Eligibility. All individuals in the target population are eligible for the study, so no screening will be conducted.
B.1.3 Response Rates. Our goal is to achieve an overall response rate of 98 percent. We feel that this is a likely response rate for this survey for several reasons. First, o reduce the respondent burden, we have kept the questionnaire length to a minimum. The survey contains 5 questions that should take no more than 5 minutes. Additionally, potential respondents will be invited to complete the survey immediately after receiving assistance from the CBO staff person. This eliminates the challenges of locating and following up potential respondents. Finally, the survey addresses a subject matter that is likely to be important and relevant to the respondents.
B.1.6 Reliability of Estimates. Overall, estimates of satisfaction percentages (such as the percentage of clients with specific experiences) for the CPIs across the five States will have 95-percent, two-tailed confidence intervals of between 1.4 and 3.1 percentage points. In addition to making survey comparisons across all demonstration States, estimates of satisfaction will also be computed within each State.
B.1.7 Estimation Procedures. The primary purpose of the analyses is to assess the level of client satisfaction and nature of experience by SNAP applicants who were interviewed by a CBO staff person.
We will use SPSS v22 for data management and analysis, which will include statistical techniques such as cross-tabulations and frequency distributions, t-tests, and chi-square tests. Variations in output, per type of analysis, will depend on what statistics are appropriate for the variable and the measurement level (i.e., nominal, ordinal, or scale) for each defined variable. For example, a nominal measure (e.g., nondirectional categories, related to the respondents’ feelings about privacy in the CBO office) will be analyzed using frequencies and percentages. An ordinal measure (e.g., directional categories, such as strongly agree to strongly disagree) will produce counts, percentages, and an overall mean for the variables.
The interview experience and satisfaction questions include Likert-type scales of ordered responses. Analysis of Likert scales cannot assume equal intervals among response options, so it is best to analyze the data in terms of frequencies or percentages. For the most part, chi-square tests or other nonparametric tests are appropriate for testing significance.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection;
Estimation procedure;
Degree of accuracy needed for the purpose described in the justification;
Unusual problems requiring specialized sampling procedures, and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
B.2.1 Data Collection. The proposed satisfaction survey is designed as an in-person survey. Potential participants will be invited to complete the survey after being interviewed by a CBO staff person. This is a highly efficient way to reach a substantial number of respondents where the sampling frame is sufficiently large because the steps to locate and recruit the study participants are eliminated.
The data collection methodology is as follows:
Develop and pretest the survey. (Appendix B and C)
Should a potential respondent refuse to complete the survey; no attempts will be made to encourage the individual to complete the survey.
Response rates by State will be monitored.
Cross-sectional design. The survey is cross-sectional, so no future contacts are planned after a completed survey is submitted.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
The methods described above have been proven in methodological research to yield reasonably high response rates when the survey is of reasonable length and sample members consider the topic important. The following strategies will be used to help achieve a response rate of 98%:
Place visible signs about the survey in the CBO offices
Give a brief introduction that underscores the minimal amount of time required to complete the survey and the importance of the survey topic to every SNAP applicant served by the CBO staff.
Make it clear to potential respondents that they have the option to decline completing the survey and that their choice to complete or not complete the survey does not affect the disposition of their application or the amount of benefits they receive.
The likelihood of acceptance is greatly increased when sample members are told early why the survey is being conducted and why their responses are important.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
We have asked a local partnering CBO in each State to pilot-test the survey under our guidance with one SNAP applicant that they interview. No changes were made to the survey after pilot-testing.
The survey instrument comprises six questions about why they chose the said CBO, their experience with the CBO, if a repeat SNAP applicant, their previous experience applying for SNAP, and satisfaction with the location. We estimate that the satisfaction survey will take approximately five minutes to complete. See Appendices B and C for the English and Spanish versions of the survey.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
FNS consulted with Brian Richardson at the National Agricultural Statistics Services (NASS) about the design, level of burden, and clarity of instructions for the collection. David Hancock serves as a NASS OMB Clearance Officer in the Survey Development and Support Branch. His telephone number is (202) 720-5617. Additionally, FNS consulted with Karen Nelson-Huss – Minnesota, Program Consultant, MN Department of Human Services and Darlene Dougherty – Nevada, SNAP-Ed Nutrition Specialist and SNAP Outreach Coordinator about the availability of data, design, level of burden, and clarity of instructions.
1 The lower bound of this range reflects the 95-percent confidence interval when the population mean of a binary variable is 10 or 90 percent; the upper bound when it is 50 percent.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Yaeko Tise |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |