Food and Nutrition Service
Assessment of Administrative Costs of Electronic Healthy Incentives Projects
Review conducted by Prakash Adhikari, Mathematical Division
Summary, Estimation, and Disclosure Methodology Branch
National Agricultural Statistical Service (NASS)
Date Received: August 2, 2024
Comments:
The purpose of the study is well defined – understanding the costs of integrating nutrition incentive programs into State electronic benefits transfer (EBT) systems and how these costs are different from the costs of setting up and running incentive programs that are not integrated in EBT systems for three eHIP States: Colorado, Washington, and Louisiana. It also summarizes the more specific goals such as Return on Investment (ROI) study of eHIP, where the cost would incur etc. The data collection process and timeline are thoroughly outlined. It also provides the detailed tasks of conducting the survey- who the respondents are, the step-by-step guide of the interview process, and the types of questions they will be asked.
However, the paper does not address statistical methods to be used and explain why they are not needed currently. It is stated that Westat will conduct the qualitative and quantitative analyses, but there is no mention of statistical methods they will be using. Perhaps, Westat will provide them at a later stage.
It seems we are expecting a one hundred percent response rate, but we did not notice any methods that will be used should there be a need of response rate adjustment due to non-response.
Overall, the paper is well outlined and details all aspects of the survey.
Summary:
Legal Authority: Food and Nutrition Act of 2008 – Section 17: Good
B. Research Objectives and Questions Crosswalk: Good, clearly states the research objectives and cost associated with various methods of capturing the data.
Project Summary:
Section A: Study Overview
Brief Overview: Good, states the goal of the study which is to understand the costs of integrating nutrition incentive programs into State electronic benefits transfer (EBT) systems and how these costs are different from the costs of setting up and running incentive programs that are not integrated in EBT systems. It also summarizes the more specific goals such as ROI study of eHIP, where the cost would incur etc.
Research activities undertaken by Westat: Good, has a clear timeline of the project from contacting the three states involved from 2023 to submitting the datasets and related document by July 2027.
Partners and their roles: Good, specifies the three people involved in providing the input in the research.
Not applicable, the study does not involve an experiment with treatment and control conditions.
Good, defines how the human subjects (program staffs, retailers, EBT vendors, and third-party processors) for the quantitative data of the study will be involved.
Section B. Informed Consent Process
Informed Consent Process: Good, the respondents will be read the consent. information and asked if they agree to the interview.
Incentives: Not applicable, no incentives provided for the participation.
Waivers: Not applicable, no waiver or modification is requested.
Section C. Risks
Good, there is little to no risk to the respondents. There will be no personal questions asked and the respondents are free to skip over any questions they are uncomfortable with.
Section D. Benefits
Good, provides examples of both direct (knowledge about the cost and ROI of the eHIP study) and indirect benefits (possible involvement of other states) to the respondents.
Section E. Confidentiality, Data Security, and Destruction Procedures
Good, has a plan to use multiple datasets to store the information where each dataset will have a common ID in Westat’s secure server with limited access to project staff only.
Electronic Letter to States with Data Request and Reminder
Good, the memo is well written, has concise and clear instructions for the Quarterly State Administrative Cost Workbook for each tab. There are further extensive instructions defined related to Quarterly Expenses, Personal Hours, Salaries, Non-personal Costs, and Summary Worksheets. The reminder memo also has specific detail on what is expected.
Non-Personal Cost Activity
Incentive Report Template
Retailer list
Electronic Letter with Request to schedule Interview
Electronic Letter with Reminder to Schedule Interview
Electronic Letter with Reminder about Interview
Good, The Previous sections (E-J) provide the templates or memo to be used for each section.
In-Depth Interview Protocol:
Introduction: Good, the interviewer introduces himself/herself and state they work for Westat
Purpose: Good, clearly stated as in the Project summary section (Section C)
Information to be collected: Good, states the types of information asked during the interview.
Risks and privacy: Good, explained that there is no risk to the respondent. For privacy concern, the report will consist of the quotes but will not identify the speaker.
Study costs and compensation: Good, there is no cost to the respondent (about 60 minutes).
Voluntary participation: Good, the participants can refuse an interview or questions.
Questions: Good, there is a Westat phone number for any questions.
The next portion of the document outlines exactly how the interview will proceed. It includes the details of types of questions the respondents will be asked based on whether it is their first interview or second interview.
Interview Worksheets: Good, provides the template.
Interview Consent Form
Good, the form asks for consent from the respondents. The respondents are provided with the details of the type of information to be collected, risks and privacy, study costs and compensation, voluntary participation, and a phone number and email address should they have any questions.
Interview Follow Up and Thank You Note
Electronic Letter to GusNIP Grantees with Data Request
60-Day Federal Notice
NASS Comments
Responses to NASS Comments
Institutional Review Board Approval Letter
Westat Information Security and Confidentiality Pledge
Good, The Previous sections (N-T) provide the templates, placeholder or memo to be used for each section.
Total Public Burden Hours and Costs:
Good, provides the hourly wage rate and total annualized cost of respondent burden.
Pretest Protocol
Good, gives the respondents the opportunity to ask and/or comment on detailed questions regarding the Administrative Cost of Workbook, Activities, Personnel Hours, Salaries, and Incentive Report.
Pretest Methods and Summary of Findings
Good, provides the questions and/or comments from the states resulting from the Section V: Pretest Protocol.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 2023-000: |
| Author | Kelley Calvin |
| File Modified | 0000-00-00 |
| File Created | 2024-11-26 |