Appendix M Comments from TWG and FNS Responses_01202022

Appendix M. Comments From Technical Working Group and FNS Responses

The first virtual technical working group (TWG) meeting was held April 20, 2021. The purpose of this meeting was to gather input on the study design and the proposed methods for collecting survey data and conducting in-depth interviews. Table 1 summarizes comments from TWG members and responses from FNS.

Table 1. TWG Comments and Responses From FNS

TWG Comment	Response
Survey data collection methods
Consider the challenges associated with conducting a mail survey in Puerto Rico. Specifically, mail delivery can be unreliable in Puerto Rico because addresses are not standardized, and different residences can have the same mailing address.	The households that will receive a mail survey are those selected from the NAP participant list frame. When we request case files from ADSEF, we will ask for each household’s mailing address. These addresses tend to be up to date because ADSEF and program participants rely on them for sending or receiving communications about program participation and benefits. We expect most of the completed surveys (approximately 75 percent) will come from the area probability frame; data collectors will hand deliver paper surveys to sampled households in this sample frame.
Common weather-related events, such as hurricanes, have been known to destroy street signs and damage roads, making local travel difficult or unsafe. Such events can prevent or delay door-to-door delivery of surveys to sampled households from the area probability frame.	We plan to use local data collectors who reside in or near the selected area segments to deliver paper surveys to sampled households. These data collectors are likely to be familiar with the roads and alternate routes. To reduce the likelihood of data collection being derailed by weather-related events, we aim to begin the field period after the typical hurricane season has ended.
Completing a paper survey will be challenging for low-literacy populations. Survey respondents may be skewed to those who are younger and have high literacy.	Sample households will have the option to schedule an interview to complete the survey by phone.
Consider offering a web-based option for completing the survey. Although internet connectivity is not perfect in Puerto Rico, use and access have improved as schools moved to online classes and many office workers started working from home.	We have added a web-based option based on this recommendation.
Widespread mistrust of the government in Puerto Rico could negatively affect response rates. Sample members might also have concerns about sharing any information that might affect their benefits.	We will ask trusted organizations to endorse the study. The Food Bank of Puerto Rico has agreed to support the study, and we note this endorsement in our invitation letter. We plan to request endorsements from up to two additional local organizations. All written materials will note that participation in the study will not affect any government-issued benefits.
Avoid the use of potentially sensitive phrases such as food insecurity. This term, in particular, carries a stigma among some households, and perceptions of what it means to be food insecure can differ widely.	Study materials refer to health and well-being instead of potentially charged terms such as food insecurity or hunger. We are using the Economic Research Service’s validated food security scale to measure this construct, but the section is labeled “Feeding your household.”
In-depth interview data collection methods
Instead of planning to complete 12 interviews with sample members representing 12 subpopulations, consider creating a range of recruitment—for example, 6–12 for each category or even 6–10. For analysis, you might be looking at the big categories (very low food security versus low food security) or households with children versus households without. Therefore, respondents can be less stratified. Consider taking geography into account for the in-depth interviews to ensure people across Puerto Rico are part of the interview process in an equitably represented way.	Interviewers will meet weekly or biweekly throughout data collection to discuss recruitment efforts and emerging themes. These discussions will help guide decisions about an appropriate range of interviews per subgroup. We will include geographic region in our segmentation plan for the in-depth interviews to help ensure representation from households across the island.
Sample members who are the least likely to complete the survey because of their circumstances (e.g., working food-insecure families, transient households) may be the most difficult to recruit for an in-depth interview, but their experiences will be important to capture.	Throughout the data collection period, interviewers will meet regularly to discuss recruitment metrics and high-level findings. These meetings will help identify gaps and focus subsequent recruitment efforts. If a subgroup of interest is missing or underrepresented among the survey respondents who agreed to be contacted for an interview, we will seek assistance from local community partners that serve the population. Like procedures used in the pretest, we will ask partners to post an announcement on social media about the study. Interested callers will be screened by a member of the study team to determine their eligibility to participate in the in-depth interview.
Concept map data collection
When recruiting stakeholders for the concept-mapping task, include representatives from local recovery agencies, climate experts working on food production, employment and economic development organizations, and government agencies.	We will identify individuals from the TWG’s recommended groups when recruiting stakeholders for the concept-mapping task.

Food Security Status and Well-Being of NAP Participants in Puerto Rico, Appendix M. Comments From M-1
Technical Working Group and FNS Responses