Memo on Second Round of Cognitive Interviews

Hoke Wilson

February 3, 2021

Page 3

TO: Hoke Wilson

FROM: RTI Project Team

DATE: August 28, 2009

SUBJECT: Findings from Additional Cognitive Testing of the Impact Evaluation Instruments

On August 21, 2009, RTI International conducted 5 additional cognitive interviews in Raleigh, NC to test the draft pre-intervention instrument for demonstration projects in which we are surveying parents of children. The purpose of the interviews was to test the revised questions on fruit and vegetable intake using cups (instead of servings) and to test the revised scale for questions that ask about the frequency of certain activities during the week (number of days vs. yes, sometimes; yes, often; yes, everyday; no scale). This memorandum summarizes the findings from the cognitive interviews and our recommendations for revising the instrument. The revised instrument for NYSDOH is included to illustrate the proposed revisions. If FNS agrees with the proposed changes, we will revise the other instruments.

PARTICIPANTS

We worked with an Extension Associate for the Expanded Food and Nutrition Education Program (EFNEP) at the Wake County Cooperative Extension Center to recruit individuals for the interviews. All participants were SNAP recipients/eligibles who are parents of children aged 3 to 8 years old who are enrolled in school or a Head Start program. One participant was male and four were females. One participant was 18 to 24 years old, three participants were 25 to 34 years old, and one participant was 35 to 44 years old. Two participants were white, Hispanic and three participants were Black, non-Hispanic. All participants, except one, speak English all of the time at home.

PROCEDURES

The interviewer began by asking participants to read and sign an informed consent form. The participants then completed the questionnaire, and the interviewer timed how long it took each participant to complete the questionnaire. After participants completed the questionnaire, the interviewer used a debriefing guide to lead the participants in a discussion to understand why they chose their responses and to identify questions or terms that were confusing or difficult to understand. At the conclusion of the interviews, participants received a cash honorarium of $60.

FINDINGS

Participants found the survey instrument interesting and easy to complete. It took participants, on average, 7 minutes to complete the questionnaire. Specific findings from the cognitive interviews and our suggested revisions to the instrument are summarized below.

Questions on Intake of Fruits and Vegetables

• For question 2, one participant included fruit juice in determining her response. Other participants’ responses did not include fruit juice. One of these participants asked the interviewer whether she should include fruit juice in her answer, but she decided not to include it in her response.
  
  Recommendation: To ensure consistency in responses, we recommend adding the statement, “Do not include fruit juice.”

• For question 3, all participants used the pictures of the measuring cups to answer the question on fruit intake. Most participants found the pictures to be helpful, but one participant said that the pictures were not that helpful since she buys fruit cups which come in containers smaller than 1 cup. When asked to describe how they came up with their answers, participants would recall what types of fruit their child usually ate each day and consider the amount consumed. Some participants included fruit juice in their response and some did not.
  
  Four of the five participants did not read the statement following the question, “In general, 1 cup of fruit or 100% fruit juice or ½ cup of dried fruit can be considered as 1 cup from the fruit group.” When asked to explain in their own words what this statement meant, participants usually mentioned the types of products listed, but did not mention the units (e.g., ½ cup vs. 1 cup). It is not clear why participants did not read this statement, but the use of fractions and conversions, e.g., ½ cup of dried fruit = 1 cup, may have been intimidating or confusing to some participants.
  
  Recommendation: We believe the statement is not necessary and may even be confusing to respondents, thus we recommend deleting it. To ensure consistency in responses, we recommend revising the question as follows, “During the past week, how many cups of fruit did your child usually eat each day? Do not include fruit juice.”

• Similarly, all participants used the pictures of the measuring cups to determine their responses for question 5 on vegetable intake. Four of the five participants did not read the sentence, “In general, 1 cup of raw or cooked vegetables or vegetable juice or 2 cups of raw leafy greens can be considered as 1 cup from the vegetable group.” One participant found the vegetable question more difficult to answer than the fruit question since some vegetables are ingredients (e.g., tomatoes in tomato sauce and corn in tortillas). Participants did not generally count potatoes as a vegetable.

Recommendation: To ensure consistency in responses, we recommend revising the question as follows, “During the past week, how many cups of vegetables did your child usually eat each day? Do not include vegetable juice.”

• For question 6, two participants commented that their children are eligible to receive free breakfast and lunch at school, but they choose not to eat it (i.e., bring lunch instead). Although their children are eligible, they chose the response, “No, did not receive breakfast, lunch, or snacks.”

Recommendation: We may use the responses to this question as a covariate in our analysis to distinguish between respondents with children who do vs. do not consume meals provided by the school or daycare. We recommend revising this question as follows: “During the past week, did your child eat any meals or snacks that were provided by his or her school or day care.”

Questions on Willingness to Try New Fruits and Vegetables

• For question 9, one participant said that her 6-year old does not ask her if he can have fruit as a snack; instead, he helps himself. When probed, she said that children age 4 and older can get their own snacks.

Recommendation: We recommend revising this question as follows: “How many days during the past week did your child ask or help him or herself to fruit for a snack?” The same change would need to be made to Question 13.

Questions on Intake of Low-Fat Dairy

• One participant answered questions 17 and 18 the same. She did not distinguish between low-fat/fat-free yogurt and regular-fat yogurt although she only buys yogurt made from 2% milk.

Recommendation: We do not recommend revising this question; no other participants had problems with this question.

Questions that Used the Revised Scale

• Participants were able to answer questions with the revised scale of none, 1 to 2 days, 3 to 4 days, 5 to 6 days, everyday. Participants were able to think back over the past week and “count” the number of times their children did a particular activity (e.g., number of days ate more than one kind of fruit or vegetable).

Recommendation: We recommend keeping the revised scale.