Appendix A_PreTest

AppendixA_PreTest.docx

Generic Clearance for Survey Research Studies

Appendix A_PreTest

OMB: 0536-0073

Document [docx]
Download: docx | pdf

Appendix A: Pre-Test


Sample and Recruitment


We pre-tested our draft survey instrument (see Exhibit A.1) with 6 persons in 3 research teams. The respondents were selected from the contact list (Appendix E). We stratified the contact list by the four affiliations through which research teams could have gained access to the FoodAPS-1 dataset. These strata included: 1) National Bureau of Economic Research (NBER) grantees, 2) University of Kentucky Center for Poverty Research (UKCPR) grantees, 3) cooperative agreement researchers, and 4) independent external researchers. We purposively selected one principal investigator per stratum to recruit for the pre-test.


For recruitment purposes, we communicated with potential participants via email (see Exhibit A.1). If we did not receive a response, we followed up via email within six business days (see Exhibit A.2). Of the four principal investigators we contacted, three agreed to complete the survey or to circulate the survey instrument within their research teams. Of the three research teams that completed the survey, one team also provided solicited feedback on the survey instrument itself. A total of 6 individuals participated in the pre-test.


Format


We offered participants the option of completing the survey either in writing (by completing the instrument in Word document format and returning it by email), in person, or via telephone. Of the three research teams that participated in the pre-test, two teams responded in writing, and one team provided responses both in person and via telephone. For the pre-test only, we included additional questions about the quality of the survey instrument and about completion time. The mean time reported under each mode was 30 minutes.


Pre-Test Survey Instrument


The initial instruments for the pre-test are reported in Exhibit A-3. The instruments include a mix of closed and open-ended questions. Topics include background information, data usage, and recommendations. We summarize revisions below.

Recruitment Script and Survey Revisions


All three principal investigators involved in the pre-test indicated a need to share the survey with collaborators who had more direct contact with the FoodAPS-1 data. Acknowledging this feedback, we revised our recruitment script and the survey instrument. For the recruitment script, revisions involved including language, asking principal investigators to identify their research team’s data analysts for us to contact directly. For the survey instrument, revisions involved including section headings that indicate whether survey sections contain general or specific questions.


The revised recruitment script (see Appendix B-1) indicates that principal investigators will identify the data analyst, who is better able to answer detailed questions. For example, researchers who directly analyzed FoodAPS-1 data may be better equipped to answer specific questions about data usage; whereas, principal investigators who had indirect contact with the data may be able to answer general questions. The revised survey instrument (see Appendix D) indicates whether survey sections are general or specific. This version of the survey instrument also allows respondents to indicate if another member of their research team will enter responses for a given section.


Other revisions we made based on pre-test feedback include: 1) reducing the telephone interview time burden estimate provided in the recruitment script from 40 to 30 minutes and 2) including an optional comments field after closed-end questions in the “Data Usage” section.





Exhibit A-1. Initial Recruitment Email for Pre-Test


Subject: A Questionnaire about FoodAPS User Experience (Pre-Test)


Dear [name of contact person]:


Please help us pre-test a questionnaire that we will use to help USDA learn from researchers' experiences with the National Household Food Acquisition and Purchase Survey (FoodAPS).


We have developed a draft questionnaire to assess the dataset’s uses, strengths, and needs for improvement. We will report results to USDA Economic Research Service to inform a possible next round of FoodAPS. 

 

Would you be willing to participate in this pre-test? Participation is voluntary and confidential and entails: 

  • Completing the FoodAPS researcher questionnaire, either in writing (approximately 30 minutes) or via telephone (approximately 40 minutes)

  • Answering a few short additional questions to help us improve our questionnaire. See questions a-d, appearing in the body of this email (approximately 7 minutes). 

Attached, please find the questionnaire and an informed consent form. If you are willing to complete the questionnaire, please review and return the informed consent form via email (on the consent form, a typed signature is sufficient).

 

Using your input in this pre-test, we will revise and improve the questionnaire and use it with a broader group of potential respondents. 

 

Please let us know if you have any questions or concerns. Then: (1) let us know by email within about 2 days (by Tuesday, 3/14) whether you will participate in this pre-test and (2) either return the questionnaire or let us know that we may contact you to schedule a telephone call within about 2 weeks (before Friday, 3/24).


Thank you for your consideration of this request!


    1. How long did it take you to complete the questionnaire?

    2. Were the questions easy or difficult to understand? Please highlight your response.

Easy


Neither easy nor difficult


Difficult

1

2

3

4

5

Was the questionnaire’s format easy or difficult to follow? Please highlight your response.

Easy


Neither easy nor difficult


Difficult

1

2

3

4

5

    1. Were there any questions that you would add or remove from the questionnaire?

Exhibit A-2. Follow-up Recruitment Email for Pre-Test


(Sent electronically to potential pre-test participants if no response received within six business days of initial recruitment email)



Subject: A Questionnaire about FoodAPS User Experience (Pre-test)


Dear [name of contact person]:


I am writing to follow-up on a recent email about an invitation to pre-test a questionnaire on FoodAPS user experience. Would you be willing to participate in this pre-test?  

  

If yes, please send us a quick note by Wednesday, March 22 to confirm your participation. 


I am re-sending the informed consent form, questionnaire, and questionnaire feedback items. Please feel free to share with your collaborators if there are questions that might be better aligned with their user experience. 


Participation in this pre-test is voluntary and confidential. If you choose to participate, your input will help us refine our questionnaire before collecting feedback from a broader group of FoodAPS users. 


Thank you for your consideration of this request! 



Exhibit A-3. Draft Survey Questionnaire for Pre-Test


FoodAPS Users Questionnaire (Pre-test)

March 2017


Background


  1. What motivated you and your research team to use data from the National Household Food Acquisition and Purchase Survey (FoodAPS)?




  1. Which other datasets, if any, could you have used to answer your research question(s) if FoodAPS had not been available?




Data Usage


  1. Please consider the documentation materials you used or are using for your study. How well did these materials meet your needs?


Check or mark with X if used

Poorly

1


2

Neutral

3


4

Well

5

Comments (optional)

User’s Guide








Household-Level


Household Codebook








HH Geodata Codebook








Access Codebook








SNAP Eligibility Estimation Codebook








Individual-Level


Individual Codebook








Meals Codebook









FoodAPS Users Questionnaire (Pre-test)

March 2017


  1. (Continued) Please consider the documentation materials you used or are using for your study. How well did these materials meet your needs?


Check or mark with X if used

Poorly

1


2

Neutral

3


4

Well

5

Comments (optional)

Event and Item-Level


Places Codebook








FAH Events Codebook








FAH Items Codebook








FAH Item IRI Codebook








FAFH Supplementary Documentation








Event and Item-Level (continued)


FAFH Events Codebook








FAFH Items Codebook








Nutrient


FoodAPS Nutrient Coding Overview








FAH Nutrient Codebook








FAFH Nutrient Codebook








Geography Component


Retail Environment Study Codebook








Variable List: Retail Environment Study








Construction of Weekly Store-Level Food Basket Costs: Documentation













FoodAPS Users Questionnaire (Pre-test)

March 2017


  1. Please consider the data files you used or are using for your study. How well did these data files meet your needs?


Check or mark with X if used

Poorly

1


2

Neutral

3


4

Well

5

Comments (optional)

Household-Level Data


faps_household








faps_hhweights








faps_hhgeodata








faps_access








faps_snapelig_hh








faps_snapelig_unit








faps_snapelig_ind








Individual-Level Data


faps_individual








faps_reportstatus








faps_meals








Event and Item-Level Data


faps_places








faps_fahevent








faps_fahitem








faps_fahitem_iri








faps_fafhevent








faps_fafhitem








Nutrient Data


faps_fahnutrient








faps_fafhnutrient











FoodAPS Users Questionnaire (Pre-test)

March 2017


  1. (Continued) Please consider the data files you used or are using for your study. How well did these data files meet your needs?


Check or mark with X if used

Poorly

1


2

Neutral

3


4

Well

5

Comments (optional)

Geography Component Data








BGMain­_v6








BGRestaurant_v2








Tract_v5








County_v6








basketprices








basketprices_raw








Store Linker Files


PlaceID_IRI_TempERSID








PlaceID_TDLinx_TempERSID








PlaceID_TempERSID










User Experience


  1. Compared to your experience using similar datasets, how long did it take you to orient yourself with FoodAPS? Please consider the time you took to review documentation, learn about the data enclave, and any other preliminary tasks. Please highlight your response or mark it with an X.

    Less time


    Usual time


    More time

    1

    2

    3

    4

    5






  2. How would you rate the ease or difficulty of linking FoodAPS data files? Please highlight your response or mark it with an X.

Very easy


Neither easy nor difficult


Very difficult

1

2

3

4

5

FoodAPS Users Questionnaire (Pre-test)

March 2017


  1. How would you rate the ease or difficulty of applying survey weights to FoodAPS data? Please highlight your response or mark it with an X.

Very easy


Neither easy nor difficult


Very difficult

1

2

3

4

5


  1. What barriers, if any, did you encounter when using FoodAPS data?




Recommendations


  1. What are your suggestions, if any, for adjustments to be made for a possible next round of FoodAPS? For example, your response may consider sample size, groups for oversampling, content focus areas, questions to ask, etc.




  1. Are there any other comments you wish to share?




  1. Would you be willing to answer follow-up questions via telephone? The purpose of a follow-up telephone call would be to clarify answers provided in writing. Please highlight your response or mark it with an X.

Yes

No


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorMehreen Ismail
File Modified0000-00-00
File Created2021-01-22

© 2024 OMB.report | Privacy Policy