SUPPORTING STATEMENT A FOR:
FOOD REPORTING COMPARISON STUDY (FORCS)
AND
FOOD AND EATING ASSESSMENT STUDY (FEAST)
(NCI)
FORMERLY TITLED:
24-HOUR DIETARY RECALL METHOD COMPARISON STUDY &
VALIDATION AND OBSERVATIONAL FEEDING STUDY
OMB No. 0925-0605,
Expiry Date: 10/31/2011
July, 2011
Yellow highlights indicate changes from the 2009 submission.
Frances E. Thompson, Ph.D.
Applied Research Program,
Division of Cancer Control and Population Sciences
National Cancer Institute
EPN 4005
6130 EXECUTIVE BLVD MSC 7344
BETHESDA, MD 20892-7344
Telephone: 301-435-4410
E-mail: [email protected]
Table of Contents
A. Justification 1
A.1 Circumstances that Make the Collection of Information Necessary 1
A.2 Purpose and Use of the Information Collection 6
A.2.1 Research Questions 7
A.2.2 Audiences for Data and Results 8
A.2.3 Reason for Delay 9
A.3 Use of Improved Information Technology and Burden Reduction 9
A.4 Efforts to Identify Duplication and Use of Similar Information 10
A.5 Impact of Small Business and Other Small Entities 11
A.6 Consequences of Collecting the Information Less Frequently 11
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 11
A.8 Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency 12
A.9 Explanation of Any Payment or Gift to Respondents 13
A.10 Assurance of Confidentiality Provided to Respondents 14
A.11 Justification for Sensitive Questions 15
A.12 Estimates of Annualized Burden Hours and Costs 16
A.13 Estimates of Other Total Annual Cost Burden to Respondents and
Recordkeepers 20
A.14 Annualized Cost to the Federal Government 21
A.15 Explanation for Program Changes or Adjustments 21
A.16 Plans for Tabulation and Publication and Project Time Schedule 23
A.16. 1 Analysis of the 24HR Recall Comparison Study Data 23
A.16.2 Analysis of the Observational Feeding Study Data 23
A.16.3 Methods of Dissemination 23
A.17 Reason(s) Display of OMB Expiration Date Is Inappropriate 26
A.18 Exceptions to Certification for Paperwork Reduction Act Submission 26
REFERENCES 27
Tables
A.12-1 Estimates of annual burden hours 18
A.12-2 Annualized Cost to Respondents 20
A. 14-1 Annual Cost to the Federal government 21
A.16-A Data Collection Timetable 22
A.16-1 Research Questions and Outcome Measures for 24HR Recall Comparison Study 23
A.16-2 Research Questions and Outcome Measures for the validation and observational feeding study 24
List of Attachments
Automated Multiple Pass Method (AMPM) Screenshots*
Automated Self-Administered 24-hour Recall (ASA24) Screenshots*
Invitation letter for the Food Reporting Comparison Study
3A: Invitation letter for the Food Reporting Comparison Study
3B: Follow-up Invitation letter for the Food Reporting Comparison Study
3C: Evite letter for the Food Reporting Comparison Study
3D: Follow-up Evite letter for the Food Reporting Comparison Study
Screenshots of Food Reporting Comparison Study Information and Consent
4A: Screenshots of Food Reporting Comparison Study Information*
4B: Screenshot of Food Reporting Comparison Study Consent
Screening Script - Food Reporting Comparison Study*
Demographic and Health Questionnaire - Food Reporting Comparison Study*
Demographic, Health, and Preference Questionnaire - Food Reporting Comparison Study*
Screening Script - Food and Eating Assessment Study*
Study Information Letter - Food and Eating Assessment Study
Reminder Telephone Call Script - Food and Eating Assessment Study*
Consent form - Food and Eating Assessment Study
Demographic and Health Questionnaire - Food and Eating Assessment Study*
External Working Group Members
Thank You Letters - Food Reporting Comparison Study
14A: Group 1 and 4: Thank you for completing first dietary recall
14B: Group 2 and 3: Thank you for completing first dietary recall
14C: Group 3: Thank you with AMPM kit prior to second recall
14D: End of study thank you for those who completed only first recall
14E: End of study thank you for those who completed only second recall
14F: End of study thank you for those who completed first recall and survey
14G: End of study thank you for those who completed second recall and survey
14H: End of study thank you for those who completed both recalls and survey
Thank You Letter - Food Reporting Comparison Study
NIH Privacy Act Officer's Letter
Certification of Institutional Review Board Approval
18. Voicemail and recorded messages for completion of ASA24 for Food Reporting Comparison Study
19. Privacy Impact Assessment for FORCS and FEAST
* Indicates a questionnaire or telephone script.
JUSTIFICATION
A.1 Circumstances That Make the Collection of Information Necessary
The Public Health Service Act, Section 412 (42 USC § 285a-1) and Section 413 (42 USC § 285a-2) authorizes the National Cancer Institute (NCI) to establish and support programs “for the detection, diagnosis, prevention and treatment of cancer…”; and “to collect, identify, analyze and disseminate…information on cancer research, diagnosis, prevention, and treatment….”
The mission of the Risk Factors Monitoring and Methods Branch (RFMMB) is to contribute to reducing cancer in the US population by serving as a critical link between etiologic research on cancer risk factors and the translation of such research into targeted and effective interventions for prevention. One way of carrying out this mission is by developing and improving the methods of assessing the risk factors, which include dietary intake.
Currently, the interviewer-administered 24-hour dietary recall (24HR) is considered the best dietary data collection methodology primarily because it provides the highest quality and least biased food intake data for a single day (past 24 hours)1. The interviewer-administered protocol captures detailed information about dietary intake, including amount of specific food(s) consumed on a given day. Since 24HRs are conducted after the foods are eaten, subjects do not have the opportunity to change their eating habits for the reporting period. In addition, the recalls are conducted by highly trained professionals, within 24 hours of food consumption. The short time lag between food intake and recall minimizes memory and cognitive issues that afflict other dietary assessment methodologies. Repeat administration is needed to assess usual intake when using 24HR. The major limitation of using the 24HR is the high cost of administration and data collection. The methodology requires the use of trained interviewers and the complex coding of food items, which results in high cost per instrument completion.
Currently, 24HR data are collected using the Automated Multiple Pass Method (AMPM) (http://www.ars.usda.gov/Services/docs.htm?docid=7710), a computer-assisted five-step system designed by the United States Department of Agriculture (USDA)2 and used in the National Health and Nutrition Examination Survey (NHANES) to collect 24HR intake data from individuals3. In this system, respondents report their food consumption in the first three steps (reporting foods consumed, forgotten foods, and eating occasions). In the fourth step, detail-oriented questions and probes are carefully executed by the interviewer to elicit a thoughtful response. The fifth step provides one last opportunity to remember foods that were consumed (Attachment 1).
The web-based Automated Self-Administered 24-hour Recall (ASA24) transforms 24HR methodology into a convenient, self-administered, low-cost method of collecting dietary intake data. ASA24 draws its format and design from the AMPM but uses an automated approach (Attachment 2). Because it is a web-based, self-administered, and uses 24HR methodology, the ASA24 makes it feasible to collect multiple days of dietary intake data in large-scale epidemiologic studies, behavioral trials, or clinical research, and may help advance our understanding of the nutritional determinants of chronic diseases4. Use of a web-based, automated data collection system also offers the advantage of automated coding of food items and calculation of nutrient intakes. The intention is for the government to provide a free or low-cost instrument, publicly available to researchers and clinicians with a need for such an instrument. The test version of ASA24 can be viewed on-line at: http://asa24.westat.com/.
24HR Recall Comparison Study
The ASA24 evaluation involves two studies using an experimental design: a 24HR recall comparison study and an observational feeding study. The 24HR Recall Comparison Study will be referred to as the FOod Reporting Comparison Study (FORCS); the study website and the Observational Study will be referred to as the Food and Eating Assessment STudy (FEAST).
Self-reported dietary intake data from the new ASA24 method will be compared to self-reported dietary intake data from the standard AMPM method. A single day of 24HR data will be collected for two days (4 to 6 weeks apart), from a sample of 1080 men and women with diverse age and race/ethnicity, and in three geographical areas of the country. The sample will be drawn from three health maintenance organizations (HMO) ---Security Health Plan (using the Marshfield Clinic), Wisconsin; Henry Ford Health System, Michigan; and Northern California Kaiser-Permanente, California. These HMOs are part of NCI’s Cancer Research Network and thus are already participating in collaborative research and have critical research staff. Overall, sample respondents will be approximately equally divided among three age groups (20-34, 35-54, and 55-70). NCI will provide each HMO center staff with specifications on the required sample from each center (gender, age, and race/ethnicity mix). Each center will identify current users of their internet services, stratify all users into aforementioned sampling strata, randomly select individuals from each sampling stratum for recruitment, and mail invitation letters (Attachment 3) to potential study participants. Through the invitation letter, interested participants will be directed to a study website. With the Kaiser-Permanente site only, half of the potential respondents will receive the written letter as described, and half will receive instead an e-mail invitation with a direct link to the study website. The website will include information on study procedures and an on-line consent form (Attachment 4). Interested participants will provide their contact information through a secure on-line consent form (Attachment 4A, Screen 5). For participants who choose not to consent with FORCS a short survey will be given to find out their demographics and reason for not consenting (Attachment 4A, Screen 8). Westat, the contractor for this study, will monitor and track consents from this initial contact and provide the centers with a list of those who do not respond within 10 days of mail out. Each center will follow-up with these non-responders and send a second mailing. Westat will continue to track consents until the quota for each stratum is met. Westat will follow-up with individuals who have consented to participate and administer by telephone, a screening questionnaire (Attachment 5) to ascertain eligibility for the study. Those deemed eligible will be randomly assigned by sampling stratum to one of the following four experimental study groups:
Group 1 – Complete two ASA24 self-administered recalls, 4 to 6 weeks apart.
Group 2 – Complete two AMPM telephone-administered interviews, 4 to 6 weeks apart.
Group 3 – Complete one ASA24 followed by one AMPM, 4 to 6 weeks apart.
Group 4 – Complete one AMPM followed by one ASA24, 4 to 6 weeks apart.
All recalls will be unannounced, i.e. without prior scheduling, so as to avoid the potential of changing diets for the reporting day. Participants completing the ASA24 will receive pre-recorded message reminding them to check their email and complete the ASA24 recall online (Attachment 18). Once participants have completed both dietary recalls, they will be directed to complete the on-line Demographic and Health Questionnaire (Attachment 6). Participants assigned to Group 3 and 4 will also complete an on-line Demographic, Health and Preference questionnaire (Attachment 7), indicating their preference for either AMPM or ASA24.
The NCI Validation and Observational Feeding Study
Researchers will unobtrusively weigh the food intake of study participants, and compare the measured intake to self-reported intake using the AMPM (Attachment 1) vs. the ASA24 (Attachment 2) on the next day. Dietary data will be collected from 80 different individuals over a two-day period. Subjects will be drawn from a dataset of individuals who have volunteered to be in research projects. A pool of potential participants will be identified by EurekaFacts, a research firm, who will recruit the sample matching the specifications for age and gender. To the extent possible, we will enroll subjects who have not yet participated in research studies. A screening questionnaire (Attachment 8) will be administered to exclude subjects with dietary allergies, practices or preferences that would interfere with the protocol. Subjects will be excluded if they do not have high-speed internet at home. This type of computer connection is required for the ASA24 and therefore necessary for research studies employing the ASA24. This exclusion is required for inference of results to the research community. Subjects who have previously been involved in a research study will be excluded. If sufficient numbers of subjects cannot be drawn with the last criterion, then subjects who have not been involved in a study for over one year and never in a nutrition-related study will be sampled.
Once participants are selected, a letter containing information on the study (FAQs), their appointment dates, and a map to the facility will be mailed to them (Attachment 9). Participants will be scheduled to come to the research facility at Westat in Rockville, Maryland, for two consecutive days. Westat will bring in 9 people per day and 3 at a time may eat together in a social setting but will obtain their food separately so the amount taken can be ascertained. There will be a reminder telephone call the night before the first appointment (Attachment 10). On the first day, participants will sign the informed consent (Attachment 11) and then consume three meals throughout the day. On the second day of the study, participants will return to Westat and report their preceding day’s dietary intake, using either the AMPM or the ASA24. Once participants have completed the dietary interview, they will be directed to complete the on-line Demographic and Health Questionnaire (Attachment 12).
The current proposal builds from previous work in developing and evaluating the ASA24. Multiple small-scale cognitive and usability tests were included in the development of the ASA24 application from its onset. Testing has been conducted in eight different settings, either as individual cognitive testing or using focus groups of nine or less individuals. Testing addressed issues such as validity of portion size presentation options, meal-based vs. unstructured food list, design and format of various passes, and the general ease of use. Necessary adjustments were made to the application based on the feedback from testing5 6.
The purpose of this study is to compare the newly developed web-based Automated Self Administered 24-hour Recall (ASA24) (Attachment 2) to the current standard of interview-administered 24-hour recall, the Automated Multiple Pass Method (AMPM) (Attachment 1). The ultimate intent of this study is to provide evaluative information to prospective users in their decision to use the new ASA24 method. The ASA24 method would offer a low-cost alternative to the AMPM method, and thus could allow a wider use of 24HR methodology within existing resources.
Study 1 (24HR Recall Comparison Study; FORCS):
How does reported food and nutrient intake compare between the ASA24 and the AMPM?
What is the completion percentage of the ASA24 at the first stage of data collection, and how does this compare to the analogous completion percentage for the AMPM? What is the completion percentage of the ASA24 at the second stage of data collection (i.e. 1 – attrition rate), and how does this compare to the analogous completion percentage for the AMPM?
Is there a methodology preference for reporting dietary intake among individuals who reported their dietary intake using both the ASA24 and the AMPM approaches?
Study 2 (NCI Validation and Observational Feeding Study; FEAST):
How does reported food intake using the ASA24 compare to observed and measured intake?
How does reported food intake using the AMPM compare to observed and measured intake?
Is there a difference in reporting bias between the two instruments?
The combined findings from the 24HR recall comparison study and the observational feeding study will inform the public health community about the feasibility and validity of the ASA24 relative to the standard AMPM methodology. Because the ASA24 offers significant cost savings, many potential users will be interested in these evaluative results. Currently, scores of inquiries have been made concerning the ASA24. These inquiries have come from U.S. federal partners who collect national level dietary intake data, academic researchers, clinicians, and other public health providers, not only from the U.S. but also from Canada, Great Britain, and other European Union countries. Summary reports, articles, and presentations will be disseminated through multiple methods (See Section A.16.3 for more detail).
The ASA24 respondent application is currently available as a beta version only. There has been a delay in the completion of a fully functional Version 1 due to a contractual conflict over intellectual property rights with the subcontractor who designed the graphic user interface. A virtual work stoppage occurred for approximately a year as legal and contractual issues were discussed with contracts staff, legal counsel, and contractors. Ultimately, NCI decided to develop a new graphical user interface for the respondent application. This work began in August of 2010 with a new contract in place. Progress has been substantial and continuous. Version 1, which is ultimately the ASA24 tool that will be available to the public in the future, will be free of the bugs in the beta version and will offer improved functionality and features, such as additional optional modules (e.g., supplement intake) and a Spanish language option. It is expected to be available in the summer of 2011. Ideally, fielding of the validation studies would begin in September 2011. To date, no participants have been recruited for these studies.
In the meantime, study materials were further refined and programming was implemented to track study respondents. Currently, we are completing the programming tasks needed prior to fielding the study. There are no substantive changes in the study protocols.
The participant materials were revised for two reasons (1) we customized the web pages and participant materials to include the website and study name, selected after OMB approval; (2) we simplified the language to reflect lower grade reading level, per IRB guidance. All revised materials are included as attachments in this submission.
On the FORCS study, the current OMB submission includes the voicemail message as well as the automated voice recording reminders for participants completing the ASA24 recall (Attachment 18). We also received approval for the domain name: http://www.FoodReportStudy.gov and established a toll free number for participants to leave a message with questions for the project staff.
On the FEAST study a telephone reminder script has been added. Minor revisions were made to the screening script and the study information letter; these scripts and letters also contain the toll-free telephone number described for the Comparison Study, to enable FEAST study participants to leave a voice mail message for the project staff. Additionally, the participant incentive has been increased due to concerns about respondent burden and attrition.
A.3 Use of Improved Technology and Burden Reduction
In the AMPM approach, respondents provide data either in-person or over the telephone and trained interviewers use a computerized system to collect and enter data. The ASA24 collects intake data directly from the respondent without an interviewer. The ASA24 uses state-of-the-art automated computer technology, including graphic enhancements and animated characters to guide participants, and audio language/cues. In addition, the ASA24 software includes pictures of foods in multiple portion sizes to aid portion size estimation by the respondent. The software has the capacity to immediately compute nutrient and food group estimates for each recall day. A computerized Study Management System (SMS) will be used to manage study activities and track completion of all study activities to allow for monitoring and calculation of response rates.
A Privacy Impact Assessment (PIA) was completed and published by HHS in January, 2009. The IT systems were divided into two systems and are named, “NIH NCI Automated Self-Administered 24-Hour Recall (ASA24)” and the “NIH NCI Automated Self-Administered 24-Hour Recall (ASA24) Researcher Website.”
A.4 Efforts to Identify Duplication and Use of Similar Information
ASA24 is currently in use in the NIH-AARP (formerly American Association of Retired Persons) interactive Comprehensive Lifestyle Interview by Computer study (iCLIC: OMB# 0925-0594). This is a feasibility study and formative evaluation of the quality and completeness of four self-administered web-based instruments to be used in phase II of the NIH-AARP Diet and Health Study. The study will assess completion rates for the ASA24 on the first and second times, two months apart. However, the iCLIC study does not address validation of the ASA24 to the recognized standard for conducting 24HR--- the AMPM method—with standard diet. This feeding study will address the issue of validity for nutritional epidemiologic researchers.
To our knowledge, ASA24 is the only web-based, self-administered method that has been developed for collection of complete 24HR data for medical, academic and government purposes; ASA24 has yet to be released for public use. Other websites and newer technologies attempt to collect 24hr recalls but none have the multiple passes, detailed probes and highly specific food designations that support complete reporting and more accurate estimates of daily intakes. The 24HR recall comparison and observational feeding studies will be the first to evaluate dietary intake using ASA24 and compare it with the highly regarded standard of the AMPM.
A.5 Impact of Small Business and Other Small Entities
No small entities will be involved in this survey. All respondents will be individuals who participate voluntarily.
Participants will be asked to complete dietary questionnaires either once or twice within a six week period depending on their study assignment. The 24HR comparison study will compare the reported food and nutrient intake of four assigned study groups; AMPM only, ASA24 only, ASA24 followed by AMPM, AMPM followed by ASA24. Participants will complete assigned questionnaires at two time points four to six weeks apart. Feeding study participants will complete either the AMPM or ASA24 once. Reported intakes will be compared to observed and measured intakes and differences in reporting bias between the two instruments will be determined. Collecting information less frequently would not answer the research questions of this study.
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are two special circumstances that would cause this information collection to be conducted in a manner inconsistent with 5 CFR 1320.5. First, respondents in Groups 1 and 2 are required to answer the same questionnaires twice within 4 to 6 weeks. The need for repeat measurements is due to high day-to-day variability in individuals’ diets. Statisticians require a minimum of 2 measurements in order to be able to control for day-to-day variability in the analyses. The timing of these 2 measurements is critical. Since individuals tend to eat differently in different seasons (e.g. more hot dogs in the summer), it is important to collect for each individual the 2 measurements in the same season. On the other hand, if the days of report are too close together, there is a danger of the individual eating a similar diet (e.g., leftovers) and thus not providing reports of a fuller range of his/her diet. Thus, the 4-6 weeks interval between measurements was chosen, and is feasible.
Second, a limitation of the study is that findings will not be generalizable to the entire U.S. adult population, but only to population subgroups with access to high-speed internet. The ASA24 can only be completed with high-speed internet and this is a limitation for this study. The usual interviewer-administered 24HR will always be required for those with literacy or other limitations. Currently, it is estimated that 75% of Americans have access to the internet. Of those with a home computer, 85% have high-speed internet. This proportion can only grow in the future. In summary, the experimental design proposed will allow comparison of the ASA24 and AMPM in a sample of those with the required access to high-speed internet.
A.8 Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency
The 60-Day Federal Register notice soliciting comments on this study prior to initial submission to OMB was published on April 15, 2011 (76 FR 21383). One public comment was received on April 15 requesting a copy of the data collection package. The submission was sent to the requestor on April 21. ASA24 was developed through NCI’s collaboration with: Westat Inc., Baylor College of Medicine, Archimage Inc., a software developing company, and the U.S. Department of Agriculture (USDA). During the development of ASA24, an External Working Group met twice (Nov. 4, 2005; Aug. 23, 2006) to discuss the needs and interests of potential users and provide advice to the project (Attachment 13).
A.9 Explanation of Any Payment or Gift to Respondents
To maximize participation, many research studies offer compensation in studies of the public where there is no link to health or disease outcomes. Incentives have been shown to increase response to web-based questionnaires7. Both the FORCS and the FEAST studies are using an experimental design to compare different treatments in matched groups. Complete participation is optimal for useful information; thus an important objective is to support continuing and complete participation in the study protocols.
Participants of the FORCS will be offered $52.00 in four installments through postal mail. The initial mailed letter will include a $2 incentive to encourage traffic to the study website. Following consent, establishment of eligibility and assignment into study group, participants will receive in a timely manner, a second incentive of $5. Those assigned to complete the AMPM, will also receive measuring guides to aid with the recall. After completion of the first 24HR, participants will receive an additional $15. After completion of the second 24HR, the Demographic and Health Questionnaire and for groups 3 and 4, the Demographic, Health and Preference Questionnaire participants will receive $30 and a thank you letter (Attachment 14).
Participants in the observational feeding study will have breakfast, lunch, and dinner provided for them on the first study day, and will be offered $40 at the end of dinner on that day. Participants will be thanked and receive an additional $80 after completion of the dietary interview and Demographic and Health Questionnaire (Attachment 12).
A.10 Assurance of Confidentiality Provided to Respondents
Participants in this study will be subject to assurances and safeguards as provided by the Privacy Act of 1974 (5 USC 552a), which requires the safeguarding of individuals against invasion of privacy. The Privacy Act allows information maintained by a Federal agency according to either the individual’s name or some other identifier will be kept private. This information collection is covered by NIH Privacy Act Systems of Record 09-25-0156, “Records of Participants in Programs and Respondents in Surveys Used to Evaluate Programs of the Public Health Service, HHS/PHS/NIH/OD.” All persons working with ASA24 studies will adhere to the provisions stipulated within that announcement (see Attachment 16 for Privacy Act Officer’s Letter). In accordance with the Privacy Act of 1974, the privacy of individual respondents will be protected. The data sets created will contain no means of identifying individual respondents. The following describes the measures taken to protect the privacy of the participants.
After respondents have provided consent, they will be able to review an agreement memo, assuring that all collected information will be kept private and not disclosed in any identifiable form to anyone but the researcher conducting this study, except as otherwise required by law. IRB documentation is provided as an attachment (Attachment 17) to the clearance package.
Security protocols will be implemented to ensure that all data are recorded and stored in such a manner that individual research subjects cannot be identified directly or through identifiers. Each questionnaire will include a unique ID number for each respondent, but only the data management contractor will have the secure database to link ID numbers with individuals. No identifying information will be recorded in the data file and there will be no way to detect the identification of any respondent. After the data collection is completed, the contractor will store the paper informed consent forms (for the observational feeding study) and paper questionnaires in a locked, secure facility for a year, and then they will be shredded. Electronic data will be password protected and stored by the data management contractor, and also will be destroyed after a year.
A.11 Justification for Sensitive Questions
Sensitive questions are defined as those whose answers, if made public, could cause physical, mental, emotional, economic, or other harm to an individual. Participants will be asked to provide their year of birth, gender, and race/ethnicity information as part of the enrollment process (Attachment 4 – Screens 5 and 8). Additionally, there are questions on the Demographic and Health Questionnaire that are considered sensitive as they relate to use of alcohol, cigarettes, and income (Attachments 6 and 12). Personally identifiable information (PII) is collected in the form of the respondent’s name, address and phone number which is needed to contact potential respondents.
In the FORCS, respondents are identified by three health maintenance organizations (HMO). In the FEAST, a subcontractor will identify and administer the screening questionnaire to the respondents. This information is important to recruit a sample from diverse backgrounds. Participation in the study is voluntary and participants have the right not to answer any questions without consequences. Section A.10 discusses the steps taken to safeguard this information.
A.12 Estimates of Annualized Burden Hours and Costs
Data collection activities for all participants involve completion of a telephone screener and Demographic and Health Questionnaire; participants in the FORCS will complete two 24HR; participants in the two groups using both ASA24 (Attachment 2) and AMPM (Attachment 1) will complete an additional preference questionnaire (Attachment 7); participants in the FEAST will complete one 24HR. The burden estimates including annualized hourly costs and the total estimated burden for the FORCS and FEAST are summarized in Table A.12.1 and A.12.2.
24HR Recall Comparison Study (FORCS)
After recruitment, participants in the 24HR recall comparison study will complete a brief (5 minutes) screener questionnaire (Attachment 5). Those enrolled in the study will be randomized to one of four groups; each respondent will complete two 24HR lasting approximately 30 minutes each, 4 to 6 weeks apart:
Group 1 will complete two ASA24 recalls;
Group 2 will complete two AMPM recalls;
Group 3 will complete one ASA24 followed by one AMPM; and
Group 4 will complete one AMPM followed by one ASA24.
On completion of the second recall, Groups 1 and 2 participants will be instructed to complete the on-line Demographic and Health Questionnaire (10 minutes). Groups 3 and 4 participants will complete the on-line Demographic, Health and Preference Questionnaire (15 minutes). Non-response to the web-based surveys will trigger a phone interview to obtain the needed data.
NCI Validation and Observational Feeding Study (FEAST)
Participants in the FEAST will complete a brief (5 minutes) screener (Attachment 8) prior to participating in the study. Study respondents will be invited to a central location for three meals. There will be a reminder telephone call the night before the first appointment (Attachment 10). Participants will be offered a variety of foods to choose from and their observed food consumption will be recorded by computing the difference in measured weight of each food taken on the plate and remaining after eating. The measurements will be taken in an unobtrusive manner. Participants will be asked to return to the central location the day after the event, during which time they will participate in a 24HR, conducted using either AMPM or ASA24, and complete a Demographic and Health Questionnaire on-line. The estimated time for consuming three meals is 135 minutes and time for completing the 24HR is about 30 minutes; the estimated time for completing the Demographic and Health Questionnaire is 10 minutes. The goal is to have 80 respondents complete the 24HR after meal consumption. Though the data collection is anticipated to be completed within a 12 month period, this request is for a 3-year period in the event there are unanticipated delays, additional questions or changes that will be requested and taken into account.
The total estimate of respondent burden is 2598 hours for the FORCS and the FEAST over the 3-year information collection period. This amounts to an annualized estimate of respondent burden to be 867 hours for both studies (see Table A.12-1). The 2009 submission accounted for participant attrition by decreasing the estimated participants who would complete the study. Though participant attrition is anticipated it is difficult to estimate the exact levels of attrition and as a result Table A.12-1 has been adjusted to accommodate the maximum number of participants, assuming that all participants will complete the study. Additionally, the burden has decreased slightly since last submission as a result spreading the same number of anticipated participants over a period of 3 years, rather than 2 years (which was the time frame requested in the 2009 submission).
Table A.12-1 Estimates of Annual Burden Hours
Participants and Study |
Questionnaire |
Number of respondents |
Frequency of response |
Average time per response Minutes/Hour |
Annual hour burden |
General Public for FORCS |
Refusal
Reasons and Demographics |
1770 |
1 |
5/60 (0.083) |
148 |
Contact
Information |
400 |
1 |
5/60 (0.083) |
33 |
|
Screener (Attach 5) |
400 |
1.00 |
5/60 (0.083) |
33 |
|
AMPM (Attach 1) |
400 |
1.00 |
30/60 (0.50) |
200 |
|
ASA24 (Attach 2) |
400 |
1.00 |
30/60 (0.50) |
200 |
|
Demographics and Health Questionnaire (Attach 6) |
360 |
1.00 |
10/60 (0.167) |
60 |
|
Demographics, Health and Preference Questionnaire (Attach 7) |
360 |
1.00 |
15/60 (0.25) |
90 |
|
General Public for FEAST |
Screener (Attach 8) |
33 |
1.00 |
5/60 (0.083) |
3 |
Reminder Telephone Call (Attach 10) |
33 |
1.00 |
5/60 (0.083) |
3 |
|
Eating 3 meals |
33 |
1.00 |
135/60 (2.25) |
74 |
|
Either AMPM or ASA24 (Attach 1 or 2) |
33 |
1.00 |
30/60 (0.50) |
17 |
|
Demographics and Health Questionnaire (Attach 12) |
33 |
1.00 |
10/60 (0.167) |
6 |
|
|
|
4255 |
|
|
867 |
The cost burden to respondents is essentially the time required to read the instructions and sign the consent form, and complete the screener, dietary recalls, demographics, and preference questions. The cost to the respondents for the total burden is estimated to be $54,293 over a 3-year period. The annualized cost to the respondents is estimated to be $18,098 for the FORCS and FEAST studies, calculated at $20.90 per hour (based on the mean hourly wage estimates for all occupations from the U.S., Department of Labor, Bureau of Labor Statistics, May 2009). The costs are summarized in Table A.12-2.
Table A.12-2 Annualized Cost to Respondents
Participants and Study |
Questionnaire |
Number of respondents |
Annual hour burden |
Hourly Wage Rate |
Respondent Cost |
General Public for FORCS |
Refusal
Reasons and Demographics |
1770 |
83.33 |
20.90 |
$3082.75 |
Contact
Information |
400 |
33.33 |
20.90 |
$696.67 |
|
Screener (Attach 5) |
400 |
33.33 |
20.90 |
$696.67 |
|
AMPM (Attach 1) |
400 |
200.00 |
20.90 |
$4,180.00 |
|
ASA24 (Attach 2) |
400 |
200.00 |
20.90 |
$4,180.00 |
|
Demographics and Health Questionnaire (Attach 6) |
360 |
60.00 |
20.90 |
$1,254.00 |
|
Demographics, Health and Preference Questionnaire (Attach 7) |
360 |
90.00 |
20.90 |
$1,881.00 |
|
General Public for FEAST |
Screener (Attach 8) |
33 |
2.75 |
20.90 |
$57.48 |
Reminder Telephone Call (Attach 10) |
33 |
2.75 |
20.90 |
$57.48 |
|
Eating 3 meals |
33 |
74.25 |
20.90 |
$1,551.83 |
|
Either AMPM or ASA24 (Attach 1 or 2) |
33 |
16.50 |
20.90 |
$344.85 |
|
Demographics and Health Questionnaire (Attach 12) |
33 |
5.50 |
20.90 |
$114.95 |
|
|
|
4255 |
865.92 |
|
$18,097.66 |
A.13 Estimates of Other Total Annual Cost Burden to Respondents and Recordkeepers
There are no other costs to respondents beyond those presented in section A.12. There are no operating, maintenance, and capital costs associated with the 24HR recall comparison study and the observational feeding study data collection.
A.14 Annualized Cost to the Federal Government
The largest cost to the federal government is to pay a contractor $963,973 to conduct the study and deliver data files. This is based on an estimate of 4,666 hrs for salaried labor (salary range from $21-50 per hour), 10,655 hrs for hourly office labor (salary range from $12-40 per hour), and 596 hrs for offsite salaried labor (salary range $35-42 per hour), the total cost of which is $341,878 prior to overhead and other expenses. In addition the contractor estimates other direct costs of $205,169 for computing, copying, supplies, postage/shipping, miscellaneous items, and incentives. Finally, the contractor has additional fees for overhead (50-100%), general/administration (16.7%) and fixed fees (6.0%).
NCI costs are based entirely on labor. It is estimated that the study will require about 0.5 FTE total per year spread over 4-5 scientists (nutritionists, epidemiologists, statisticians) at the GS14 level or above, totaling $125,000, These expenses are related to directing contractors, overseeing and solving problems as they arise, developing materials, supervising data collection, data coding, data cleaning, data analyses, and preparation of manuscripts and presentations.
Finally, the government estimates about $6,000 for data cleaning and analyses via use of a separate biomedical computing support contract.
In summary, based on the current budget, the estimated overall cost to the Federal Government for the 24HR recall comparison and observational feeding studies is $1,094,973 for 24 months. Thus, the annualized cost to the federal government is $547,486.50.
Table 14-1 Annual Cost to the Federal Government
|
TOTAL |
ANNUAL AVERAGE |
Contractor Costs |
$963,973 |
$481,986.50 |
NCI Personnel Subtotal |
$125,000 |
$62,500.00 |
Analysis |
$6,000 |
$3,000.00 |
Grand Total |
$1,094,973 |
$547,486.50 |
A.15 Explanation for Program Changes or Adjustments
This adjustment is an extension of the previously approved project. The ASA24 Respondent application is currently available as a beta version only. There has been a delay in the completion of a fully functional Version 1 due to a contractual conflict over intellectual property rights with a subcontractor who designed the graphic user interface. A virtual work stoppage occurred for approximately a year as legal and contractual issues were discussed with contracts staff, legal counsel, and contractors. Ultimately, NCI decided to develop a new graphical user interface for the respondent application. This work began in August of 2010 with a new contract in place. Progress has been substantial and continuous. Version 1, which is ultimately the ASA24 tool that will be available to the public in the future, will be free of the bugs in the beta version and will offer improved functionality and features, such as additional optional modules (e.g., supplement intake) and a Spanish language option is expected to be available in the summer of 2011.
The burden has decreased slightly since last submission as a result spreading the same number of anticipated participants over a period of 3 years, rather than 2 years, which was the time frame requested in the 2009 submission.
The 24HR recall comparison and observational feeding studies will begin within 3 months of obtaining OMB approval. The contract period will include fielding, analyzing, and disseminating findings from these studies. The contractor will be responsible for preparing the analytic databases resulting from the two studies. The timetable for the data collection for the 24HR recall comparison and observational feeding studies is shown below, in Table A.16-A.
Table A.16-A. Data Collection Timetable
Study activity |
Months after OMB approval |
24HR recall comparison study
|
|
HMO centers will mail invitation letters to members |
month 3 |
On-line consent from respondents |
months 3 - 5 |
Westat to conduct telephone screening |
months 3 - 6 |
Westat to mail out measuring guide to two study groups and initial incentive to all four study groups. |
months 3 - 9 |
First round of data collection begins (both modes) |
months 4 - 5 |
Second round of data collection begins |
months 5 - 6 |
Demographic survey follow-up |
months 6 - 7 |
Preference questionnaire |
months 6 - 7 |
Data processing and analysis |
months 8 - 24 |
Validation and Observational feeding study
|
|
Recruitment |
month 3 |
Screening |
months 3 - 4 |
Begin Feeding (and consent) |
months 4 - 5 |
Begin 24HR (24HR after feeding day) |
months 4 - 5 |
Data processing and analysis |
A.16.1 Analysis of the 24HR Recall Comparison Study Data
Statistical analysis will be conducted to compare the nutrient and food group estimates obtained from the ASA24 and the AMPM approach. All data from the two studies will be analyzed by the NCI survey methodologists and statistical staff working on this study. Table A.16-1 and A.16-2 present the research questions and analysis that will be conducted.
Table A.16-1 Research Questions and Outcome Measures for 24HR Recall Comparison Study
Research Questions |
Analysis |
How does reported food and nutrient intake compare between
the ASA24 and the AMPM?
|
The median intake reported on the ASA24 will be compared to the median intake reported on the AMPM. Due to the skewness inherent in dietary intake data, comparisons will be made on a transformed scale, where the median corresponds to the mean.
|
Sample size determinations are motivated by trying to detect whether the median in the original scale for one instrument is within 90% or 95% of the median in the original scale for the other instrument.
|
|
For some nutrients, we expect sufficient sample size to investigate differences across gender or race-ethnicity domains, but for other nutrients and foods, only an overall comparison can be performed. |
|
What is the completion percentage of the ASA24 at the first
stage of data collection,
|
The fraction of respondents who complete the first application of the ASA24 (first-stage completion rate) will be compared to the analogous fraction for the AMPM. |
What is the completion percentage of the ASA24 at the second stage of data collection (i.e. 1 – attrition rate), and how does this compare to the analogous completion percentage for the AMPM? |
The fraction of respondents who complete the second application of the ASA24 will be compared to the analogous fraction for the AMPM to assess second-stage completion rate differences (equivalent to attrition rate differences). Sample size permitting, differences in these fractions will be compared across gender and race/ethnicity domains.
|
Is there a methodology preference for reporting dietary intake among individuals who reported their dietary intake using both the ASA24 and the AMPM approaches?
|
The fractions of respondents who prefer ASA24 over the AMPM will be computed for those individuals who complete both instruments.
Sample size permitting, differences in this fraction will be compared across gender and race/ethnicity domains. |
The research questions and the measures to be used in the analysis are shown in Table A.16-2 below.
Table A.16-2 Research Questions and Outcome Measures for the Validation and Observational Feeding Study
Research Questions |
Analysis |
How does reported food intake using the ASA24 compare to
observed intake? |
The consistency of the food intake reports between the ASA24 and
observed intake will be examined, as will the consistency of food
intake reports from the AMPM and observed intake.
Some foods are more
difficult than others to quantify. Pieces of foods such as fish
fillets, slices of meat & chops may be easier to quantify
than amorphous foods such as mashed potatoes. Food
quantification accuracy will be evaluated for individual foods
and groups of food item types (i.e., piece, amorphous, etc).
|
A.16.3 Methods of Dissemination
Findings from the 24HR recall comparison and observational feeding studies will be disseminated through multiple methods, including summary reports available in electronic and hard copy format. These summary reports will also be publicly accessible through the National Cancer Institute’s Applied Research Program website: http://appliedresearch.cancer.gov/, which contains information on the ASA24. In addition, NCI staff will analyze the data and prepare presentations for national conferences and publish articles in peer-reviewed journals (in conjunction with other researchers). The NCI staff will work within NCI and with other federal agencies (e.g., USDA) to disseminate the results.
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
The 24HR recall comparison and observational feeding studies will not require exemption from displaying the expiration date of OMB approval. Any reproduction of the data collection instrument will prominently display the OMB approval number and expiration date.
A.18 Exceptions to Certification for Paperwork Reduction Act Submission
The 24HR recall comparison and observational feeding studies do not require any exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).
References
1Thompson FE, Subar AF. Dietary Assessment Methodology. In: Coulston AM, Boushey CJ, eds. Nutrition in the Prevention and Treatment of Disease. San Diego: Academic Press, 2008.
2 U.S. Department of Agriculture. USDA automated multiple-pass method (Beltsville, MD), 2005. http://www.ars.usda.gov/Services/docs.htm?docid=7710
3 National Center for Health Statistics. National Health and Nutrition Examination Survey, 2009. http://www.cdc.gov/nchs/about/major/nhanes/nhanes2005-2006/questexam05_06.htm
4 Schatzkin A, Subar AF, Moore S, et al. Observational Epidemiologic Studies of Nutrition and Cancer: The Next Generation (with Better Observation). Cancer Epidemiol Biomarkers Prev 2009;18:1026-32.
5 Subar AF et al. Formative research of a quick list for an automated self-administered 24-hour dietary recall. J Am Diet Assoc. 2007 Jun; 107(6):1002-7.
6 Subar AF et al. Accuracy of portion size reports using computer-based food photography aids in the development of an automated self-administered 24-hour recall (ASA24). J Am Diet Assoc 2010; 110:55-64.
7 Bowling JM, Rimer BK, Lyons EJ, et al. Methodologic challenges of e-health research. Evaluation and Program Planning 2006; 29:390-6.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |