SUPPORTING STATEMENT A FOR:
24-HOUR DIETARY RECALL METHOD COMPARISON STUDY
AND
NATIONAL CANCER INSTITUTE (NCI) VALIDATION AND OBSERVATIONAL FEEDING STUDY
National Cancer Institute
8/3/2009
Frances E. Thompson, Ph.D.
Applied Research Program,
Division of Cancer Control and Population Sciences
National Cancer Institute
EPN 4005
6130 EXECUTIVE BLVD MSC 7344
BETHESDA, MD 20892-7344
Telephone: 301-435-4410
E-mail: [email protected]
Table of Contents
A. Justification 1
A.1 Circumstances that Make the Collection of Information Necessary 1
A.2 Purpose and Use of the Information 6
A.2.1 Research Questions 6
A.2.2 Audiences for Data and Results 7
A.3 Use of Improved Technology and Burden Reduction 8
A.4 Efforts to Identify Duplication and Use of Similar Information 8
A.5 Impact of Small Business and Other Small Entities 9
A.6 Consequences of Collecting the Information Less Frequently 9
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 10
A.8 Comments in Response to Federal Register Notice and Efforts to Consult Outside the Agency 11
A.9 Explanation of Any Payment or Gift to Respondents 11
A.10 Assurance of Confidentiality Provided to Respondents 12
A.11 Justification for Sensitive Questions 13
A.12 Estimates of Annualized Burden Hours and Costs 14
A13 Estimates of Other Total Annual Cost Burden to Respondents and
Recordkeepers 17
A.14 Annualized Cost to the Federal Government 18
A.15 Explanation for Program Changes or Adjustments 19
A.16 Plans for Tabulation and Publication and Project Time Schedule 19
A.16.1 Analysis of Pilot Data 20
A.16.2 Analysis of Main Survey 22
A.16.3 Methods of Dissemination 22
A.17 Reason(s) Display of OMB Expiration Date Is Inappropriate 23
A.18 Exceptions to Certification for Paperwork Reduction Act Submission 23
REFERENCES 24
Tables
A.12-1 Estimate of Respondent Hour Burden 16
A.12-2 Annualized Cost to Respondents 17
A. 14-1 Annual Cost to the Federal government 18
A.16-A Data Collection Timetable 19
A.16-1 Research Questions and Outcome Measures for 24HR Recall Comparison Study 21
A.16-2 Research Questions and Outcome Measures for the NCI Feeding Study 22
List of Attachments
AMPM Screenshots
ASA24 Screenshots
Invitation letter for the 24HR Recall Comparison Study
Screenshots of 24HR Recall Comparison Study Information and Consent
Screening Script - 24HR Recall Comparison Study
Demographic and Health Questionnaire - 24HR Recall Comparison Study
Preference Questionnaire - 24HR Recall Comparison Study
Screening Script - Observational Feeding Study
Study Information Letter - Observational Feeding Study
Reminder Telephone Call Script - Observational Feeding Study
Consent form - Observational Feeding Study
Demographic and Health Questionnaire - Observational Feeding Study
External Working Group Members
Thank You Letter - 24HR Recall Comparison Study
Thank You Letter - Observational Feeding Study
NIH Privacy Act Officer's Letter
Certification of Institutional Review Board Approval
JUSTIFICATION
A.1 Circumstances That Make the Collection of Information Necessary
The Public Health Service Act, Section 412 (42 USC § 285a-1) and Section 413 (42 USC § 285a-2) authorizes the National Cancer Institute (NCI) to establish and support programs for the detection, diagnosis, prevention and treatment of cancer; and to collect, identify, analyze and disseminate information on cancer research, diagnosis, prevention, and treatment.
The mission of the Risk Factors Monitoring and Methods Branch (RFMMB) is to contribute to reducing cancer in the US population by serving as a critical link between etiologic research on cancer risk factors and the translation of such research into targeted and effective interventions for prevention. One way of carrying out this mission is by developing and improving the methods of assessing the risk factors, which include dietary intake.
Currently, the interviewer-administered 24-hour dietary recall (24HR) is considered the best dietary data collection methodology primarily because it provides the highest quality and least biased food intake data for a single day (past 24 hours)1. The interviewer-administered protocol captures detailed information about dietary intake, including amount of specific food(s) consumed on a given day. Since 24HRs are conducted after the foods are eaten, subjects do not have the opportunity to change their eating habits for the reporting period. In addition, the recalls are conducted by highly trained professionals, within 24 hours of food consumption. The short time lag between food intake and recall minimizes memory and cognitive issues that afflict other dietary assessment methodologies. Repeat administration is needed to assess usual intake when using 24HR. The major limitation of using the 24HR is the high cost of administration and data collection. The methodology requires the use of trained interviewers and the complex coding of food items, which results in high cost per instrument completion.
Currently, 24HR data are collected using the Automated Multiple Pass Method (AMPM) (http://www.ars.usda.gov/Services/docs.htm?docid=7710), a computer-assisted five-step system designed by the United States Department of Agriculture (USDA)2 and used in the National Health and Nutrition Examination Survey (NHANES) to collect 24HR intake data from individuals3. In this system, respondents report their food consumption in the first three steps (reporting foods consumed, forgotten foods, and eating occasions). In the fourth step, detail-oriented questions and probes are carefully executed by the interviewer to elicit a thoughtful response. The fifth step provides one last opportunity to remember foods that were consumed (Attachment 1).
The newly developed web-based Automated Self-Administered 24-hour Recall (ASA24) transforms 24HR methodology into a convenient, self-administered, low-cost method of collecting dietary intake data. ASA24 draws its format and design from the AMPM but uses an automated approach (Attachment 2). Because it is a web-based, self-administered, and uses 24HR methodology, the ASA24 makes it feasible to collect multiple days of dietary intake data in large-scale epidemiologic studies, behavioral trials, or clinical research, and may help advance our understanding of the nutritional determinants of chronic diseases4. Use of a web-based, automated data collection system also offers the advantage of automated coding of food items and calculation of nutrient intakes. The intention is for the government to provide a free or low-cost instrument, publicly available to researchers and clinicians with a need for such an instrument. The test version of ASA24 can be viewed on-line at: http://asa24.westat.com/.
24HR Recall Comparison Study
The proposed ASA24 evaluation involves two studies using an experimental design: a 24HR recall comparison study and an observational feeding study.
Self-reported dietary intake data from the new ASA24 method will be compared to self-reported dietary intake data from the standard AMPM method. A single day of 24HR data will be collected for two days (4 to 6 weeks apart), from a sample of 1080 men and women with diverse age and race/ethnicity, and in three geographical areas of the country. The sample will be drawn from three health maintenance organizations (HMO) ---Security Health Plan (using the Marshfield Clinic), Wisconsin; Henry Ford Health System, Michigan; and Northern California Kaiser-Permanente, California. These HMOs are part of NCI’s Cancer Research Network and thus are already participating in collaborative research and have critical research staff. Overall, sample respondents will be approximately equally divided among three age groups (20-34, 35-54, and 55-70). NCI will provide each HMO center staff with specifications on the required sample from each center (gender, age, and race/ethnicity mix). Each center will identify current users of their internet services, stratify all users into aforementioned sampling strata, randomly select individuals from each sampling stratum for recruitment, and mail invitation letters (Attachment 3) to potential study participants. Through the invitation letter, interested participants will be directed to a study website. With the Kaiser-Permanente site only, half of the potential respondents will receive the written letter as described, and half will receive instead an e-mail invitation with a direct link to the study website. The website will include information on study procedures and an on-line consent form (Attachment 4). Interested participants will provide their contact information through a secure on-line consent form. Westat, the contractor for this study, will monitor and track consents from this initial contact and provide the centers with a list of those who do not respond within 10 days of mail out. Each center will follow-up with these non-responders and send a second mailing. Westat will continue to track consents until the quota for each stratum is met. Westat will follow-up with individuals who have consented to participate and administer by telephone, a screening questionnaire (Attachment 5) to ascertain eligibility for the study. Those deemed eligible will be randomly assigned by sampling stratum to one of the following four experimental study groups.
Group 1 – Complete two ASA24 self-administered recalls, 4 to 6 weeks apart.
Group 2 – Complete two AMPM telephone-administered interviews, 4 to 6 weeks apart.
Group 3 – Complete one ASA24 followed by one AMPM, 4 to 6 weeks apart.
Group 4 – Complete one AMPM followed by one ASA24, 4 to 6 weeks apart.
All recalls will be unannounced, i.e. without prior scheduling, so as to avoid the potential of changing diets for the reporting day. Once participants have completed both dietary recalls, they will be directed to complete an on-line demographic survey (Attachment 6). Participants assigned to Group 3 and 4 will also complete an on-line preference questionnaire (Attachment 7), indicating their preference for either AMPM or ASA24.
The NCI Validation and Observational Feeding Study
Researchers will unobtrusively weigh the food intake of study participants, and compare the measured intake to self-reported intake using the AMPM (Attachment 1) vs. the ASA24 (Attachment 2) on the next day. Dietary data will be collected from 80 different individuals over a two-day period. Subjects will be drawn from a dataset of individuals who have volunteered to be in research projects. A pool of potential participants will be identified by EurekaFacts, a research firm, who will recruit the sample matching the specifications for age and gender. To the extent possible, we will enroll subjects who have not yet participated in research studies. A screening questionnaire (Attachment 8) will be administered to exclude subjects with dietary allergies, practices or preferences that would interfere with the protocol. Subjects will be excluded if they do not have high-speed internet at home. This type of computer connection is required for the ASA24 and therefore necessary for research studies employing the ASA24. This exclusion is required for inference of results to the research community. Subjects who have previously been involved in a research study will be excluded. If sufficient numbers of subjects cannot be drawn with the last criterion, then subjects who have not been involved in a study for over one year and never in a nutrition-related study will be sampled.
Once participants are selected, a letter containing information on the study (FAQs), their appointment dates, and a map to the facility will be mailed to them (Attachment 9). Participants will be scheduled to come to the research facility at Westat in Rockville, Maryland, for two consecutive days. Westat will bring in 9 people per day and 3 at a time may eat together in a social setting but will obtain their food separately so the amount taken can be ascertained. There will be a reminder telephone call the night before the first appointment (Attachment 10). On the first day, participants will sign the informed consent (Attachment 11) and then consume three meals throughout the day. On the second day of the study, participants will return to Westat and report their preceding day’s dietary intake, using either the AMPM or the ASA24. Once participants have completed the dietary interview, they will be directed to complete the on-line demographic survey (Attachment 12).
The current proposal builds from previous work in developing and evaluating the ASA24. Multiple small-scale cognitive and usability tests were included in the development of the ASA24 application from its onset. Testing has been conducted in eight different settings, either as individual cognitive testing or using focus groups of nine or less individuals. Testing addressed issues such as validity of portion size presentation options, meal-based vs. unstructured food list, design and format of various passes, and the general ease of use. Necessary adjustments were made to the application based on the feedback from testing5 6.
The purpose of this proposed study is to compare the newly developed web-based Automated Self Administered 24-hour Recall (ASA24) (Attachment 2) to the current standard of interview-administered 24-hour recall, the Automated Multiple Pass Method (AMPM) (Attachment 1). The ultimate intent of this study is to provide evaluative information to prospective users in their decision to use the new ASA24 method. The ASA24 method would offer a low-cost alternative to the AMPM method, and thus could allow a wider use of 24HR methodology within existing resources.
Study 1 (24HR Recall Comparison Study):
How does reported food and nutrient intake compare between the ASA24 and the AMPM?
What is the completion percentage of the ASA24 at the first stage of data collection, and how does this compare to the analogous completion percentage for the AMPM? What is the completion percentage of the ASA24 at the second stage of data collection (i.e. 1 – attrition rate), and how does this compare to the analogous completion percentage for the AMPM?
Is there a methodology preference for reporting dietary intake among individuals who reported their dietary intake using both the ASA24 and the AMPM approaches?
Study 2 (NCI Validation and Observational Feeding Study):
How does reported food intake using the ASA24 compare to observed and measured intake?
How does reported food intake using the AMPM compare to observed and measured intake?
Is there a difference in reporting bias between the two instruments?
The combined findings from the 24HR recall comparison study and the observational feeding study will inform the public health community about the feasibility and validity of the ASA24 relative to the standard AMPM methodology. Because the ASA24 offers significant cost savings, many potential users will be interested in these evaluative results. Currently, scores of inquiries have been made concerning the ASA24. These inquiries have come from U.S. federal partners who collect national level dietary intake data, academic researchers, clinicians, and other public health providers, not only from the U.S. but also from Canada, Great Britain, and other European Union countries. Summary reports, articles, and presentations will be disseminated through multiple methods (See Section A.16.3 for more detail).
A.3 Use of Improved Technology and Burden Reduction
In the AMPM approach, respondents provide data either in-person or over the telephone and trained interviewers use a computerized system to collect and enter data. The ASA24 collects intake data directly from the respondent without an interviewer. The ASA24 uses state-of-the-art automated computer technology, including graphic enhancements and animated characters to guide participants, and audio language/cues. In addition, the ASA24 software includes pictures of foods in multiple portion sizes to aid portion size estimation by the respondent. The software has the capacity to immediately compute nutrient and food group estimates for each recall day. A computerized Study Management System (SMS) will be used to manage study activities and track completion of all study activities to allow for monitoring and calculation of response rates.
A Privacy Impact Assessment (PIA) was completed and approved by HHS in January, 2009. The IT systems were divided into two systems and are titled, “NIH NCI Automated Self-Administered 24-Hour Recall (ASA24)” and the “NIH NCI Automated Self-Administered 24-Hour Recall (ASA24) Researcher Website.”
A.4 Efforts to Identify Duplication and Use of Similar Information
ASA24 is currently in use in the NIH-AARP (formerly American Association of Retired Persons) interactive Comprehensive Lifestyle Interview by Computer study (iCLIC: OMB# 0925-0594). This is a feasibility study and formative evaluation of the quality and completeness of four self-administered web-based instruments to be used in phase II of the NIH-AARP Diet and Health Study. The study will assess completion rates for the ASA24 on the first and second times, two months apart. However, the iCLIC study does not address validation of the ASA24 to the recognized standard for conducting 24HR--- the AMPM method—with standard diet. This feeding study will address the issue of validity for nutritional epidemiologic researchers.
To our knowledge, ASA24 is the only web-based, self-administered method that has been developed for collection of complete 24HR data for medical, academic and government purposes; ASA24 has yet to be released for public use. Other websites and newer technologies attempt to collect 24hr recalls but none have the multiple passes, detailed probes and highly specific food designations that support complete reporting and more accurate estimates of daily intakes. The 24HR recall comparison and observational feeding studies will be the first to evaluate dietary intake using ASA24 and compare it with the highly regarded standard of the AMPM.
A.5 Impact of Small Business and Other Small Entities
No small entities will be involved in this survey. All respondents will be individuals who participate voluntarily.
Participants will be asked to complete dietary questionnaires either once or twice within a six week period depending on their study assignment. The 24HR comparison study will compare the reported food and nutrient intake of four assigned study groups; AMPM only, ASA24 only, ASA24 followed by AMPM, AMPM followed by ASA24. Participants will complete assigned questionnaires at two time points four to six weeks apart. Feeding study participants will complete either the AMPM or ASA24 once. Reported intakes will be compared to observed and measured intakes and differences in reporting bias between the two instruments will be determined. Collecting information less frequently would not answer the research questions of this study.
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are two special circumstances that would cause this information collection to be conducted in a manner inconsistent with 5 CFR 1320.5. First, respondents in Groups 1 and 2 are required to answer the same questionnaires twice within 4 to 6 weeks. The need for repeat measurements is due to high day-to-day variability in individuals’ diets. Statisticians require a minimum of 2 measurements in order to be able to control for day-to-day variability in the analyses. The timing of these 2 measurements is critical. Since individuals tend to eat differently in different seasons (e.g. more hot dogs in the summer), it is important to collect for each individual the 2 measurements in the same season. On the other hand, if the days of report are too close together, there is a danger of the individual eating a similar diet (e.g., leftovers) and thus not providing reports of a fuller range of his/her diet. Thus, the 4-6 weeks interval between measurements was chosen, and is feasible.
Second, a limitation of the study is that findings will not be generalizable to the entire U.S. adult population, but only to population subgroups with access to high-speed internet. Since the ASA24 cannot be completed without high-speed internet, this is not a limitation for this particular evaluation study. The usual interviewer-administered 24HR will always be required for those with literacy or other limitations. Currently, it is estimated that 75% of Americans have access to the internet. Of those with a home computer, 85% have high-speed internet. This proportion can only grow in the future. In summary, the experimental design proposed will allow comparison of the ASA24 and AMPM in a sample of those with the required access to high-speed internet.
A.8 Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency
The 60-Day Federal Register notice soliciting comments on this study prior to initial submission to OMB was published on June 3, 2009, Vol. 74, p. 26702.
One public comment was received on June 5 requesting a copy of the data collection plans. The plans were sent to the responder on June 10.
ASA24 was developed through NCI’s collaboration with: Westat Inc., Baylor College of Medicine, Archimage Inc., a software developing company, and the U.S. Department of Agriculture (USDA). During the development of ASA24, an External Working Group met twice to discuss the needs and interests of potential users and provide advice to the project (Attachment 13).
A.9 Explanation of Any Payment or Gift to Respondents
To maximize participation, many research studies offer compensation in studies of the public where there is no link to health or disease outcomes. Incentives have been shown to increase response to web-based questionnaires7. Both ASA evaluation studies are using an experimental design to compare different treatments in matched groups. Complete participation is optimal for useful information; thus an important objective is to support continuing and complete participation in the study protocols. Participants of the 24HR recall comparison study will be offered $52.00 in four installments through postal mail. The initial mailed letter will include a $2 prepayment or coupon to encourage traffic to the study website. Following consent, establishment of eligibility and assignment into study group, participants will receive in a timely manner, a second incentive payment of $5. Those assigned to complete the AMPM, will also receive measuring guides to aid with the recall. After completion of the first 24HR, participants will receive an additional $15. After completion of the second 24HR, the demographic questionnaire and for groups 3 and 4, the preference questionnaire, participants will receive $30 and a thank you letter (Attachment 14).
Participants in the observational feeding study will have breakfast, lunch, and dinner provided for them on the first study day, and will be offered $40 at the end of dinner on that day. Participants will be thanked and receive an additional $80 after completion of the dietary interview and demographic survey (Attachment 15).
A.10 Assurance of Confidentiality Provided to Respondents
Participants in this study will be subject to assurances and safeguards as provided by the Privacy Act of 1974 (5 USC 552a), which requires the safeguarding of individuals against invasion of privacy. The Privacy Act also provides for the confidential treatment of records maintained by a Federal agency according to either the individual’s name or some other identifier. This information collection is covered by NIH Privacy Act Systems of Record 09-25-0156, “Records of Participants in Programs and Respondents in Surveys Used to Evaluate Programs of the Public Health Service, HHS/PHS/NIH/OD.” All persons working with ASA24 studies will adhere to the provisions stipulated within that announcement (see Attachment 16 for Privacy Act Officer’s Letter). In accordance with the Privacy Act of 1974, the confidentiality of individual respondents will be protected. The data sets created will contain no means of identifying individual respondents. The following describes the measures taken to protect the confidentiality of the participants.
After respondents have provided consent, they will be able to review an agreement memo, assuring that all collected information will be kept confidential and not disclosed in any identifiable form to anyone but the researcher conducting this study, except as otherwise required by law. IRB documentation is provided as an attachment (Attachment 17) to the clearance package.
Security protocols will be implemented to ensure that all data are recorded and stored in such a manner that individual research subjects cannot be identified directly or through identifiers. Each questionnaire will include a unique ID number for each respondent, but only the data management contractor will have the secure database to link ID numbers with individuals. No identifying information will be recorded in the data file and there will be no way to detect the identification of any respondent. After the data collection is completed, the contractor will store the paper informed consent forms (for the observational feeding study) and paper questionnaires in a locked, secure facility for a year, and then they will be shredded. Electronic data will be password protected and stored by the data management contractor, and also will be destroyed after a year.
A.11 Justification for Sensitive Questions
Sensitive questions are defined as those whose answers, if made public, could cause physical, mental, emotional, economic, or other harm to an individual. Participants will be asked to provide their year of birth, gender, and race/ethnicity information as part of the enrollment process. Additionally, there are questions on the Demographic and Health Questionnaire (Attachments 6 and 12) that are considered sensitive as they relate to use of alcohol, cigarettes, and income. Personally identifiable information (PII) is collected in the form of the respondent’s name and phone number which is needed to contact potential respondents. In the 24HR Recall Comparison Study, respondents are identified by three health maintenance organizations (HMO). In the NCI Observational Feeding Study, a subcontractor will identify and administer the screening questionnaire to the respondents. This information is important to recruit a sample from diverse backgrounds. Participation in the study is voluntary and participants have the right not to answer any questions without consequences. Section A.10 discusses the steps taken to safeguard this information.
A.12 Estimates of Annualized Burden Hours and Costs
Data collection activities for all participants involve completion of a telephone screener and demographic survey; participants in the 24HR recall comparison study will complete two 24HR; participants in the two groups using both ASA24 (Attachment 2) and AMPM (Attachment 1) will complete an additional preference questionnaire; participants in the observational feeding study will complete one 24HR. The burden estimates including annualized hourly costs and the total estimated burden for the 24HR recall comparison study and the observational feeding study are summarized in Table A.12.1 and A.12.2.
24HR Recall Comparison Study
After recruitment, participants in the 24HR recall comparison study will complete a brief (3 minutes) screener questionnaire (Attachment 5). Those enrolled in the study will be randomized to one of four groups; each respondent will complete two 24HR lasting approximately 30 minutes each, 4 to 6 weeks apart. Group 1 will complete two ASA24 recalls; group 2 will complete two AMPM recalls; Group 3 will complete one ASA24 followed by one AMPM; and group 4 will complete one AMPM followed by one ASA24. On completion of the second recall, all participants will be instructed to complete an on-line demographic survey and, for groups 3 and 4, a preference questionnaire as well. Non-response to the web-based surveys will trigger a phone interview to obtain the needed data. The estimated times for completing the demographic survey and the preference questionnaire are 8 and 3 minutes respectively. Based on our previous experience, we expect respondent attrition at various stages, and account for attrition in the respondent burden hours in Table A.12.1 below.
NCI Validation and Observational Feeding Study
Participants in the observational feeding study will complete a brief (3 minutes) screener (Attachment 8) prior to participating in the study. Study respondents will be invited to a central location for three meals. There will be a reminder telephone call the night before the first appointment (Attachment 10). Participants will be offered a variety of foods to choose from and their observed food consumption will be recorded by computing the difference in measured weight of each food taken on the plate and remaining after eating. The measurements will be taken in an unobtrusive manner. Participants will be asked to return to the central location the day after the event, during which time they will participate in a 24HR, conducted using either AMPM or ASA24, and complete a demographic questionnaire on-line. The estimated time for consuming three meals is 135 minutes and time for completing the 24HR is about 30 minutes; the estimated time for completing the demographic questionnaire is 8 minutes. The goal is to have 80 respondents complete the 24HR after meal consumption. Though the data collection is anticipated to be completed within a 12 month period, this request is for a 2-year period in the event there are unanticipated delays, additional questions or changes that will be requested and taken into account.
The total estimate of respondent burden is 1580 and 525 hours for the 24HR recall comparison study and the observational feeding study, respectively over the 2-year information collection period. This amounts to an annualized estimate of respondent burden to be 790 and 263 hours for the 24HR recall comparison study and the observational feeding study; an estimated annualized total burden of 1052 hours for both studies (see Table A.12-1).
Table A.12-1 Estimates of Annual Burden Hours
Study |
Questionnaire |
Number of respondents |
Frequency of response |
Average time response (Minutes) |
Annual hour burden |
24HR recall comparison study: |
|||||
|
Information and Consent |
650 |
1 |
15/60 |
162.50 |
Screener |
600 |
1 |
3/60 |
30.00 |
|
Dietary Recall 1 |
540 |
1 |
30/60 |
270.00 |
|
Dietary Recall 2 |
486 |
1 |
30/60 |
243.00 |
|
Demographics questionnaire |
540 |
1 |
8/60 |
72.00 |
|
Preference survey |
243 |
1 |
3/60 |
12.15 |
|
Subtotal |
789.65 |
||||
NCI validation and observational feeding study: |
|||||
|
Screener |
100 |
1 |
3/60 |
5.00 |
Reminder Telephone Call |
90 |
1 |
3/60 |
4.50 |
|
Eating 3 meals |
90 |
1 |
135/60 |
202.50 |
|
Dietary Recall |
80 |
1 |
30/60 |
40.00 |
|
Demographics questionnaire |
80 |
1 |
8/60 |
10.67 |
|
Subtotal |
262.67 |
||||
Total |
1052.32 |
||||
The cost burden to respondents is essentially the time required to read the instructions and sign the consent form, and complete the screener, dietary recalls, demographics, and preference questions. The cost to the respondents for the total burden is estimated to be $37,210 over a 2-year period.
The annualized cost to the respondents is $13,961 and $4,644 for the 24HR recall comparison study and the observational feeding study respectively. The total annualized cost to the respondents is estimated to be $18,605, calculated at $17.68 per hour (U.S., Department of Labor, Bureau of Labor Statistics, 2007). The costs are summarized in Table A.12-2.
Table A.12-2 Annualized Cost to Respondents
Study |
Questionnaire |
Annual hour burden |
Hourly Wage Rate |
Respondent Cost |
|
Information and Consent |
162.50 |
$17.68 |
2,873.00 |
Screener |
30.00 |
$17.68 |
530.40 |
|
Dietary Recall 1 |
270.00 |
$17.68 |
4,773.60 |
|
Dietary Recall 2 |
243.00 |
$17.68 |
4,296.24 |
|
Demographics questionnaire |
72.00 |
$17.68 |
1,272.96 |
|
Preference survey |
12.15 |
$17.68 |
214.81 |
|
Subtotal |
789.65 |
|
$13,961.01 |
|
|
Screener |
5.00 |
$17.68 |
88.40 |
Reminder Telephone Call |
4.50 |
$17.68 |
79.56 |
|
Eating 3 meals |
202.50 |
$17.68 |
3,580.20 |
|
Dietary Recall |
40.00 |
$17.68 |
707.20 |
|
Demographics questionnaire |
10.67 |
$17.68 |
188.59 |
|
Subtotal |
258.17 |
|
$4643.95 |
|
Total |
1052.32 |
|
$18,604.96 |
|
A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record keepers
There are no other costs to respondents beyond those presented in section A.12. There is no operating, maintenance, and capital costs associated with the 24HR recall comparison study and the observational feeding study data collection.
A.14 Annualized Cost to the Federal Government
The largest cost to the federal government is to pay a contractor $963,973 to conduct the study and deliver data files. This is based on an estimate of 4,666 hrs for salaried labor (salary range from $21-50 per hour), 10,655 hrs for hourly office labor (salary range from $12-40 per hour), and 596 hrs for offsite salaried labor (salary range $35-42 per hour), the total cost of which is $341,878 prior to overhead and other expenses. In addition the contractor estimates other direct costs of $205,169 for computing, copying, supplies, postage/shipping, miscellaneous items, and incentives. Finally, the contractor has additional fees for overhead (50-100%), general/administration (16.7%) and fixed fees (6.0%).
NCI costs are based entirely on labor. It is estimated that the study will require about 0.5 FTE total per year spread over 4-5 scientists (nutritionists, epidemiologists, statisticians) at the GS14 level or above, totaling $125,000, These expenses are related to directing contractors, overseeing and solving problems as they arise, developing materials, supervising data collection, data coding, data cleaning, data analyses, and preparation of manuscripts and presentations.
Finally, the government estimates about $6,000 for data cleaning and analyses via use of a separate biomedical computing support contract.
In summary, based on the current budget, the estimated overall cost to the Federal Government for the 24HR recall comparison and observational feeding studies is $1,094,973 for 24 months. Thus, the annualized cost to the federal government is $547,486.50.
Table 14-1 Annual Cost to the Federal Government
|
TOTAL |
ANNUAL AVERAGE |
Contractor Costs |
$963,973 |
$481,986.50 |
NCI Personnel Subtotal |
$125,000 |
$62,500.00 |
Analysis |
$6,000 |
$3,000.00 |
Grand Total |
$1,094,973 |
$547,486.50 |
A.15 Explanation for Program Changes or Adjustments
This is new data collection.
The 24HR recall comparison and observational feeding studies will begin within 3 months of obtaining OMB approval. The contract period will include fielding, analyzing, and disseminating findings from these studies. The contractor will be responsible for preparing the analytic databases resulting from the two studies. The timetable for the data collection for the 24HR recall comparison and observational feeding studies is shown below, in Table A.16-A.
Table A.16-A. Data Collection Timetable
Study activity |
Months after OMB approval |
24HR recall comparison study
|
|
HMO centers will mail invitation letters to members |
month 3 |
On-line consent from respondents |
months 3 - 5 |
Westat to conduct telephone screening |
months 3 - 6 |
Westat to mail out measuring guide to two study groups and initial incentive to all four study groups. |
months 3 - 9 |
First round of data collection begins (both modes) |
months 4 - 5 |
Second round of data collection begins |
months 5 - 6 |
Demographic survey follow-up |
months 6 - 7 |
Preference questionnaire |
months 6 - 7 |
Data processing and analysis |
months 8 - 24 |
Validation and Observational feeding study
|
|
Recruitment |
month 3 |
Screening |
months 3 - 4 |
Begin Feeding (and consent) |
months 4 - 5 |
Begin 24HR (24HR after feeding day) |
months 6 - 7 |
Data processing and analysis |
months 8 - 24 |
A.16.1 Analysis of the 24HR Recall Comparison Study Data
Statistical analysis will be conducted to compare the nutrient and food group estimates obtained from the ASA24 and the AMPM approach. All data from the two studies will be analyzed by the NCI survey methodologists and statistical staff working on this study. Table A.16-1 and A.16-2 present the research questions and analysis that will be conducted.
Table A.16-1 Research Questions and Outcome Measures for 24HR Recall Comparison Study
Research Questions |
Analysis |
How does reported food and nutrient intake compare between
the ASA24 and the AMPM?
|
The median intake reported on the ASA24 will be compared to the median intake reported on the AMPM. Due to the skewness inherent in dietary intake data, comparisons will be made on a transformed scale, where the median corresponds to the mean.
|
Sample size determinations are motivated by trying to detect whether the median in the original scale for one instrument is within 90% or 95% of the median in the original scale for the other instrument.
|
|
For some nutrients, we expect sufficient sample size to investigate differences across gender or race-ethnicity domains, but for other nutrients and foods, only an overall comparison can be performed.
|
|
What is the completion percentage of the ASA24 at the first
stage of data collection,
|
The fraction of respondents who complete the first application of the ASA24 (first-stage completion rate) will be compared to the analogous fraction for the AMPM. |
What is the completion percentage of the ASA24 at the second stage of data collection (i.e. 1 – attrition rate), and how does this compare to the analogous completion percentage for the AMPM? |
The fraction of respondents who complete the second application of the ASA24 will be compared to the analogous fraction for the AMPM to assess second-stage completion rate differences (equivalent to attrition rate differences). Sample size permitting, differences in these fractions will be compared across gender and race/ethnicity domains.
|
Is there a methodology preference for reporting dietary intake among individuals who reported their dietary intake using both the ASA24 and the AMPM approaches?
|
The fractions of respondents who prefer ASA24 over the AMPM will be computed for those individuals who complete both instruments.
Sample size permitting, differences in this fraction will be compared across gender and race/ethnicity domains. |
The research questions and the measures to be used in the analysis are shown in Table A.16-2 below.
Table A.16-2 Research Questions and Outcome Measures for the Validation and Observational Feeding Study
Research Questions |
Analysis |
How does reported food intake using the ASA24 compare to
observed intake? |
The consistency of the food intake reports between the ASA24 and
observed intake will be examined, as will the consistency of food
intake reports from the AMPM and observed intake.
Some foods are more difficult than others to
quantify. Pieces of foods such as fish fillets, slices of meat &
chops may be easier to quantify than amorphous foods such as
mashed potatoes. Food quantification accuracy will be evaluated
for individual foods and groups of food item types (i.e., piece,
amorphous, etc). |
A.16.3 Methods of Dissemination
Findings from the 24HR recall comparison and observational feeding studies will be disseminated through multiple methods, including summary reports available in electronic and hard copy format. These summary reports will also be publicly accessible through the National Cancer Institute’s Applied Research Program website: http://appliedresearch.cancer.gov/, which contains information on the ASA24. In addition, NCI staff will analyze the data and prepare presentations for national conferences and publish articles in peer-reviewed journals (in conjunction with other researchers). The NCI staff will work within NCI and with other federal agencies (e.g., USDA) to disseminate the results.
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
The 24HR recall comparison and observational feeding studies will not require exemption from displaying the expiration date of OMB approval. Any reproduction of the data collection instrument will prominently display the OMB approval number and expiration date.
A.18 Exceptions to Certification for Paperwork Reduction Act Submission
The 24HR recall comparison and observational feeding studies do not require any exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).
References
1Thompson FE, Subar AF. Dietary Assessment Methodology. In: Coulston AM, Boushey CJ, eds. Nutrition in the Prevention and Treatment of Disease. San Diego: Academic Press, 2008.
2 U.S. Department of Agriculture. USDA automated multiple-pass method (Beltsville, MD), 2005. http://www.ars.usda.gov/Services/docs.htm?docid=7710
3 National Center for Health Statistics. National Health and Nutrition Examination Survey, 2009. http://www.cdc.gov/nchs/about/major/nhanes/nhanes2005-2006/questexam05_06.htm
4 Schatzkin A, Subar AF, Moore S, et al. Observational Epidemiologic Studies of Nutrition and Cancer: The Next Generation (with Better Observation). Cancer Epidemiol Biomarkers Prev 2009;18:1026-32.
5 Subar AF et al. Formative research of a quick list for an automated self-administered 24-hour dietary recall. J Am Diet Assoc. 2007 Jun; 107(6):1002-7.
6 Subar et al. Accuracy of portion size reports using computer-based food photography aids in the development of an automated self-administered 24-hour recall (ASA24). In press.
7 Bowling JM, Rimer BK, Lyons EJ, et al. Methodologic challenges of e-health research. Evaluation and Program Planning 2006; 29:390-6.
| File Type | application/msword |
| Author | Bosire, Claire (NIH/NCI) |
| Last Modified By | Vivian Horovitch-Kelley |
| File Modified | 2009-08-03 |
| File Created | 2009-07-30 |