Ssb_asa24_8-3-09

SUPPORTING STATEMENT B FOR:

24-HOUR DIETARY RECALL METHOD COMPARISON STUDY

AND

NATIONAL CANCER INSTITUTE (NCI) VALIDATION AND OBSERVATIONAL FEEDING STUDY (NCI)

National Cancer Institute

8/3/2009

Frances E. Thompson, Ph.D.

Applied Research Program,

Division of Cancer Control and Population Sciences

National Cancer Institute

EPN 4005

6130 EXECUTIVE BLVD MSC 7344

BETHESDA, MD 20892-7344

Telephone: 301-435-4410

E-mail: [email protected]

Table of Contents

B. statistical methods 1

B.1 Respondent Universe and Sampling Methods 1

B.2. Procedures for the Collection of Information 3

B.3 Methods to Maximize Response Rates and Deal with Nonresponse 7

B.4 Test of Procedures or Methods to be Undertaken 8

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting

and/or Analyzing Data 12

Tables

B.1-1 Expected numbers of sample participants completing the first day of 24HR by site, gender, and race/ethnicity 2

B.1-2 Expected numbers of sample participants completing the 24HR by gender and age 3

List of Attachments

1. AMPM Screenshots

2. ASA24 Screenshots

3. Invitation letter for the 24HR Recall Comparison Study

4. Screenshots of 24HR Recall Comparison Study Information and Consent

5. Screening Script - 24HR Recall Comparison Study

6. Demographic and Health Questionnaire - 24HR Recall Comparison Study

7. Preference Questionnaire - 24HR Recall Comparison Study

8. Screening Script - Observational Feeding Study

9. Study Information Letter - Observational Feeding Study

10. Reminder Telephone Call Script - Observational Feeding Study

11. Consent form - Observational Feeding Study

12. Demographic and Health Questionnaire - Observational Feeding Study

13. External Working Group Members

14. Thank You Letter - 24HR Recall Comparison Study

15. Thank You Letter - Observational Feeding Study

16. NIH Privacy Act Officer's Letter

17. Certification of Institutional Review Board Approval

B. STATISTICAL METHODS

B.1 Respondent Universe and Sampling Methods

24HR Recall Comparison Study

The 24HR recall comparison study will be conducted among members of three health maintenance organizations (HMO): Security Health Plan (specifically the Marshfield Clinic), Wisconsin; Henry Ford Health System, Michigan; and Northern California Kaiser-Permanente, California. The advantages of recruiting the study participants from the HMOs include: 1) initial ability to identify internet users from a defined population; 2) knowledge of demographic characteristics of the potential respondent pool; and 3) recruitment advantages of initial contact from the member’s organization.

We expect 1200 members to begin the study, and 972 participants to complete all measures in the study. After identification of known internet users, each HMO will stratify this pool by gender, age, and race/ethnicity. Using the sample plan shown on Table B.1-1 as a guide, each HMO will randomly select a set number of potential participants from each stratum, and send a letter inviting them to participate in the study. Individuals who are interested in participating will be directed to the study website using a URL provided in the invitation letter. For the Kaiser-Permanente site, half of the initial contact group will be contacted via email with a direct link to the study site. The internet study site will include several links that will explain the study, respondent confidentiality, and consent procedures. Respondents will be asked to consent to participate in the study by reading the information provided to them and indicating their agreement by checking the “I agree” box (see Attachment 4 for website screenshots). Once a respondent has consented to participating in the study, Westat will contact them by telephone to screen them for eligibility (Attachment 5). Final enrollees will be randomly assigned to one of the four experimental study groups by sampling stratum. It is expected that each participant will complete two rounds of dietary recalls and one demographic survey (Attachment 6); participants providing data using both the AMPM and ASA24 approaches will also complete a preference questionnaire (Attachment 7). The three HMOs vary by race/ethnicity distribution; Henry Ford has few Hispanics; Marshfield has few Hispanics and African-Americans; Kaiser has substantial numbers of Whites, Blacks and Hispanics. The sampling plan takes advantage of this diversity by sampling particular race/ethnicity groups at different rates among the sites. Overall, sample respondents will be approximately equally divided among three age groups (20-34, 35-54, and 55-70).

Table B.1-1 Expected numbers of sample participants completing the first day of 24HR by site, gender, and race/ethnicity

Center	Males (N=540)				Females (N=540)				TOTAL
	W	B	H	Total	W	B	H	Total
Kaiser	25	52	180	257	25	51	180	256	513
Henry Ford	65	128	0	193	65	129	0	194	387
Marshfield	90	0	0	90	90	0	0	90	180
Total	180	180	180		180	180	180		1080

^{Overall, sample respondents will be approximately equally divided among 3 age groups (20-34, 35-54, 55-70) with approximately 360 subjects per group.} 

NCI Validation and Observational Feeding Study

A total of 100 participants will be recruited to participate in the observational feeding study. Westat will work with a subcontractor to conduct recruitment and enroll participants into the study. Participants will be divided into two study groups, approximately evenly distributed among gender and age. The subcontractor will recruit participants and administer the screening questionnaire (Attachment 8). Westat will then contact the participants to schedule their visits to Westat, the site of the study. They will mail participants a reminder of their appointments along with directions to Westat (Attachment 9). There will be a reminder telephone call the night before the first appointment (Attachment 10). Upon arrival at Westat, participants will sign the informed consent (Attachment 11), and then eat breakfast, lunch, and dinner throughout the course of the day. Participants will return to the facility on the following day to complete a 24HR interview by either the ASA24 (Attachment 2) or AMPM (Attachment 1) and a demographic questionnaire (Attachment 12). It is expected that 90 of the 100 recruited participants will attend the meals, and that 80 participants will complete the dietary interviews and demographic questionnaire.

Table B.1-2 Expected numbers of sample participants completing the 24HR by gender and age

	Age range (years)			Total
	20-34	35-54	55-70
Males	12	14	14	40
Females	12	14	14	40
TOTAL	24	28	28	80

B.2 Procedures for the Collection of Information

24HR Recall Comparison Study

Each HMO will develop a list of members with known use of internet for accessing the medical center services. Individuals will be classified by gender, age, and race/ethnicity, and for Kaiser-Permanente only also by e-mail status. Each center will randomly choose an initial contact group from each of that center’s sampling stratum. The initial contact group will receive a letter (or email) (Attachment 3) from the center giving information about the study and inviting participation. In order to encourage traffic to the study website, the initial mailed letter will include a $2 prepayment; the initial emailed letter will include a $2 coupon and link to the study website. A center contact name and number will be provided for questions. The letter will include a link directing interested members to the study site. The internet study site will include links that will explain the study, respondent confidentiality and security of websites, consent procedures, and contact information if consent is given. All materials will be written in plain and clear language. Once members consent online, an email alert will be received at Westat. Westat staff will screen consenting individuals by telephone interviews to verify that they meet eligibility criteria. Exclusion criteria include: 1) not having easy and personal access to DSL/broadband/high-speed internet; 2) not having an email account; 3) consuming a liquid-only diet; 4) being pregnant; 5) poor understanding of and/or facility with spoken English; 6) not having a telephone, either cellular or landline; and 7) not being available for the expected duration of the study.

Westat staff will monitor and track consents from this initial letter and provide the centers with a list of those who do not respond within 10 days of mail out. Each center will follow-up with these non-responders and send them a second invitation letter. Westat will continue to track consents until the quota for each stratum in each center is met. If a quota is not met, the center will be contacted in order to draw a second random sample from the relevant sampling strata, and send a second round of invitation letters to those individuals. When the quota of a particular stratum is met, Westat will randomly assign participants to one of the four study groups within each sampling strata.

Group 1 – Complete two ASA24 self-administered recalls, 4 to 6 weeks apart.

Group 2 - Complete two AMPM telephone-administered interviews, 4 to 6 weeks apart.

Group 3 – Complete one ASA24 followed by one AMPM, 4 to 6 weeks apart.

Group 4 - Complete one AMPM followed by one ASA24, 4 to 6 weeks apart.

Because dietary intakes vary from day to day, assigned dietary recall days will be balanced by weekday (Monday through Thursday) and weekend (Friday through Sunday) across groups. All recalls will be unannounced, i.e. without prior scheduling, so as to avoid the potential of changing diets for the reporting day. Approximately 1/3 of each of the four groups will be assigned two weekdays, 1/3 will be assigned one weekend day and one week day, and 1/3 will be assigned two weekend days. Trained interviewers will conduct computer-assisted telephone interviews with participants in the AMPM assigned groups. Prior to their telephone interview, measuring cups and spoons, a ruler and a USDA-developed food model booklet will be sent to participants for their use in estimating portion size. Participants in Groups 1, 3, and 4, who are completing an ASA24, will receive both an email and a phone call on the target day, instructing them to log-in to the ASA24 website prior to midnight and complete the dietary recall for the previous day.

Failure to log-in and complete the dietary recall will result in participants’ inability to access the website until a second target date is assigned and email and telephone notification provided. Up to four attempts, during a 4-week period, will be made to obtain the target recall. Participants unable to complete a recall in that time frame will be eligible to complete a recall in the next time period. If they do so, these individuals will count as providing a single recall.

Once participants have completed the dietary recall(s), they will be directed to complete an on-line demographic survey. Participants assigned to Group 3 and 4 who have completed two recalls will also complete an on-line preference questionnaire, indicating their preference for either AMPM or ASA24. This experimental study will enroll approximately 1200 respondents, with the aim of having a complete single day of recall data from 1080 and two days of recall data on 972. All study materials are written with the objective of using clear plain language.

Validation and Observational Feeding Study

Once the sample is selected as described above, participants will be scheduled to come to Westat for three meals on one day. Participants will meet with a study manager upon arrival at Westat to review the study activities, have a chance to ask questions, and sign the informed consent form. Westat will bring in 9 people per day and 3 at a time may eat together in a social setting but will obtain their food separately so the amount taken can be ascertained. For each meal, participants will be able to choose from a variety of foods and beverages, buffet-style, with no limit on the amount of food each participant may take. A different set of foods and beverages will be offered for the second and third meals. The food items and displays will be identical for all participants.

All foods taken by the participant will be weighed prior to and after the meal in an unobtrusive manner so the respondents are unaware of this activity. The method by which this will occur is via weighing of the common pot or item before and after the food is self-served, and then weighing the amount left on the plate when the participant is finished. In this way, exact weights of all foods consumed by each participant will be obtained. Subjects will come through the line one at time and so the communal item can be weighed after they have taken their portion and waste will be weighed after they leave the room or in another room. These activities will not be within eyesight. Westat will bring in 9 people per day and 3 at a time may eat together in a social setting to simulate real life meal situations. A Westat staff person will be in the dining area to prevent food sharing or discarding and will be able to initiate innocuous conversation if needed to draw attention away from conversation about study aims.

The next day, participants will return to Westat and be asked to complete a 24-hour recall about their preceding day’s dietary intake. Group 1 participants will complete the ASA24 recall. Group 2 will complete an interviewer-administered AMPM 24HR via a telephone interview with trained interviewers from Westat’s Telephone Research Center (TRC). The telephone interviewers will be blinded as to foods served or consumed and will be in a separate location in Rockville, Maryland. Measuring cups and spoons, a ruler, and the USDA Food Model Booklet will be available to the participant during the interviews to estimate portion size. Since time elapsed since the previous day can impact recall memory, there will be a balance of the two methods, to the extent possible, so that at each interview appointment there will be one AMPM and one ASA24. These appointments will be scheduled throughout the day to mimic real-life completion times for these instruments. In addition to the 24-hour recall, participants in both groups will complete a brief online questionnaire regarding topics such as demographics, height, weight, smoking history, and physical activity (Attachment 12).

B.2.1. Quality Control

The contractor for this study will establish and maintain quality control procedures to ensure standardization, and high standards of data collection and data processing. The contractor will maintain a log of all decisions that affect sample enrollment and data collection. The contractor will monitor response rates and completeness of acquired data.

B.3 Methods to Maximize Response Rates and Address Non-Response

24HR Recall Comparison Study

The experimental nature of this study requires that the four comparison groups be similarly composed with respect to reporting ability. We will use known demographic factors and randomization to attain this basic objective. A secondary objective is to maximize initial response, so as to minimize the potential problem of selection bias. The initial invitation letter will include a $2 prepayment or coupon to encourage traffic to the web site. In addition, we will follow up non-response to the initial contact (defined as within 10 days of mail-out) with a second mailed invitation. Prior to data collection, all screened and enrolled participants will receive an initial incentive of $5. Upon completion of the first recall day, a third mailing will distribute an additional $15. Upon completion of the second and final recall day and the demographic and preference questionnaires, a fourth mailing will distribute $30.

Validation and Observational Feeding Study

As the study requires participants to come to a central location for the meals and dietary interviews, recruited participants will be screened to ensure that they are able to attend both days of study participation. In addition, participants will be given $25 at the end of dinner on the first day, and an additional $75 after completing the dietary interview and demographic questionnaire on the second day. Westat will telephone participants to remind them of their scheduled appointments for meals and dietary interview; staff will also telephone any participant who does not show up for a scheduled appointment and attempt to re-schedule the appointment. In a similar Westat study, it was observed that 98% of participants who ate two meals also completed the dietary interview the following day.

B.4 Test of Procedures or Methods to be undertaken

24HR recall comparison study

Sample size was determined for each evaluative component of the 24HR recall comparison study.

Sample size determination for the formative evaluation component of this study is based upon comparing median intake of nutrients reported on the ASA24 to median intake reported on the AMPM. Medians were chosen as the measure of central tendency because most dietary data collected from 24-hour recalls exhibit skewed distributions. Typically, statistical analysis is performed on recall data only after a suitable transformation to approximate normality. Medians in such a transformed scale coincide with the mean in the transformed scale because the normal distribution is symmetric. Therefore, standard formulas for sample size calculations can be utilized based on means in the transformed scale. But, care must be taken to interpret comparisons between means in the transformed scale as comparisons between medians in the original scale, since applying an inverse transformation to quantiles (medians in particular) in the transformed scale exactly reproduces quantiles in the original scale.

The sample size required to detect a specified percentage difference between two original-scale medians depends upon 1) the transformation required to produce approximate normality of the distribution of reported intake, 2) the coefficient of variation in the transformed scale, and 3) the median of the original-scale distribution. We considered a wide range of values for these three parameters, which should encompass a correspondingly wide range of possible dietary components of interest. We found that a sample size of only 168 recalls of each type will be sufficient to guarantee at least 85% two-sided power to detect a relative difference in medians of 20% even for the most extreme scenarios; 500 recalls of each type will guarantee at least 80% two-sided power for detecting a relative difference of as little as 10%. Thus, comparisons within age, race-ethnicity, or gender groups for a wide range of nutrients are possible.

The second component is the estimation of completion rates at both stages of data collection. Data from the National Health and Nutrition Examination Survey (NHANES), which attempts to administer up to 2 applications of the AMPM to individuals who have initially agreed to participate in a fairly intensive survey protocol, are used to determine the sample size. The most recent data available (2005-2006) indicate that approximately 10% of persons who initially agreed to participate failed to satisfactorily complete the first AMPM. Of those who satisfactorily completed the first AMPM, approximately 10% failed to complete the second AMPM. For the proposed study, under the proposed sampling plan, half of the 1200 people who initially agree to participate and successfully screen in will be given an opportunity to complete an ASA24 as their first assessment. The other half will be given a chance to complete an AMPM as their first assessment. Each chance to complete an instrument counts as one “Bernoulli trial” for purposes of evaluating response rates. Then, standard formulas dealing with binomial proportions are applied to calculate sample sizes, power, and other quantities of interest below. Assuming a 90% first-stage completion rate for both instruments, 540 initial ASA24s and 540 initial AMPMs would be expected to be satisfactorily completed. Assuming a 90% second-stage completion rate for both instruments, approximately 486 satisfactory completions of the second instrument would be obtained, for a total of 1026 completed ASA24s and 1026 completed AMPMs with which to perform subsequent analyses. If one of the instruments has a completion rate of only 75%, fewer than 90 completions of the instrument would yield 90% power at a two-tailed significance level of 5% to detect the situation, and the recruitment scheme could be adjusted to maintain a balanced accrual of completions. Subsequent testing of the difference in completion rates between an instrument with a 90% completion rate and one with only a 75% completion rate would require only 135 completions of each instrument to achieve 90% power at a two-tailed significance level of 5%. Testing for a smaller difference between completion rates would require larger sample sizes: detecting a 10% difference (80% for one instrument, 90% for the other) would require 263 completions of each instrument, while detecting a 5% difference (85% vs. 90%) would require 914 completions. The last test could only be performed to test the difference in overall completion rates (combining first-and second stages).

The third component is estimating differences in preference of the two instruments. Here, the statistic of interest is the fraction of respondents who prefer the ASA24 to the AMPM. Overall, approximately 500 respondents are expected to complete both instruments. If there is no real preference (i.e. the true percentage is 50%) then 500 respondents is only enough to estimate the percentage within plus or minus 4%. Therefore, an observed percentage of between 46% and 54% would not be considered significant. However, if there is a clear preference, e.g. 75% of respondents prefer one of the instruments, 500 respondents is sufficient to estimate the percentage within plus or minus 2%.

Validation and Observational feeding Study

Sample size determination for the observational feeding study is based upon detecting differences in percentages of:

1. matches – where a food that is consumed is also reported

2. intrusions – where a food that is not consumed is nevertheless reported, and

3. exclusions – where a food that is consumed is not reported

Standard tests for the difference between two sample proportions will be applied. Power and sample size calculations for all three types of percentage are made assuming that each instance of a match, intrusion, or exclusion of a particular food counts as an independent Bernoulli trial with a constant overall success probability. In previous population-based studies using the AMPM, the average individual reports 10-12 distinct foods on a given day, corresponding to 10-12 “trials”. In reality, the trials for a particular individual are likely to be positively correlated, rather than independent. Therefore, the calculations which follow are likely to be conservative, to some (unknown) extent.

In order to have even 80% two-sided power to detect a true difference of 5% between match, inclusion, or exclusion percentages for the two instruments, between 1100 and 1550 trials per instrument would be required. Even with 12 trials per day, almost 130 completed recalls per instrument would be required – clearly out of consideration. However, 80% two-sided power to detect a true difference of 10% requires no more than 400 trials, corresponding to 33 completed recalls. With approximately 500 trials (42 completions), 90% two-sided power would be achieved. Detecting a true difference of 20% is even easier – 90% two-sided power can be achieved with no more than 130 trials (11 completions).

One of the limitations of the study is that findings will not be generalizable to the entire U.S. adult population, but only to population subgroups with access to high-speed internet. Since the ASA24 cannot be completed without high-speed internet, this is not a limitation for this particular evaluation study. The usual interviewer-administered 24HR will always be required for those with literacy or other limitations. Currently, it is estimated that 75% of Americans have access to the internet. Of those with a home computer, 85% have high-speed internet. This proportion can only grow in the future. In summary, the experimental design proposed will allow comparison of the ASA24 and AMPM in a sample of those with the required access to high-speed internet.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals were critical in developing the research plan, the conceptual framework, survey questions, and sampling strategies underlying Evaluation of the ASA. Many of the same individuals will be involved with analysis once the data are collected.

Kevin W. Dodd, Ph.D. Biometry Research Group Division of Cancer Prevention National Cancer Institute National Institutes of Health Department of Health and Human Services 6130 Executive Blvd Room 3129 MSC 7362 Bethesda, MD 20892-7362 Phone: (301) 496-7461 Fax: (301) 402-0816 Email: [email protected]	Amy F. Subar, PhD, MPH, RD National Cancer Institute Division of Cancer Control and Population Sciences Applied Research Program Risk Factor Monitoring and Methods Branch 6130 Executive Boulevard EPN 4005 Bethesda, MD 20892-7344 Phone:  301-594-0831 Fax:  301-435-3710 Email: [email protected]
Nancy Potischman, Ph.D. Nutritional Epidemiologist Applied Research Program Division of Cancer Control and Population Sciences National Cancer Institute 6130 Executive Blvd MSC 7344 Bethesda, MD 20892-7344 Tel. 301-594-6573 Fax: 301-435-3710 Email: [email protected]	Frances E. Thompson, Ph.D., MPH Epidemiologist Applied Research Program, Division of Cancer Control and Population Sciences National Cancer Institute EPN 4095B 6130 Executive Blvd MSC 7344 Bethesda, MD  20892-7344 Phone:  301-435-4410 Fax: 301-435-3710 Email: [email protected]
Gwen Alexander, PhD, MPH Asst. Research Scientist Biostatistics and Research Epidemiology Henry Ford Health System One Ford Place, 5C Detroit, MI 48202 Phone: 313-874-6737	Laura A. Coleman, Ph.D., R.D. Project Scientist Epidemiology Research Center Marshfield Clinic Research Foundation 1000 N. Oak Ave, ML2 Marshfield, WI 54449 Phone: 715-389-3350 Fax: 715-389-3880 Email: [email protected]
Lawrence H. Kushi, Sc.D. Associate Director for Etiology & Prevention Research Division of Research, Kaiser Permanente 2000 Broadway Oakland, CA 94612 Phone: 510-891-3837 Fax: 510-891-3761 Email: [email protected]	Arthur Schatzkin, M.D., Dr.P.H. Chief of the Nutritional Epidemiology Branch and Senior Investigator Division of Cancer Epidemiology and Genetics National Cancer Institute Executive Plaza South, Room 3040 Phone: 301-594-2931 Fax: 301-496-6829 E-mail: [email protected]
Stephanie M. George, M.P.H., M.A. Nutritional Epidemiology Branch Division of Cancer Epidemiology and Genetics National Cancer Institute Executive Plaza South, Room 320 Phone: 301-451-9872 E-mail: [email protected]	Victor Kipnis, Ph.D. Biometry Research Group Division of Cancer Prevention National Cancer Institute 6130 Executive Blvd Room 3131 Bethesda, MD 20892-7362 Phone: (301) 496-7464 Email: [email protected]
Douglas Midthune, M.S. Biometry Research Group Division of Cancer Prevention National Cancer Institute 6130 Executive Blvd Room 3131 Bethesda, MD 20892-7362 Phone: 301-496-7463 Email: [email protected]

 

3