Power Analysis

Appendix B. Power Analysis for FDA Front of Pack Nutrition Labeling Experiment

OMB No. 0910-New Expiration Date: XX/XX/XXXX

Paperwork Reduction Act Statement

Public reporting burden for this collection of information is estimated to average 15 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to:

Food and Drug Administration
CFSAN/PRA Comments/HFS-24
5001 Campus Dr.
College Park, MD 20740-3835.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This collection of information is being conducted on behalf of the U.S. Food and Drug Administration. The proposed sample size is 9,000. This sample size is sufficient to detect even small effects due to scheme on choosing the correct healthy/unhealthy label (Section 1) or agreement about product appropriateness depending on diet preferences (Section 2) with 90% power.

Variables:

In Section 1, label schemes will be randomized three at a time to each respondent. In Section 2, one label scheme will be randomized to each respondent.

The main test predictor is label scheme, consisting of 8 levels (GDA, Nutr Info B/W, Nutr Info Color, Nutr Info with DV B/W, Nutr Info with DV Color, High In, High In with DV, No scheme - Control). Two more schemes: Nutr Info B/W w/ Magnifying Glass and Nutr Info B/W Placement (lower right) will be further tested in Section 2.

The covariates, which will be adjusted for in the model, are nutrition literacy (yes=literate, no=not literate) and use of the nutrition facts label (none, some, always). Covariates further included in the Section 2 analysis will be product type (breakfast cereal, frozen meal, or soup), and nutrition profile (healthiest, middle, or least healthy).

Aggregate data will be analyzed, and any subgroup analyses might be conducted post hoc. The full model will contain the test predictor, covariates, and any interactions deemed of value. The model may further account for sociodemographic factors such as race, ethnicity, age, gender, income, and education.

Appendix A1 (Section 1)

In Section 1, each respondent will be presented with 3 labels per scheme (healthiest, middle, or least healthy) and 3 schemes total with both scheme order and label order within scheme randomized. Using a repeated-scheme design will improve the power of the study and result in a lower required sample size.

Sample sizes were calculated using a logistic regression scenario for predicting percent incorrect (healthiest or unhealthiest label; choice (1=incorrect, 0=correct) using label scheme as the main predictor (7 label schemes + control), and nutrition literacy (designed to capture nutrition literacy - 50% higher literacy, 50% lower literacy) and regular use of NFL (regular use, moderate use, no use) as co-predictors with moderate effect (Odds Ratio (OR)=2.5).

For % incorrect choice ranging from 5% (rare) to 30% (more prevalent), the sample size was estimated for small scheme effect (OR=1.5), medium scheme effect (OR=3), and high scheme effect (OR=5). The sample size was also modified for multiplicity (Appendix A1a).

Figure 1: Power for sample sizes ranging from 0 to 12,000, % incorrect choice ranging as .05, .1, .2, or .3, and small (OR=1.5) or medium (OR=3) effect of scheme on % incorrect choice.

% incorrect choice

5%

10%

20%

30%

label scheme effect:

small effect (OR=1.5)

moderate effect (OR=3) +

Even for low % incorrect choice (5%), n=10,000 consumers is sufficient to detect even small scheme effects (OR=1.5) with 90% power. Adjusted for the repeated measures design (Appendix A1b), the recommended sample size is 8,333. Rounding to nearest hundred, we recommend 9,000 consumers.

Randomizing 8 label schemes, three at a time, results in 56 possible 3-label permutations, and further randomizing nutrition literacy to each permutation results in 112 ‘cells’; i.e., 9000/112=80 respondents per ‘cell’.

A1a. Adjusting sample size for multiplicity

The type 1 error rate, α=.05, was adjusted for multiplicity, using Bonferroni, to account for the number of all pairwise label scheme comparisons (k=28).

.

A1b. Adjusting sample size for repeated measures

In the proposed repeated-scheme design, where each consumer would get 3 schemes to evaluate, the ‘cluster’ (i.e., consumer) size (m) is 3. Assuming a slight correlation ( ) within consumer of .1 (Hemphill, 2003), the design effect (DE) accounting for correlated observations within consumer (Eldridge et al., 2006) is

The proposed sample size of 9,000 is adjusted for this DE: the effective sample size is n/DE = 10,000/1.2 = 8,333.

Appendix 2 (Section 2)

An ordinal regression will be used to assess scheme effect on Likert-type dependent variables -- e.g., “a person can eat this product even if they are limiting their consumption of saturated fat, sodium, or added sugars” (ranging from strongly disagree to strongly agree), while adjusting for the covariates.

Adjusting alpha for multiplicity of test predictor (k=45 pairwise comparisons), 9,000 is sufficient to detect even small scheme effects with over 95% power.

Randomizing scheme, one at a time, product type, nutrition profile to respondents results in 10*3*3=90 permutations. Further splitting by nutrition literacy, 9,000 respondents would be randomized to 180 ‘cells’ with 50 respondents per ‘cell’.

SAS code available upon request.

References:

Eldridge, Sandra M., Ashby, Deborah, Kerry, Sally. (2006). Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method, International Journal of Epidemiology, Volume 35, Issue 5, October 2006, Pages 1292–1300, https://doi.org/10.1093/ije/dyl129.

Hemphill, James. (2003). Interpreting the Magnitude of Correlation Coefficients. The American psychologist. 58. 78-9. 10.1037/0003-066X.58.1.78.