Supporting Statement B For:
Solar Cell: A Mobile UV Manager for Smart Phones (NCI)
5/31/2012
Patricia Weber, DrPH
Program Director
National Cancer Institute
SBIR Development Center
6116 Executive Blvd, Suite 402
Rockville, MD 20852
Phone: 301-594-8106
Fax: 301-480-0482
Table of Contents
B. Collection of Information employing statistical methods 1
B.1 Respondent Universe and Sampling Methods 1
B.2. Procedures for the Collection of Information 4
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 8
B.4 Test of Procedures or Methods to be Undertaken 9
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data 10
Reference List 11
List of APPENDICES
Appendix A: Data Collection Instrument (Pretest)
Appendix B: Data Collection Instrument (Posttest)
Appendix C: Experts and Contributors
Appendix D: Standard Email Survey Reminder
Appendix E: Privacy Act Memo
Appendix F: IRB Approval
Appendix G: Telephone Screener
Appendix H: Consent Form (online) / Research Subject Information Sheet
Appendix I: Targeted/Planned Enrollment Table
Appendix J: Study Diagram
Appendix K: Additional Research by Klein Buendel
Participants will be recruited and enrolled from an existing national panel created and maintained by NCI’s sub-contractor, Knowledge Networks (KN). KN has recruited an online research panel—KnowledgePanel®—that is representative of the U.S. population. KnowledgePanel® currently consists of about 50,000 adult members (ages 18 and older). Panel members are recruited by Random Digit Dialing (RDD) telephone (until 2009) or by mail (since 2009), and households are provided with access to the Internet and hardware, if needed. For the address-based sampling, a probability-based sample of addresses from the U.S. Postal Service’s Delivery Sequence File. For the RDD-based sampling, a list-assisted RDD sampling techniques on the sample frame consisting of the entire U.S. residential telephone population was used. Because panelists have been recruited from two different sample frames—RDD and Address-Based Sampling (ABS) —several technical steps are taken to merge samples sourced from these frames. Their approach preserves the representative structure of the overall panel for the selection of individual client study samples. An advantage of mixing ABS frame panel members in any KnowledgePanel® sample is a reduction in the variance of the weights. ABS-sourced sample tends to align more true to the overall population demographic distributions and thus the associated adjustment weights are somewhat more uniform and less varied. This variance reduction effectively attenuates the sample’s design effect and confirms a real advantage for study samples drawn from KnowledgePanel® with its dual frame construction.
The eligible study population is defined as follows: U.S., non-institutionalized adults, age 18 and older, who use an Android 1.6 or higher smart phones, are non-Hispanic or Hispanic white (Appendix K), and are proficient in English. It is estimated that 10% of the adult population will fall into this category in 2012, which is when the study will be conducted. The population is limited to non-Hispanic or Hispanic whites because their incidence of skin cancer is far higher than in other minority populations.1 Race, ethnicity, age and gender will be collected by Knowledge Networks (KN), a sub-contractor, and thus these questions do not appear in either the screener or pretest. KN will draw a sample to meet the race, ethnicity, age and gender target enrollment criteria (Appendix I).
The survey sample will be drawn using an implicitly stratified systematic sample design based on the methodology for which Knowledge Networks was assigned a U.S. Patent (U.S. Patent No. 7,269,570) in September 2007. The selection methodology, which has been used by the sub-contractor since 2000, and ensures that the panel samples will closely track the U.S. population, and that survey panelists will not be over-burdened with survey requests. The panel provides sampling coverage of 97% of the U.S. non-institutionalized adult population via address-based sampling based on a frame including both listed and unlisted telephone numbers, regardless of Internet access. Because every sample unit (i.e., U.S. household) has a known selection probability and the sub-contractor does not accept self-selected volunteers, the panel is not susceptible to the "professional respondent" problem and other hazards of "opt-in" online panels based on convenience sampling. The American Association For Public Opinion Research (AAPOR) Online Task Force recommended the use of such probability-based sampling when measuring population characteristics by online surveys. Recent comparison research has demonstrated that KnowledgePanel’s® accuracy rates are comparable to high-quality random-digit dialing surveys and superior to online opt-in panels. The sub-contractor constantly ensures that the panel is representative across various social and demographic characteristics of the U.S. population by weighting to U.S. Census figures. Households without Internet access are provided with a laptop computer and free Internet service, so coverage bias has been minimized in the online panel. Also, non-response has been substantially reduced in the panel.
A sample of smart phone users will be drawn from the panel and screened for using an Android smart phone (Appendix G), using an implicitly stratified systematic sample design. The selection methodology will ensure that the sample will closely track the U.S. population. KnowledgePanel® participants complete an extensive profile survey when joining the panel that provides known information to use when sampling from the panel, such as demographics.
To achieve this sample, the sub-contractor will initially recruit 490 adults from its panel and randomly assign half to the experimental group and half to the untreated control group. The experimental group will receive instructions to download the Solar Cell mobile application and operate it on their Android smart phone. The control group will not be given access to the Solar Cell application. This is explained to the members in the consent form (Appendix H).
The sample will receive a personal notification email (Appendix D) inviting them to participate in the study and those who respond by logging on to the online system will be screened for using an Android smart phone. Eligible members will complete an IRB-approved online consent form (Appendix H) through the online electronic survey system.
The consent form and baseline surveys (Appendix A) will be completed with 490 eligible members. It is expected that 75% of eligible members will participate. Approximately 370 of the consented and pretested participants (75%) will complete the posttest survey (Appendix B) three months later.
B.2 Procedures for the Collection of Information
Sampling and Recruiting Procedures
A sample of non-Hispanic and Hispanic white adults with Android smart phones will be recruited for this trial. It is estimated that there will be approximately 35% non-participation at pretest. Initial sampling will be adjusted to ensure that the needed total sample size of 490 eligible adults is recruited, consented and pretested. Further, KN has achieved high follow-up rates (50-90%) in previous pre-post study designs recruited from KnowledgePanel®, and also estimates that 75% of participants who enroll, consent and complete the pretest in this randomized trial will be successfully followed-up. The sample of adult Android smart phone users can be recruited in a very short period of time (i.e., 1 month). KN recently received approval from the General Services Administration as an approved contractor; KnowledgePanel® has been used in several federally-funded studies2 and data from the panel has been published in many peer-reviewed scientific journals,3 including on topics related to health information technology and cancer control and prevention.4,4-6
Prior to sample selection, membership is weighted to be as representative of the English-speaking U.S. adult population as possibly by using a post-stratification raking procedure with geographic and demographic benchmarks from the latest Current Population Survey estimates available from the U.S. Census Bureau. Panel member base weights incorporated in this weighting adjust for panel recruitment cohort sample designs and related selection probabilities at the panel recruitment stage. The weighted panel members are pre-profiled on their cellular telephone status and type of device. Participants complete an extensive profile survey when joining the panel which provides known information to use when sampling from the panel. Using this profile information, smart phone users can thus be identified and, with their panel weights in place, they will be demographically representative of English-speaking smart phone owners in the U.S. As in the U.S. population, approximately 35% of weighted KnowledgePanel® members also have smart phones as expected.
Identified among the smart phone users will be two distinct groups: 1) Hispanic Whites, and 2) non-Hispanic Whites. These two groups will constitute two independent strata, each with their own weighted demographic profile. Each group’s demographic data will be used as benchmarks to select and weight the two representative samples for this study, one being Hispanic White smart phone users and the other being non-Hispanic White smart phone users.
The two samples for this study will be drawn randomly and proportional to the size of key demographics within each of the two strata. In this way, each sample will be a replicate of the smart phone user picture for Hispanic Whites and for non-Hispanic Whites. The sample will be drawn with the ratio of Hispanic to non-Hispanic being at about 1:5 resulting in a slight oversampling of Hispanics relative to non-Hispanics in the U.S. adult population where it is closer to 1:7.
The panel members in these samples will receive a personal notification email inviting them to participate in the study and those who respond (i.e., logging on to the online survey system) will be screened for their use of an Android smart phone. It is estimated that 10% of U.S. adults will own an Android smart phone (about a third of smart phone users). A total sample of 490 eligible adults across both strata will be enrolled and complete the IRB-approved online consent form. It is expected that 75% of eligible members invited in the sample will participate. Post-stratification weights will correct for this non-response component and return each strata’s sample to its representative picture.
All adults from the two strata who consent and complete the baseline survey will be further randomized into two groups by the project statistician. One group will receive instructions to download the Solar Cell mobile application and operate it on their Android smart phone. The other group will not be given access to the Solar Cell application.
Three months after being pretested, all participants will be asked to complete the posttest survey. The sub-contractor will re-contact the panel members in the trial and complete the posttest through its online survey system. A total of 370 pretested participants are expected to complete the posttest (75% follow-up rate across strata). To maximize completion of the pretest and posttest surveys, an email remainders will be sent and telephone calls to non-responding sample members encouraging them to complete the surveys.
Data Collection Procedures
Data collection for the pretest and posttest will be conducted using the online electronic survey system through KN’s secure, password-protected web servers using Secure Socket Layer (SSL) encryption. A Project Director and Research Analyst at KN will coordinate all sampling, recruitment and data collection for the trial with Klein Buendel’s research team. A pilot test of the pretest and posttest surveys with 50 respondents prior to initiating recruitment to the trial will be conducted. The pilot test will verify the survey functions correctly and respondents understand the wording and response categories. The pilot test will also confirm eligibility and consent rates1.
Data on use of Solar Cell mobile application by participants will be tracked through KB’s secure web server. A website from which the Solar Cell mobile application is downloaded and the website from which Solar Cell obtains the UVI forecast data will be housed on KB’s web server. KB staff will keep records of requests for technical support on electronic forms custom designed by KB’s data managers.
Sample Tracking Procedures
Participants will be tracked from pretest to posttest by KN through their Internet panel procedures. Panel members have previously provided their name, address, home, work and cell telephone numbers, and email address to KN so they can be located. To maximize completion of the pretest and posttest surveys, email reminders will be sent (Appendix D) to non-responding sample members encouraging them to complete the surveys.
Statistical Power Estimates
The latest nationwide estimates published from the Behavioral Risk Factor Surveillance Survey showed a sunburn prevalence of 34% of American adults. In KB’s previous trial on personal protection education in a North American recreation industry, the intervention reduced sunburn prevalence by 14% pre to post. Thus, the statistical power of the evaluation of Solar Cell amounted to a difference of 14% in the experimental condition compared to no change in the no-treatment control condition. A two group test with a 0.050 2-sided significance level will have 80% power to detect the difference between the experimental group proportion of 0.340 and the control group proportion of 0.200 at posttest (odds ratio of 1.918) when the sample size is 160 adults in each group (368 adults total).
To achieve this sample, Knowledge Networks will initially recruit 490 adults from its KnowledgePanel® and randomly assign half to the experimental group and half to the untreated control group. The untreated control group will not receive the SolarCell mobile application. KN expects that 75% of adults to complete the posttest, yielding 184 adults with complete data in both groups.
B.3 Methods to Maximize Response Rates and Deal with Nonresponse
Methods to Maximize Response Rates
Every effort will be made to increase response and reduce missing data due to non-response. Data collection will be conducted using KN’s online electronic survey system through its secure, password-protected web servers using Secure Socket Layer (SSL) encryption.
Sample Tracking Procedures
KN expects about 75% of eligible adults identified in the KnowledgePanel® will participate. A panel survey response rate can be calculated using a cumulative response rate calculation. 7 This takes into account the panel’s recruitment, profile and retention rates for each recruitment cohort of all participants in the study sample as well as the study’s final completion rate.
Participants will be tracked from pretest to posttest by KN through their Internet panel procedures. Panel members have previously provided their name, address, home, work and cell telephone numbers, and email address to KN so they can be located. To maximize completion of the pretest and posttest surveys, KN will send email remainders to non-responding sample members encouraging them to complete the surveys. Incentives of 5,000 points (the cash equivalent of $5) for the baseline survey and 5,000 points for the posttest survey will be offered to encourage participants to consent to downloading the smart phone application (if they are assigned to the treatment group), to return to complete the posttest survey, and ultimately to maintain high response.
Analyses will be done on data collected, i.e., case-wise deletion and imputation of missing data. KB’s statistician will assign all lost individuals to extreme category, in either direction and perform multiple imputation utilizing Markov Chain Monte Carlo – Data Augmentation Method, a 2-step iterative procedure to obtain P(θ|Yobs) that can handle various amounts of missing data and use covariates and propensity scores.
The pretest and posttest surveys will be programmed into the electronic survey system and provided to the Project Director and data managers with access to review and approve its formatting and flow. A pilot test2 of the pretest and posttest surveys with 50 respondents prior to initiating recruitment to:
Verify the survey functions correctly and respondents understand the wording and response categories and,
Confirm eligibility and consent rates.
Should changes need to be made to the pre- or post-test, a change request to OMB will be made prior to implementing.
To date, only the cognitive interviews have been completed. The cognitive interviews were to test the survey questions. The pretest and posttest questionnaires for the trial were pretested using cognitive interviews in March, 2012. Results indicated that nearly all questions were performing as intended. A few questions will need small wording changes to improve respondents’ comprehension.
Various individuals were involved in developing the research plan, the conceptual framework, survey questions, and sampling strategies. Details for each can be found in Appendix C. Many of these individuals will conduct analysis and interpretation of the data collected. Pretest and posttests will be collected by Knowledge Networks, Inc., using its electronic data collection system.
Reference List
1. Armstrong B. How sun exposure causes skin cancer: An epidemiological perspective. In: D.Hill, D.R.English, M.Elwood, editors. Prevention of Skin Cancer.Dordrecht: Kluwer Academic Publishers; 2004.
2. Knowledge Networks. List of selected OMB-reviewed surveys conducted on KnowledgePanel. http://www.knowledgenetworks.com/ganp/docs/OMB-Reviewed-Projects-List.pdf. http://www.knowledgenetworks.com/ganp/docs/OMB-Reviewed-Projects-List.pdf. 2010. Accessed: 1-11-2012.
3. Knowledge Networks. List of journals publishing articles based on survey data collected by Knowledge Networks. http://www.knowledgenetworks.com/ganp/docs/KN-Journals-List.pdf. http://www.knowledgenetworks.com/ganp/docs/KN-Journals-List.pdf. 2012. Accessed: 1-11-2012.
4. Baker L, Wagner TH, Singer S, Bundorf MK. Use of the Internet and e-mail for health care information: results from a national survey. JAMA. 2003;289(18):2400-2406.
5. O'Hegarty M, Pederson LL, Nelson DE, Mowery P, Gable JM, Wortley P. Reactions of young adult smokers to warning labels on cigarette packages. Am J Prev Med. 2006;30(6):467-473.
6. Wang C, Miller SM, Egleston B, Weinberg D. Genetic vs. non-genetic causal attributions for breast cancer and their association with health behaviors. Presented at the meeting of the Society of Behavioral Medicine. Meeting of the Society of Behavioral Medicine. 2012.
7. Callegaro M, DiSogra C. Computing response metrics for online panels. Public Opin Q. 2008;72:1008-1032.
1 This was submitted to OMB as a generic sub-study under OMB No. 0925-0642-04 and approved 2/16/2012.
2 OMB approved 0925-0642-04 on 2/16/2012.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | TABLE OF CONTENTS |
| Author | Vivian Horovitch-Kelley |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |