Patient Risk Tolerance Survey for Weight-Reduction Devices Supporting Statement Part A

Medical Device Decision Analysis: A Risk-Tolerance Pilot Study

0910-NEW

SUPPORTING STATEMENT

A. Justification

1. Circumstances Making the Collection of Information Necessary

At the present time, there is no existing literature or study describing the preferences or risk tolerance of obese patients when they are facing choices of weight reduction devices with different features and benefit-risk profiles. In addition, the technology and intended indications of weight reduction have been evolving during recent years. The FDA is evaluating these emerging device technologies that have different benefit-risk profiles from previously approved devices.

FDA has received anecdotal information from public testimonies, patient advocacy groups, and consumers about various weight-reduction medical devices during Advisory Panel Meetings. In addition to the anecdotal nature of these comments, the variety of marketed devices and differences in backgrounds of people who provided these comments create challenges to generalizing this information. The FDA is exploring a different approach to collecting inputs through a scientifically validated stated-preference survey. Weight-reduction devices are a suitable application for a case study to evaluate this approach.

Building upon the recent advances in research methodology of health economics, cognitive psychology, and survey research, we propose to conduct a pilot study using a choice-format conjoint-analysis survey to quantify preferences for features of medical devices that can be used to help people lose weight. Conjoint analysis is a systematic method for eliciting individual preferences through a sequence of structured trade-off questions. Subjects evaluate a series of pairs of hypothetical treatment options. The observed pattern of choices reveals the underlying preference weights associated with various treatment outcomes. Using this method, the maximum acceptable risk (MAR) of a treatment side effect can be calculated for any given level of benefit. MAR is defined as the highest level of treatment risk that subjects would accept in return for a given improvement in efficacy (i.e., the increase in treatment risk that exactly offsets treatment benefit).

This study is being conducted by FDA through its contractor, RTI Health Solutions (RTI-HS), pursuant to the FDA’s statutory authority (Medical Device Amendments of 1976, 21 U.S.C. 360c et seq., to the Federal Food, Drug, and Cosmetic Act, 21 U.S.C. 301 et seq.) to conduct and support research on the safety and efficacy of medical devices. GfK Knowledge Networks (KN), the owner of the national online panel known as KnowledgePanel®, will recruit respondents, and program and administer the online survey.

2. Purpose and Use of the Information Collection

The purpose of this pilot study is to summarize patient preferences and willingness to accept tradeoffs among attributes of medical interventions. The pilot study will help FDA evaluate the feasibility and validity of this approach for providing evidence about patients’ preferences and risk tolerance. Specifically, the weight-reduction device case study will elicit trade-off preferences of subjects with self-reported BMI of 30 kg/m² or above for the most salient features or attributes of weight reduction medical devices.

The pilot study information will be used by the FDA to evaluate stated-preference methods as a possible source of evidence about patients’ benefit-risk trade-off preferences for devices. FDA’s Center for Devices and Radiological Health (CDRH) is evaluating processes to incorporate patient preferences on treatment benefits and risks in its device review process. To obtain such preferences in a systematic and scientifically valid way, CDRH has commissioned this survey to be administered to members of a web panel that is representative of the U.S. population.

3. Use of Improved Information Technology and Burden Reduction

This study will rely on Web surveys to be self-administered at home on personal computers. The primary Web panel we are using for this study is Knowledge Networks’ (KN’s) KnowledgePanel®. KN utilizes address-based sampling (ABS) for its panel recruitment. When KnowledgePanel® began over 10 years ago, panelists were recruited via random-digit dialing (RDD) telephone surveys. At the time, RDD samples allowed access to over 90% of U.S. households. This is no longer the case due to marked declines in landline households, dramatic increases in cell-only households, the use of caller ID devices and call screening, answering machines, and do-not-call lists. Hence, a change was made in 2009 to begin recruiting entirely with the U.S. Postal Service’s Delivery Sequence File, which provides coverage to 97% of U.S. households. Under this recruitment procedure, randomly sampled addresses are invited to join KnowledgePanel® through a series of mailings and, in some cases, telephone follow-up calls to non-responders when a telephone number can be matched to the sampled address.

Operationally, households invited to participate in the KnowledgePanel® have the option to join the panel one of several ways: (1) completing and returning a paper form in a postage-paid envelop; (2) calling a toll-free hotline maintained by KN; or (3) going to a dedicated Website and completing an online recruitment form. Once these recruitment procedures are completed, invited participants become empaneled and are available to begin participating in specific online surveys. All KN panelists complete their surveys online.

Utilization of this online panel provides a number of methodological advantages including increased accuracy in measurement of key variables of interest, attractive sample characteristics, and reduced burden on study participants. This approach also yields significant cost efficiencies compared to other modes of data collection such as telephone surveys. These advantages include but are not limited to:

• The use of an online instrument allows participants to complete the survey at the time of their choosing.

• Increased privacy (compared to telephone interviewing) reduces vulnerability to socially desirable survey responses. Surveys are self-administered in a private setting and respondents do not speak to human interviewers as they would with telephone surveys.

• Coverage of non-Internet households - Households are provided with access to the Internet and hardware if needed (free Netbook laptop and free internet service). Thus unlike Internet convenience panels, also known as “opt-in” panels, that include only individuals with Internet access who volunteer themselves for research, KnowledgePanel® recruitment covers households with and without Internet access.

• ABS provides coverage to cell-phone only households.

• Flexible and timely data collection - Because KN does not involve human interviewers and all ensuing requirements for interviewer training and quality control, it is easier and cheaper to launch surveys very quickly.

• Significant cost savings over traditional telephone surveys (due to lack of human interviewers and interviewer training).

• The use of computerized systems also allows control of question sequence, including individualized randomization, recording time spent on survey segments, use of color and question layout to increase subject interest and reduce effort.

• KnowledgePanel® utilizes an unbiased general topic recruitment protocol that is free of self-selection biases related to pre-existing interests in specific research topics.

• Electronic data collection avoids the time, expense, errors, and security threats associated with manual data entry. The online questionnaire (Attachment A) will be password protected, minimizing the risk of unwarranted data entry, and maximizing data security.

Finally, KnowledgePanel® has been used for a number of similar evaluation studies, including CDC media evaluation studies led by RTI International, the research institute that controls RTI-HS. Exhibit 1 lists selected OMB-approved studies that have utilized KN’s KnowledgePanel®.

Exhibit 1. Selected OMB-Approved Studies Using the KN Online Panel
Sponsoring Agency	OMB Approval Number	Approval Date	Study Name	Data Collection Contractor	Contact Person
HHS-OPA	0990-0345	9/9/2009	Evaluation of the Parents Speak Up National Campaign: National Media Tracking Surveys	RTI International	Kevin C. Davis
HHS-OPA	0990-0325	8/15/2008	Evaluation of the Parents Speak Up National Campaign: Children’s Study	RTI International	Kevin C. Davis
HHS-CDC	0920-0752	8/24/2007	Examining the Efficacy of the HIV Testing Social Marketing Campaign for African American Women	RTI International	Kevin C. Davis
HHS-OPA	0990-0311	6/7/2007	Evaluation of the National Abstinence Media Campaign	RTI International	Kevin C. Davis
HHS-CDC	0920-0565	8/19/2002	Reactions to Canadian Style Cigarette Warning Labels	RTI International	Carol Prindle and Paul Mowery
Environmental Protection Agency	2090-0024	1/22/2004	Estimating the Value of Improvements to Coastal Waters—A Pilot Study of a Coastal Valuation Survey	RTI International	George L. Van Houtven
Environmental Protection Agency	2060-0502	2/19/2003	Eliciting Risk Tradeoffs for Valuing Fatal Cancer Risks	RTI International	George L. Van Houtven
Environmental Protection Agency	2010-0031	10/2002	Water Quality in America Pretest Round 1	Harvard University, Law School	Kip Viscusi
Environmental Protection Agency	2010-0031	2/2003	Water Quality in America Pretest Round 2	Harvard University, Law School	Kip Viscusi
Environmental Protection Agency	2010-0031	4/2003	Water Quality in America Pretest Round 3	Harvard University, Law School	Kip Viscusi
Environmental Protection Agency	2010-0031	4/2004	Water Quality in America Main Interview	Harvard University, Law School	Kip Viscusi
United States Department of Agriculture	0536-0062	12/16/2003	Estimating Consumer Benefits of Improving Food Safety	Harvard University, Center for Risk Analysis, Department of Health Policy and Management	James K. Hammitt
United States Department of Agriculture	0536-0062	3/11/2005	Estimating Consumer Benefits of Improving Food Safety	University of Wyoming, Department of Economics and Finance	Jason F. Shogren
National Oceanic and Atmospheric Agency	0648-0531	11/16/2005	Coral Reef Economic Valuation Pretest	Stratus Consulting	David Chapman

4. Efforts to Identify Duplication and Use of Similar Information

A series of literature searches and conversations with other Federal staff working on these issues determined that there are no similar data available.

5. Impact on Small Businesses or Other Small Entities

This is a one-time survey that will be voluntary for individuals who choose to participate. Given the nature of this survey, there will be no impact on small businesses and other small entities.

6. Consequences of Collecting the Information Less Frequently

This is a one-time collection. Without this data collection, FDA will not be able to evaluate the use of scientifically validated quantitative assessments of patient trade-off preference of weight-reduction medical devices.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply. Multiple copies of the form will not be required and this will be a one-time data collection activity.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60-day notice for public comment in the Federal Register of April 19, 2012 (77 FR 23484). No comments were received.

FDA has received anecdotal information from public testimonies, patient advocacy groups, and consumers about various weight-reduction medical devices during Advisory Panel Meetings. In addition to the anecdotal nature of these comments, the variety of marketed devices and differences in backgrounds of people who provided these comments, create challenges to generalizing this information. To obtain patient preferences in a systematic and scientifically valid way, CDRH has commissioned this nationally representative web-based survey.

FDA collaborated with RTI Health Solutions to design the survey.

9. Explanation of Any Payment or Gift to Respondents

All respondents to the national sample for this study will be recruited from an online research panel (the “KnowledgePanel®”) maintained by the data collection partner, GfK Knowledge Networks (KN). Individuals join the KN panel only after being directly recruited; they may not voluntarily opt-in. Those who join are given, as necessary, a computer, internet access, and ongoing technical support. These services are provided to facilitate the data collection methodology, but respondents are allowed to freely use their computing resources for personal use. Thus, these benefits are also used as an incentive for recruiting potential panel members.

Research participants will be offered a small non-cash incentive by KN to complete the proposed research survey. KN requires that any survey to its panelists provide such “points,” which can be redeemed for raffle entries, various gifts, or cash at regular intervals. The total monetary equivalent of the points for this one-time survey will not exceed $5. This incentive is intended to recognize the time burden placed on the participants, encourage their cooperation, and to convey appreciation for contributing to this important study. Numerous empirical studies have shown that incentives can significantly increase response rates (Abreu & Winters, 1999; Shettle & Mooney, 1999). The decision to use incentives for this study is based on findings reported in current research publications and several projects conducted by KN and RTI-HS, which found that use of an incentive increases response rates among adults.

All participant remuneration has been approved by the RTI-HS IRB. IRB approval is provided in Attachment E.

10. Assurance of Confidentiality Provided to Respondents

The data collection partner, Knowledge Networks (KN), maintains Information in Identifiable Form (IIF) on its web survey panel (the KnowledgePanel®) to help it draw samples for research studies. This information is not being collected anew in any way for this research study. Under subcontract to RTI-HS, KN will use current and historical reported BMI to draw a nationally representative survey sample and to issue unique, secure invitations to sampled individuals. IIF will not be included in the data released to RTI-HS or FDA. Only the answers provided in the choice-format survey (Attachment A) will be collected by RTI-HS and FDA. RTI-HS and FDA will only receive information in de-identified form.

All IRB standards for securing participant data will be maintained for data collections between participants and RTI-HS or FDA. KN’s privacy policy for collecting and storing participant information will be effective between KN and survey participants. Additional information describing KN privacy policies is described in Attachment D.

All respondents recruited for the survey will be required to read, review, and click a box indicating that they are 18 years of age or older and that they are providing informed consent to begin the survey. If a respondent does not indicate that he or she is 18 years of age or older, or does not provide informed consent, the online consent will go to a termination screen and the survey will not be collected from the respondent. Under 42 U.S.C. 1306, 20 CFR 401, and 4225 U.S.C. 552a (Privacy Act of 1974), and 42 U.S.C. 299c-3(c), individuals contacted will be informed about the protection of their information. They will also be informed of the voluntary nature of their participation in the study. The consent form, included here in Attachment A, has been reviewed and approved by RTI-HS’s Institutional Review Board (IRB) (see Attachment E).

The Office for Human Research Protections (OHRP) has granted a Federal-wide Assurance (FWA #3331 effective until July 17, 2014) to RTI-HS that grants RTI-HS the right to review and approve studies independently. In turn, OHRP has the right to audit RTI-HS’s IRB records or any study's procedures at any time to assure that RTI-HS is in compliance with the Federal regulations regarding research with human subjects.

The project team will also impose several security measures to ensure protection of private information collected from project participants. All RTI-HS computers that will be used for this study are password protected. In addition, all portable laptop computers have disk-encryption software (Pointsec) installed. Access to shared drives is limited to staff who have signed data confidentiality agreements. Any information collected in paper form will be stored in a locked file cabinet and only those staff members who have need to work with the data will have access to the file cabinet. Any paper-based data will be entered into an electronic database, stored in a password- and write-protected location on the local and/or shared drives, and the paper files will be shredded. Audio recordings will be stored in electronic formats with the protections described above and any tapes will be stored in locked filing cabinets until an electronic copy can be made at which point the tapes will be erased.

11. Justification for Sensitive Questions

The collection does not include questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. Participation is voluntary.

12. Estimates of Annualized Burden Hours and Costs

12a. Annualized Hour Burden Estimate

An invitation to the online survey will be sent to a sample of 1,000 obese adults in the United States. Among the adults who receive the invitation, about 700 are expected to complete the consent form (included in Attachment A) and about 450 are expected to qualify for the study and complete the survey in full (Attachment A). In addition to the choice-format questions, the survey also will collect information on respondent demographics, disease history, and weight-management history.

Table 1.--Estimated Annual Reporting Burden1
Survey Instrument	No. of Respondents	No. of Responses per Respondent	Total Annual Responses	Average Burden per Response	Total Hours
Survey invitation	1,000	1	1,000	.03	30
Consent form	700	1	700	.03	21
Full survey	450	1	450	.42	189
Total					240

12b. Annualized Cost Burden Estimate

There is no monetary cost to respondents.

13. Estimates of Other Total Annual Costs to Respondents and/or Recordkeepers/Capital Costs

There are no capital costs or operating and maintenance costs associated with this collection of information.

14. Annualized Cost to the Federal Government

Costs for the survey administration and in-depth interviewing include contractor expenses of $249,591 for questionnaire refinement, training interviewers, questionnaire programming, data collection, and analyzing and reporting data. In addition, government staff costs may be incurred for monitoring by the government Project Officer and an assigned team, projected to be about 25% of an FTE’s time per year. Given an FDA personnel cost of $209,632 for a fully-loaded FTE, approximately $52,408 would be spent annually on government staff salaries.

15. Explanation for Program Changes or Adjustments

This is a new collection of information.

16. Plans for Tabulation and Publication and Project Time Schedule

This is a quantitative data collection effort. In order to conduct a rigorous analysis, data analysis will begin immediately upon completion of the data collection. FDA will have a complete edited data set from the survey data, with responses from all open-ended, qualitative questions organized and coded using a taxonomy that will be determined upon completion of the data collection activity. A methodology report, detailing procedures followed in collecting these data, will be prepared. A brief analytical report identifying key findings from the data and a set of recommendations for device label formats and new research questions, with associated methodologies, that can shed further light on issues surrounding device labeling.

In addition, the available data will be saved in a machine readable format (such as Excel, SAS or SPSS) which will allow FDA to generate frequencies and percents, as well as comparative outputs at will. The resulting data and analysis will provide FDA with findings in order to better inform the decision making process in reviewing weight reduction with medical devices.

We anticipate the project will take approximately 18 months to complete.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

FDA will display the OMB expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

ATTACHMENTS:

Attachment A: Patient Risk Tolerance Survey for Obesity Devices--Survey Instrument

Attachment B: Federal Register 60-day Notice

Attachment C: Patient Risk Tolerance Survey for Obesity Devices--Email Invitation

Attachment D: KN Privacy Policy

Attachment E: IRB Approval

Attachment F: Patient Risk Tolerance Survey for Obesity Devices--Pretest Report

REFERENCES:

Abreu DA, Winters FG. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section, American Statistical Association, Washington, DC, 17, 533-538.

Shettle C, Mooney G. Monetary incentives in U.S. government surveys. Journal Off Stat. 1999;15(2):231-50.

9