SUPPORTING STATEMENT
Part A
Evaluating the Knowledge and Educational Needs of Students of Health Professions on Patient-Centered Outcomes Research
Version: December 11th, 2012
Agency of Healthcare Research and Quality (AHRQ)
1. Circumstances Making the Collection of Information Necessary 1
2. Purpose and Use of the Information 3
3. Use of Improved Information Technology 3
4. Efforts to Identify Duplication 3
5. Involvement of Small Entities 4
6. Consequences of Collecting the Information Less Frequently 4
8. Federal Register Notice and Outside Consultations 4
9. Payments/Gifts to Respondents 5
10. Assurance of Confidentiality 5
11. Questions of a Sensitive Nature 5
12. Estimates of Annualized Burden Hours and Costs 5
13. Estimates of Annualized Respondent Capital and Maintenance Costs 6
14. Estimates of Annualized Cost to the Government 6
16. Time Schedule, Publication and Analysis Plans 7
17. Exemption for Display of Expiration Date 8
The evaluation project is intended to serve AHRQ’s Office of Communication and Knowledge Transfer’s need for information about the educational requirements of students of health professions on Patient Centered Outcomes Research (PCOR), and their preferences for receipt of that information. Students and faculty in the health professions will be assessed on the extent to which PCOR is currently integrated into the training curricula and on their preferences for receipt and dissemination of PCOR information in an academic setting.
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
research that develops and presents scientific evidence regarding all aspects of health care; and
the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
AHRQ’s Effective Health Care Program, which was authorized by Section 1013 of the Medicare Prescription Drug, Improvement and Modernization Act of 2003, 42 U.S.C. 299b-7, is the Federal Government’s first program to conduct patient-centered outcomes research (PCOR) and share the findings with the public. PCOR is research that assesses the benefits and harms of preventive, diagnostic, therapeutic, palliative or health delivery system interventions. This research helps clinicians, patients and other caregivers make decisions about health care choices by highlighting comparisons and outcomes that matter to people, such as survival, function, symptoms, and health related quality of life. The Program funds individual researchers, research centers, and academic organizations to work together with the Agency to produce effectiveness and comparative effectiveness research.
The Effective Health Care Program also translates research findings into a variety of products for diverse stakeholders. These products include summary guides for clinicians, patients/consumers, and policy-makers, continuing education modules and faculty slide sets for clinicians, patient decision aids, and audio and video podcasts.
Most of the PCOR materials and translation products that are currently available are designed to help practicing clinicians, consumers/patients, and policymakers in making important decisions about health care. AHRQ recognizes the importance of insuring that clinicians in training are also exposed to PCOR and that they fully understand their role and value in shared clinical decision making. AHRQ and the Effective Health Care Program have started developing some tools, such as faculty slide sets based on comparative effectiveness reviews of the literature, to reach this audience through traditional clinical curricula. However, exposure to PCOR may occur and even be more effective in more non-traditional extracurricular settings, such as special interest projects created and sponsored by student groups or even Web-based events involving social media.
This evaluation study addresses AHRQ’s need for a report to inform strategic planning for dissemination and educational activities targeted to clinicians in training. The evaluation is intended to assess students’ and faculties needs and preferences for integrating PCOR into the health professions’ curricula, learning environment, and other training opportunities through a series of structured interviews with selected faculty members and an online survey directed at students in the health professions. The outcome will be a roadmap, which will include a set of recommendations for strategies and tools for educational and dissemination activities, along with a suggested approach and timeline for implementation of the recommendations. The recommendations will inform AHRQ’s strategic plan for future efforts which will engage and develop information and materials for the health professions student audience.
The goals of this project are to:
Understand the extent to which PCOR is currently integrated into the curriculum and how it is disseminated to students in the health professions.
Understand health professions students’ attitudes toward and knowledge of PCOR.
Explore differences in health professions student experiences with PCOR by health profession.
Identify informational and training needs and preferences of health professions students in primary care-oriented training programs.
To achieve these goals the following data collections will be implemented:
Student Survey. The purpose of the survey is to assess health professions students’ attitudes toward and knowledge of PCOR, the extent to which they value PCOR, what they would like to know, and how they would prefer to receive this information now and as they move into clinical practice. The questionnaire used in the survey is included as Attachment A and the invitation email is included as Attachment B.
Faculty Interview. The faculty interview will focus on gaining an understanding of where PCOR fits into the current curriculum for each health professions field; how both the philosophy and substantive findings of PCOR information are disseminated to instructors and subsequently to students; and perceived gaps and suggested strategies for filling these gaps. The interview guide is included as Attachment C and the invitation email is included as Attachment D.
The study directly impacts the program objective of updating educational materials for emerging clinicians. AHRQ believes this evaluation would be especially timely since AHRQ has an extensive effort underway to promote awareness and foster use of PCOR findings and products from the Effective Health Care Program. Each project is intended to work in tandem with both ongoing and new projects; reinforce messages to key target audiences; and take full advantage of all of the resources available for dissemination and implementation activities.
This study is being conducted by AHRQ through its contractor, James Bell Associates, pursuant to (1) 42 U.S.C. 299b-7, (2) AHRQ’s authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services, 42 U.S.C. 299a(a)(1), and (3) AHRQ’s authority to support the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators, 42 U.S.C. 299(b)(2).
Data will be gathered through structured interviews of faculty in health professions programs and a broad web-based survey of a cross-section of health professions students. The outcome from the project will be used immediately and directly by AHRQ’s Office of Communications and Knowledge Transfer (OCKT) staff to guide strategic planning for addressing the educational needs of health professions students. Subsequent activities may include, but are not limited to, modifying specific information about PCOR and developing novel approaches to providing information on PCOR as determined by the student survey responses. This information will also help guide the determination of the AHRQ OCKT resource needs.
The student survey will be web-based and deployed using a well-designed, low burden, and respondent-friendly survey administration process and instrument.
The 24 faculty interviews are designed as one-on-one telephone conversations with individual faculty members.
An environmental scan to obtain a general overview of the current state of dissemination of educational materials and best practices in PCOR and related concepts, including evidence-based medicine (EBM), culturally competent health care, shared decision making , and comparative effectiveness research (CER), across health professions training was conducted as the first step in the evaluation. The scan also sought to identify standardized assessments of student knowledge, skills or attitudes related to PCOR and related concepts; and to assess the existence of specific curriculum guidelines promulgated by the professional associations for each area.
In general, much of the emphasis in the literature and in the curriculum (and confirmed by expert comment) continues to be on ‘evidence-based medicine’ or ‘evidence-based practice’; although patient-centered care (or ‘client-centered care’ or ‘family-centered care’) is a commonly utilized term, ‘patient-centered outcomes research’, ‘shared decision-making’ and ‘comparative effectiveness research’ are terms that are infrequently utilized.
There was no found literature that addressed students’ preferred methods of receipt of such information. During informal interviews with three faculty interviewees, it was determined that the content and mode of practice related to PCOR education were the most critical areas to address in an inquiry with the intended target population (e.g., students, faculty, residents).
No small businesses will be involved in this study.
This is a one-time data collection.
There are no special circumstances related that would require collection to be conducted in a manner inconsistent with 5 CFR 1320.5.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on Janurary 28th, 2013 for 60 days, and again on April 15th, 2013 for 30 days (see Attachment E). No substantive comments were received.
The protocol, methods of study design and questionnaires have been evaluated by the investigators in the agency and by a review panel specifically developed for this study (the Student Workgroup membership is listed below in Table 1).
Table 1. Student Work Group Members
Name |
Area |
|
Adina Solis |
Pharmacy |
|
Courtney L. Russ |
Nurse Practitioner |
|
Elizabeth Wiley |
Medical Student |
|
Kristen J. Smith |
Physician Assistant |
|
Kristie L. Flamm |
Doctorate of Nursing Practice Program (DNP) |
|
Marisa Dowling |
Medical Student |
|
Medora McGinnis |
Nursing |
|
Shaun K Yang |
Medical Resident |
|
Thomas Keith (Tripp) Hines III |
Medical Student |
No payments or gifts will be offered.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied. Interviewers will be required to sign a confidentiality pledge (see Attachment F).
No sensitive questions are asked.
Exhibit 1 shows the estimated annualized burden hours for the respondents’ time to participate in this research. Faculty interviews will be conducted with 24 faculty members and will last about one hour. The student survey will include 1,800 students and takes 10 minutes to complete. The total burden is estimated to be 324 hours annually.
Exhibit 2 shows the estimated annualized cost burden associated with the respondents’ time to participate in this research. The total cost burden is estimated to be $4,790 annually.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Faculty Interview |
24 |
1 |
1 |
24 |
Student Survey |
1,800 |
1 |
10/60 |
300 |
Total |
1,824 |
na |
na |
324 |
Exhibit 2. Estimated annualized cost burden
Form Name
|
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Faculty Interview |
24 |
24 |
$47.70 |
$1,145 |
Student Survey |
1,800 |
300 |
$12.15 |
$3,645 |
Total |
1,824 |
324 |
na |
$4,790 |
*Based on the mean wages for Health Specialties Teachers, Postsecondary (25-1071; $47.70/hour) and Teacher Assistants (25-9041; $12.15/hour. Many of the students will be teaching and research assistants, making this the best occupational code for them), National Compensation Survey: Occupational wages in the United States May 2011, “U.S. Department of Labor, Bureau of Labor Statistics.” http://www.bls.gov/oes/current/oes_nat.htm#25-0000.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3 shows the total and annualized cost to the federal government for conducting this research. The total cost to the Federal Government is $683,335. The total annualized cost is estimated to be approximately $ 341,667. The total annual costs include the questionnaire development, administration, analysis, and study management.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$144,707 |
$72,353 |
Data Collection Activities |
$283,667 |
$141,833 |
Data Processing and Analysis |
$135,523 |
$67,762 |
Publication of Results |
$9,012 |
$4,506 |
Project Management |
$65,722 |
$32,861 |
Overhead |
$44,704 |
$22,352 |
Total |
$683,335 |
$341,667 |
This is a new collection of information.
Student Survey
The analyses will be designed to address the study’s key research questions in a one-time report.
1) Understand health professions students’ attitudes toward and knowledge of PCOR.
2) Explore differences in health professions student experiences with PCOR by health profession.
3) Identify informational and training needs and PCOR learning preferences of health professions students in primary care-oriented training programs.
Initial data analysis will include frequencies and measures of central tendency and variance, including identification of outliers. To ensure proper estimation of variance statistics given the sampling design, all analyses will control for design effects that arise from the complex survey design. Content analyses will be conducted for any open-ended questions such as free-text responses or “other, please specify” response options.
Once the descriptive analyses are conducted, a second set of analyses will be conducted to understand the effect of different student demographics and educational context factors on the key outcomes. Included in this second set of analyses will be regression models (e.g., OLS, maximum likelihood estimation) that could test hypotheses such as: students already exposed to PCOR in their formal curriculum are more likely to place a higher value on PCOR. These kinds of analyses often provide a more nuanced understanding of the research problem and provide direct implications for policy and programmatic solutions.
Faculty Interviews
The qualitative results from the faculty interviews will be summarized for themes and findings.
Schedule
The project extends over a two year period. The project time schedule is provided in Table 2, below.
Table 2. Project Time Schedule
Activity |
Time Schedule |
Conduct faculty interviews |
Spring/Summer 2013 (approximately 1-2 months after OMB approval) |
Administer student survey |
Fall 2013 |
Analyses |
Within 1 month of first data collection; ongoing thereafter |
Final evaluation and road map |
Spring 2014 |
AHRQ does not seek this exemption.
List of Attachments:
Attachment A – Student Survey
Attachment B – Invitation Email for Student Survey
Attachment C – Faculty Interview
Attachment D – Invitation Email for Faculty Interviews
Attachment E – Federal Register Notice
Attachment F – Confidentiality Pledge
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Outline of Request for OMB Review |
| Author | mcdonnell |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |