Form 1 Attachment A: K Awardee Survey/K12 Scholar Survey

D R A F T

AHRQ Patient-Centered Outcomes Research (PCOR) Trust Fund

Training Program Evaluation

Form Approved
OMB No. 0935-XXXX
Exp. Date XX/XX/20XX

Public reporting burden for this collection of information is estimated to average 25 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 5600 Fishers Lane, # 07W41A, Rockville, MD 20857.

K Awardee and K12 Scholar Appointee Survey

Item

Question

Response options

Skip pattern

000

Dear AHRQ PCOR K Awardee or Scholar Appointee,

The Agency for Healthcare Research and Quality (AHRQ) Patient-Centered Outcomes Research Trust Fund Training Program (PCORTF-TP) aimed to build capacity in the application of Comparative Effectiveness Research (CER) methods within the context of PCOR (CER/PCOR) through grant funding for research training and infrastructure development. AHRQ currently is coordinating with an independent evaluator (AFYA, Inc.) to assess PCORTF-TP processes and outcomes. Evaluation results will inform program planning and resource allocation. You are being asked to complete this survey because your experience as an awardee or scholar appointee provides important perspective regarding the PCORTF Training Program.

This survey is a tool to help assess your experience with the program. It is estimated that it will take about 25 minutes to complete this survey. If you are unable to complete the survey in one sitting, you can save what you have completed and return to it later by clicking on the link that was provided. Your name or the name of your academic institution will not be identified in any reports or publications. Your responses will be kept confidential to the extent permitted by law, including AHRQ’s confidentiality statute, 42 USC 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied unless you consent to the use of the information for another purpose.

000a

Respondent ID

Provided by Evaluation Team

______ [Respondent ID]

Respondent Characteristics

What is your title at your primary institution or workplace?

How long have you been at your primary institution/workplace?

• Under 1 year

• >1 year to 3 years

• >3 years to 5 years

• >5 years or more

Which of the following best describes your current work?

Select all that apply.

• Research with a CER/PCOR focus

• Health services research (without a CER/PCOR focus)

• Clinical practice

• Teaching

• Administration

• Other (please specify)

How would you describe your current primary institution or workplace?

Select all that apply.

• Academic institution

• Academic medical center

• Hospital or Ambulatory Care Clinic

• Behavioral health facility

• Foundation

• Non-Profit organization

• Private industry

• Professional society

• Federal government

• State or local government

• Other (please specify)

Training

For which AHRQ PCOR K Award Program(s) are/were you an award recipient? Select all that apply.

• K01 - AHRQ Patient-Centered Outcomes Research (PCOR) Mentored Research Scientist Development Award

• K08 - AHRQ Patient-Centered Outcomes Research (PCOR) Mentored Clinical Investigator Award

• K18 - AHRQ Mentored Career Enhancement Award in Patient Centered Outcomes Research (PCOR) for Mid-Career and Senior Investigators

• K99/R00 - AHRQ Patient Centered Outcomes Research (PCOR) Pathway to Independence Award

• K12 Scholar Appointment - AHRQ Patient Centered Outcomes Research (PCOR) Institutional Mentored Career Development Program

How long has it been since you completed your K grant training?

• Still active

• Under 1 year

• >1 year to 3 years

• >3 years to 5 years

• >5 years or more

At the start of your AHRQ PCOR K Award or Scholar Appointment, how would you describe your knowledge level related to comparative effectiveness research (CER) methods in general?

• Not knowledgeable

• Somewhat knowledgeable

• Knowledgeable

• Very knowledgeable

At the start of your AHRQ PCOR K Award or Scholar Appointment, how would you describe your knowledge level related to patient-centered outcomes research (PCOR) approaches in general?

• Not knowledgeable

• Somewhat knowledgeable

• Knowledgeable

• Very knowledgeable

To what extent were each of the following training formats employed to provide you with CER methods training during your AHRQ PCOR K Award or Scholar Appointment?

Research mentorship	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Clinical mentorship	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Webinars	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Workshops	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Coursework	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Research collaborations	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Community engagement activities	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A

Which CER training modalities/activities have you considered to be most important to effective training?

Research mentorship	Not very important Important Very important Essential N/A
Clinical mentorship	Not very important Important Very important Essential N/A
Webinars	Not very important Important Very important Essential N/A
Workshops	Not very important Important Very important Essential N/A
Coursework	Not very important Important Very important Essential N/A
Research collaborations	Not very important Important Very important Essential N/A
Community engagement activities	Not very important Important Very important Essential N/A

To what extent were each of the following training formats employed to provide you with training on PCOR principles and approaches during your AHRQ PCOR K Award or Scholar Appointment?

Research mentorship	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Clinical mentorship	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Webinars	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Workshops	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Coursework	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Research collaborations	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A
Community engagement activities	At least monthly throughout training More than twice a year but less than monthly throughout training No more than twice a year throughout training Not at all N/A

Which PCOR training modalities/activities have you considered to be most important for effective training?

Research mentorship	Not very important Important Very important Essential N/A
Clinical mentorship	Not very important Important Very important Essential N/A
Webinars	Not very important Important Very important Essential N/A
Workshops	Not very important Important Very important Essential N/A
Coursework	Not very important Important Very important Essential N/A
Research collaborations	Not very important Important Very important Essential N/A
Community engagement activities	Not very important Important Very important Essential N/A

Mentorship

How did you select your primary mentor for your AHRQ PCOR K Award or Scholar Appointment?

Select all that apply.

• Already affiliated with an institution with a strong CER/PCOR training program and access to qualified investigators who were available to serve as a primary mentor

• Referred to primary mentor by another mentor at my institution

• I had already started doing work with my primary mentor

• Other (please specify) __________

How did you select other investigators who served as your co-mentors for your AHRQ PCOR K Award or Scholar Appointment?

Select all that apply.

• Already affiliated with an institution with a strong CER/PCOR training program and access to qualified investigators who agreed to serve as co-mentors

• Referred to co-mentor(s) by another mentor at my institution

• I had already started doing work with my co-mentor(s)

• Other (please specify) __________

• N/A. There were no co-mentors for this AHRQ PCOR K Award or Scholar Appointment.

How satisfied were you with the following related to your primary mentor?

The quality of communication (ex. frequency, content, usefulness, actionable guidance).	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
The mentoring you received in CER/PCOR.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
The mentoring you received in how to apply for and obtain independent CER/PCOR grant funding.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for professional networking.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for peer networking.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for career planning.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for managing professional demands.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A

How satisfied were you with the following related to your primary mentor?

The quality of communication (ex. frequency, content, usefulness, actionable guidance).	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
The mentoring you received in CER/PCOR.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
The mentoring you received in how to apply for and obtain independent CER/PCOR grant funding.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for professional networking.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for peer networking.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for career planning.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your primary mentor’s support for managing professional demands.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A

How satisfied were you with the following related to your co-mentors?

The quality of communication (ex. frequency, content, usefulness, actionable guidance)	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
The co-mentoring you received in CER/PCOR	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
The mentoring you received in how to apply for and obtain independent CER/PCOR grant funding.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your co-mentors' support for professional networking.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your co-mentors' support for peer networking.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your co-mentors' support for career planning.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A
Your co-mentors' support for managing professional demands.	Very satisfied Satisfied Neutral Dissatisfied Very dissatisfied N/A

Overall, how satisfied were you with the mentoring you received?

• Very satisfied

• Satisfied

• Neither Satisfied nor Dissatisfied

• Dissatisfied

• Very dissatisfied

If dissatisfied or very dissatisfied, please say why.

Program Outcomes

To what extent has the support you received as an AHRQ PCOR K Awardee or Scholar Appointee contributed to your knowledge and skills in each of the following CER methods/approaches?

CER Methods
Designing pragmatic clinical trials	To a great extent To a moderate extent To a small extent Not at all N/A
Conducting pragmatic clinical trials	To a great extent To a moderate extent To a small extent Not at all N/A
Integrating quantitative and qualitative data sources	To a great extent To a moderate extent To a small extent Not at all N/A
Identify gaps in the literature	To a great extent To a moderate extent To a small extent Not at all N/A
Conduct systematic literature/ evidence searches	To a great extent To a moderate extent To a small extent Not at all N/A
Conduct systematic evidence appraisal and data abstraction	To a great extent To a moderate extent To a small extent Not at all N/A
Conduct rigorous evidence synthesis and meta-analyses	To a great extent To a moderate extent To a small extent Not at all N/A
Using observational studies in the synthesis of evidence related to comparative effectiveness	To a great extent To a moderate extent To a small extent Not at all N/A
Using techniques to reduce confounding and potential bias inherent in observational studies	To a great extent To a moderate extent To a small extent Not at all N/A
Implementation science methodology	To a great extent To a moderate extent To a small extent Not at all N/A
Using registries and data mining techniques	To a great extent To a moderate extent To a small extent Not at all N/A
Conducting subgroup analyses to determine which treatments and interventions work best for specific populations	To a great extent To a moderate extent To a small extent Not at all N/A

To what extent did your training as an AHRQ PCOR K Awardee or Scholar Appointee strengthen your skills and use of the following PCOR principles?

PCOR Principles
Engaging stakeholders in the formulation of research questions and the design of research projects	To a great extent To a moderate extent To a small extent Not at all N/A
Engaging stakeholders in research project implementation	To a great extent To a moderate extent To a small extent Not at all N/A
Engaging stakeholders in research dissemination	To a great extent To a moderate extent To a small extent Not at all N/A
Collaboration with other institutions or research centers capable	To a great extent To a moderate extent To a small extent Not at all N/A

Do you attribute achieving any of the following career landmarks to your receipt of the AHRQ PCOR K Award or Scholar Appointment? Select all that apply.

• Attainment of a faculty position

• Receipt of tenure

• Increased salary

• Receipt of additional research funding

• Receipt of additional research funding specifically to conduct CER/PCOR

• Publication of peer-reviewed articles and/or books

• Establishment of an independent health services research program

• Establishment of an independent CER/PCOR program

• Employment of additional researchers and support staff

• Receipt of professional honors or distinctions

• Service on editorial boards, peer review panels, advisory councils

• Appointment as department/division chair, dean, provost, president, or other leadership position

• Appointment as mentor to other researchers

• Other (please specify)

How have you applied CER/PCOR training to the research projects conducted as part your AHRQ PCOR K Award or Scholar Appointment? Select all that apply.

• Developed a plan for ongoing CER/PCOR research

• Applied for additional CER/PCOR research funding

• Conducted additional CER/PCOR research

• Served as a CER/PCOR mentor

• Professional work as a CER/PCOR clinical practitioner

• Other, please explain: __________

How has the training you received as an AHRQ PCOR K Award or Scholar Appointee influenced your career plans? Select all that apply.

• Decreased interest in conducting CER/PCOR research

• Increased interest in conducting CER/PCOR research

• Increased interest in implementing CER/PCOR in clinical practice

• Increased interest in mentoring others in CER/PCOR

• Other (specify)

What were your short-term goals when you started the program? Select all that apply.

• Gain specific knowledge and skills related to CER methods

• Gain specific knowledge and skills related to PCOR

• Expand on my CER research skills

• Expand on my PCOR research skills

• Improve my ability to secure future research funding

• Develop or improve research leadership and management skills

• Gain guidance and mentoring in health services research in general

• Author publications and presentations

• Other, please explain: __________

What were your long-term goals when you started the program? Select all that apply.

• Advance my research field of study in CER/PCOR

• Work in academia

• Obtain independent research position

• Obtain independent research position with a focus on CER/PCOR

• Obtain follow-up funding support in health services research

• Obtain follow-up funding support specifically related to CER/PCOR

• Improve the quality of patient care

• Other, please explain: ___________

To what extent were you able to achieve the short-term and long-term goals that you set?

Short-term	Exceeded all goals Completely Mostly Somewhat Not at all
Long-term	Exceeded all goals Completely Mostly Somewhat Not at all

What new CER/PCOR related research, clinical, educational projects have you developed as a result of your participation in this training program?

To what extent do you think the AHRQ PCOR K Award or Scholar Appointment has enabled you to achieve or contribute to any of the following?

Advancement of CER/PCOR methods in your field of study	To a great extent To a moderate extent To a small extent Not at all N/A
Influence on another field	To a great extent To a moderate extent To a small extent Not at all N/A
Contributions to a systematic review	To a great extent To a moderate extent To a small extent Not at all N/A
Contributions to clinical, educational, or other guidelines or standards	To a great extent To a moderate extent To a small extent Not at all N/A
Contributions to laws or policies	To a great extent To a moderate extent To a small extent Not at all N/A
Contributions to government reports	To a great extent To a moderate extent To a small extent Not at all N/A
Development of factsheets, newsletters or other educational materials	To a great extent To a moderate extent To a small extent Not at all N/A
Provision of expert testimony	To a great extent To a moderate extent To a small extent Not at all N/A
Development and testing of new or improved tools, devices, tests, measures, services, or screening approaches to identify, confirm, treat, or manage disease or disability	To a great extent To a moderate extent To a small extent Not at all N/A
Adoption of new or improved delivery methods for care or services	To a great extent To a moderate extent To a small extent Not at all N/A
Reduction in the cost of care or services	To a great extent To a moderate extent To a small extent Not at all N/A

Research

What CER methods did you use for your AHRQ PCOR K Award or Scholar Appointment research work?

Select all that apply.

• Pragmatic clinical trials

• Integrating quantitative and qualitative data sources

• Systematic literature/evidence searches

• Systematic evidence appraisal and data abstraction

• Rigorous evidence synthesis and meta-analyses

• Observational studies in the synthesis of evidence related to comparative effectiveness

• Techniques to reduce confounding and potential bias inherent in observational studies

• Implementation science methodology

• Analysis of registries and data mining techniques

• Subgroup analyses to determine which treatments and interventions work best for specific populations

• Other (please specify)

What PCOR components were incorporated in your AHRQ PCOR K Award or Scholar Appointment research work?

Select all that apply.

• Engaging stakeholders in the formulation of research questions and the design of research projects

• Engaging stakeholders in research project implementation

• Engaging stakeholders in research dissemination

• Collaboration with other institutions or research centers capable

• Other (please specify):

• None

What CER methods have you used since completing your AHRQ PCOR K Award or Scholar Appointment?

Select all that apply.

• Pragmatic clinical trials

• Integrating quantitative and qualitative data sources

• Systematic literature/evidence searches

• Systematic evidence appraisal and data abstraction

• Rigorous evidence synthesis and meta-analyses

• Observational studies in the synthesis of evidence related to comparative effectiveness

• Techniques to reduce confounding and potential bias inherent in observational studies

• Implementation science methodology

• Analysis of registries and data mining techniques

• Subgroup analyses to determine which treatments and interventions work best for specific populations

• Other (please specify)

• None

• N/A; my AHRQ PCOR K Award or Scholar appointment is still active.

• N/A; I no longer do research

What PCOR components have you incorporated in your research since completing your AHRQ PCOR K Award or Scholar Appointment?

Select all that apply.

• Engaging stakeholders in the formulation of research questions and the design of research projects

• Engaging stakeholders in research project implementation

• Engaging stakeholders in research dissemination

• Collaboration with other institutions or research centers capable

• Other (please specify);

• None

• N/A - My AHRQ PCOR K Award or Scholar appointment is still active.

• N/A - I no longer do research

Stakeholder Engagement

What types of stakeholder groups did you engage during your research as an AHRQ PCOR K Awardee or Scholar Appointee?

Select all that apply.

• Other researchers within my institution

• Researchers from other academic institutions

• Clinical care providers

• Community-based organizations

• Patient advocacy groups

• Patients and family members

• Information technology vendors/developers

• Federal, state, local government programs

• Other (please specify)

What types of stakeholder groups have you engaged for your research since completing your AHRQ PCOR K Award or Scholar Appointment?

Select all that apply.

• Other researchers within my institution

• Researchers from other academic institutions

• Clinical care providers

• Community-based organizations

• Patient advocacy groups

• Patients and family members

• Information technology vendors/developers

• Other (please specify)

• None

• N/A - My AHRQ PCOR K Award or Scholar appointment is still active.

• N/A - I no longer do research

What was the result of the interaction and collaboration with the stakeholder groups? Select all that apply.

• Improved research design

• Improved engagement with targeted participant populations

• Increased access to data

• Improved dissemination and reach

• Increased cultural appropriateness of project materials

• Other (please specify)

How have stakeholder activities contributed to your training program experiences? Select all that apply.

• Increased my cultural competence

• Increased my understanding of cultural context

• Enabled opportunities to work in field settings

• Other (please specify)

Disparities Focus

Does/did your AHRQ PCOR K Award or Scholar Appointment research project(s) have a health disparities focus?

• Yes

• No

Which AHRQ priority populations are/were the focus of your AHRQ PCOR K Award or Scholar Appointment research project(s)?

Select all that apply.

• Children/adolescents

• Elderly

• Low-Income

• Racial/Ethnic Minorities

• Rural Areas

• Inner City Areas

• Special Healthcare Needs

• Women

• Other (specify)

• N/A

How does/did your AHRQ PCOR K Award or Scholar Appointment research project(s) address equity issues?

Select all that apply.

• The conduct of research projects that aim to eliminate disparities

• Inclusion of disparate populations in research project study populations

• Inclusion in stakeholder groups engaged during research project

• Disseminate new culturally appropriate products to the community

• Other (specify)

• N/A - My AHRQ PCOR K Award or Scholar Appointment research project(s) does/did not address equity issues.

Sustainability/

Future Career Plans

How likely are you to continue in a research career in the next five years? Select one.

• I am no longer in a research career

• Very unlikely .........................................................

• Somewhat unlikely ................................................

• Unsure ..................................................................

• Somewhat likely....................................................

• Very likely..............................................................

Skip to question 43

Skip to question 43

Skip to question 43

Skip to question 44

Skip to question 44

Why did you leave your research career?

Skip to question 52

You indicated that you are unlikely or unsure about to continuing in a research career. Why?

Since receiving your AHRQ PCOR K Award or Scholar Appointment, how many non-CER/PCOR research grants or contracts have you received as PD/PI? Select one.

• 0

• 1-2

• 3 or more

Since completing your AHRQ PCOR K Award or Scholar Appointment, how many grants or contracts that build on CER/PCOR have you received as PD/PI? Select one.

• 0.............................................................................

• 1-2

• 3 or more

Skip to question 52

You indicated that since receiving your AHRQ PCOR K Award or Scholar Appointment, you have received grant or contract funding as PD/PI that builds on CER/PCOR. Please select the type of funding you received. Select all that apply.

• Independent Research grant (e.g., R01, R03, etc.)

• Research Programs Projects and Center Grants (e.g., P)

• Training grant (e.g., K, T)

• Cooperative agreement (e.g., U)

• Contract

• Other:

What type of funding source administers the grant or contract you received?

• Federal Government

• State or Local Government

• University

• Foundation

• Industry

• Other:

How likely are you to apply as PD/PI for more research grants or contracts that build on CER/PCOR? Select one.

• Very unlikely

• Somewhat unlikely

• Unsure

• Somewhat likely.........................................................

• Very likely .................................................................

Skip to question 50

Skip to question 50

You indicated that you are unlikely/unsure about applying for additional funding. Why?

Skip to question 52

For which of the following are you likely to apply?

• Independent Research grant (e.g., R01, R03, etc.)

• Research Programs Projects and Center Grants (e.g., P)

• Training grant (e.g., K, T)

• Cooperative agreement (e.g., U)

• Contract

• Other:

What type of funding source administers the grant or contract opportunity you intend to pursue?

• Federal Government

• State or Local Government

• University

• Foundation

• Industry

• Other:

Please provide any additional comments you have about the training program and research.

For reference only.

K01	Career development (individuals with research doctoral degrees [e.g., PhD, ScD, DrPH]); minimum 75% time; sustained period of “protected time” (up to 5 years)
K08	Career development (individuals with clinical doctoral degrees or PhDs in clinical areas); minimum 75% time; sustained period of “protected time” (up to 5 years)
K18	Career development (mid-career and senior investigators); minimum 50% over 6-month to 1-year period to further develop their research expertise in PCOR methodologies
K99/R00	Pathway to Independence Award (postdoctoral candidates with less than 5 years research training for mentored K99 up to 2 years; followed by R00 independent research up to 3 years). Contains two components, Activation of the independent award phase is contingent upon the investigator securing an independent research position. The expected output of these grants is accelerated transition to tenure and productivity of PCOR researcher
K12 Scholar	Career development (individuals with research doctoral degrees [e.g., PhD, ScD, DrPH]); full-time for 2-3 years for scholars; sustained period of “protected time” (up to 5 years for overall project)