Evaluation of NCCSC and SCCSC Co-Produced Science

Q1.1 OMB Number 1090-0011 Expires 6/30/2018

  The North and South Central Climate Science Centers (NC CSC/ SC CSC) are two of eight regional centers created by the US Department of Interior, under the USGS National Climate Change and Wildlife Science Center (NCCWSC), to help meet the changing needs of land and resource managers.

  We are now beginning a multi-stage science strategic planning process to cover the period of 2018 and beyond. At this time, we are conducting an evaluation of our funded projects to examine whether science outputs have proven useful and usable to stakeholders. Your guidance will help us tailor our future activities and help us provide the highest quality of service to you as possible.

Paperwork Reduction Act Statement

This information collection is voluntary. We estimate it will take about 15 minutes to prepare and submit a response for the following survey. In accordance with the Paperwork Reduction Act (44 USC 3501), an agency may not conduct or sponsor and a person is not required to respond to a collection of information unless it displays a currently valid Office of Management and Budget control number. OMB has reviewed and approved this information collection and assigned OMB Control Number 1090-011. You may submit comments on any aspect of this information collection, including the accuracy of the estimated burden hours and suggestions to reduce this burden. Send your comments to: Information Collections Clearance Officer, US Geological Survey, [email protected].

Online Consent to Participate in Research

Title of Study:

Evaluation of NCCSC and SCCSC Co-Produced Science 

Principal Investigator:

Aparna Bamzai

Doctoral student, Dept. of Geography & Environmental Sustainability, University of Oklahoma

Deputy Director, North Central Climate Science Center, U.S. Geological Survey

1499 Campus Delivery, Colorado State University, Fort Collins CO 80523

Phone: 970-889-1231

Email: [email protected] 

Co-Investigator:

Jill Lackett

Research Associate, Natural Resource Ecology Laboratory, Colorado State University

University Deputy Director, North Central Climate Science Center, Colorado State University

1499 Campus Delivery, Colorado State University, Fort Collins CO 80523

Phone: 970-491-2343

Email: [email protected] 

Purpose of Study: You are being asked to take part in a research study. Before you decide to participate in this study, it is important that you understand why the research is being done and what it will involve. Please read the following information carefully, and contact the researcher if there is anything that is not clear or if you need more information. The purpose of this research is to define a suite of appropriate and implementable evaluation metrics for science co-produced by researchers and resource managers. 

Study Procedures: If you agree to participate in this research, you will complete the following online survey about your interactions with one of our funded projects and the associated investigator team. We estimate that completing the survey will take you approximately 15 minutes. Aggregate results from the survey will be posted online and be available to the public at the LINK webpage by DATE. 

Risks and Benefits: There are no foreseeable risks involved in participating in this study other than those encountered in day-to-day life. You will receive no direct benefits from participating in this research study. However, your responses may help the CSCs improve our operations and be better able to serve our partner needs. 

Confidentiality: We will not collect your name, contact information, or IP address, so your responses to this survey will be anonymous. Any information provided in textual responses that makes it possible to identify you will not be included in research reports. Research records will be stored securely and only approved researchers and the OU Institutional Review Board will have access to the records during the period of the study. Once the study concludes, anonymized raw data will be posted at the LINK webpage by DATE. 

Voluntary Participation: Your participation in this study is voluntary. It is up to you to decide whether or not to take part in this study. If you decide to take part in this study, you will be asked for your consent. After you sign the consent form, you are still free to skip any question and can stop participating at any time. Since survey responses are anonymous, we will be unable to identify and destroy your data if you decide you want to withdraw from the study after submission.

Contact Information: If you have questions, concerns or complaints about the research, contact me, Aparna Bamzai, at (970) 889-1231 or [email protected] or Renee McPherson, OU Dept of Geography and Environmental Sustainability, at (405) 325-1272 or [email protected]

You can also contact the University of Oklahoma – Norman Campus Institutional Review Board (OU-NC IRB) at 405-325-8110 or [email protected] if you have questions about your rights as a research participant, concerns, or complaints about the research and wish to talk to someone other than the researcher(s) or if you cannot reach the researcher(s). 

Please print this document for your records.  This research has been approved by the University of Oklahoma, Norman Campus IRB.

IRB Number:  XXXX                            Approval date: XX/XX/XXXX 

ADD CSU IRB INFO 

Q1.2 Do you agree to participate in this research? (required)

• Yes. I agree to participate (1)

• No. I do not agree to participate (2)

Skip To: End of Survey If Do you agree to participate in this research? (required) = No. I do not agree to participate

Q1.3 Please select the NC CSC project/investigator team with which you have been the most involved in the previous 5 years. Use this project/investigator team as the basis for your answers for all subsequent questions in this survey. (required, select one)

• I have not been involved with any NC CSC projects or investigators (1)

• ADD PROJECTS IN AS CHOICES (2) (approximately 40 projects will be listed in the software version here. See attached spreadsheet for the draft list)

Skip To: End of Survey If Please select the NC CSC project/investigator team with which you have been the most involved in... = I have not been involved with any NC CSC projects or investigators

End of Block: Introductory Text (internal title)

Start of Block: Engagement in the knowledge production process (internal title)

Q2.1 The following questions will help us better understand the process of investigators creating new knowledge together with resource managers and decision makers. Your candid responses are appreciated. Responses will be kept confidential.

Q2.2 When did the investigator team first contact you regarding the project? (select one)

• I (or someone from my agency) contacted the investigator team first (1)

• Prior to writing the proposal (2)

• During the proposal writing process (3)

• After the project was selected for funding (4)

• After the project was completed (5)

Q2.3 How frequently did you interact with the investigator team during the lifetime of the project? (select one)

• Never (0 times a year) (1)

• Rarely (1-3 times a year) (2)

• About every other month (4-8 times a year) (3)

• About every month (8-14 times a year) (4)

• Multiple times a month (15-51 times a year) (5)

• At least every week (52 or more times a year) (6)

Q2.4 The frequency of interaction was: (select one)

• Far too much (1)

• Moderately too much (2)

• Neither too much nor too little (3)

• Moderately too little (4)

• Far too little (5)

Q2.5 How did the investigator team interact with you in each of the relevant phases of the project? (select all that apply)

	Email correspondence (1)	Telephone or video calls (2)	In-person team meetings (3)	Workshops (4)	Training events (5)
Definition of the management or research problem(s) (1)
Articulation of the research question(s) (2)
Design of the research methods (3)
Selection of project outputs (4)
Collection of project data (5)
Analysis of project data (6)
Interpretation of results (7)
Creation of tools or other project outputs (8)
Dissemination of findings (9)

Q2.6 How did the investigator team identify your agency's needs? (select one)

• A formal needs assessment was conducted by the investigator team. (1)

• The investigator team determined your agency's needs through informal iterative discussions. (2)

• Agency needs were determined by my organization or another investigator (team). (3)

• No assessment of agency needs was conducted. (4)

• I don't know. (5)

Q2.7 How did the investigator team assess the vulnerability of or risk to the resources and/or landscapes that your agency manages?  (select one)

• A formal vulnerability or risk assessment was conducted by the investigator team. (1)

• The investigator team assessed vulnerability or risk through informal iterative discussions. (2)

• Vulnerability or risk were determined by my organization or another investigator (team). (3)

• No assessment of vulnerability or risk was conducted. (4)

• I don't know. (5)

End of Block: Engagement in the knowledge production process (internal title)

Start of Block: Use of information / outputs & outcomes (internal title)

Q3.1 The following questions will help us better understand your perceptions of the products developed through this project. Your candid responses are appreciated. Responses will be kept confidential.

Q3.2 What kinds of information, data, tools, or other products did this project provide you with?

________________________________________________________________

Q3.3 The information, data, tools, or other products provided by this project were: (select one for each)

	Agree (1)	Disagree (2)	Neither agree nor disagree (3)	Not applicable (4)
Provided with any necessary guidance (1)
Made easily accessible (2)
Easy to use (3)
Accepted as rigorous and credible (4)

Q3.4 The investigator team and project helped me to: (select all that apply)

• Understand variability and changes in weather and climate in the places that I work (1)

• Link variability and changes in weather and climate to impacts to resources or places that I manage (2)

• Identify, evaluate, or select potential adaptation strategies to cope with these impacts (3)

• other (4) ________________________________________________

Q3.5 Were the project results useful to you in any of the following ways? (select one for each)

	Extremely useful (1)	Moderately useful (2)	Not useful (3)	Not applicable / Not used in this way (4)
To be better informed broadly about an issue (1)
To be better informed specifically about a particular problem (2)
To gain a new technical skill (3)
To formulate policy (4)
To inform management plans (5)
To inform management actions (6)
To inform adaptation plans (7)
To implement adaptation plans (8)
To provide training or education within your organization (9)
To provide training or education outside of your organization (10)
Other (11)

Q3.6 Please briefly describe how your agency used project results to inform any agency plans or actions.

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Q3.7 Were there any specific factors that promoted the use of project results (select one)

• Yes (1)

• No (2)

Display Q3.8 ONLY if they select “Yes” to Q3.7

Q3.8 Please briefly describe the specific factors that you feel promoted the use of project results.

________________________________________________________________

Q3.9 Did anything limit the use of project results? (select one)

• Yes (9)

• No (10)

Display Q3.10 ONLY if they select “Yes” to Q3.9

Q3.10 From your perspective, what factors may have limited the use of the project results? (select all that apply)

• Science was not of sufficient quality (1)

• Results did not match scale of decisions or plans (2)

• Production of results did not match the timeline for decision making or planning (3)

• Resource managers / decision makers were not engaged sufficiently by the investigators (4)

• Results were not communicated in a way that is understandable by resource managers / decision makers (5)

• Tools and other products were not designed in a way that is functional for resource managers / decision makers (6)

• Resource managers / decision makers need additional skills or training to make use of the results (7)

• Did not match existing internal priorities of your organization (8)

• Internal priorities of your organization shifted (9)

• Resource constraints within your organization (10)

• Other (11) ________________________________________________

End of Block: Use of information / outputs & outcomes (internal title)

Start of Block: Relationship building / impacts (internal title)

Q4.1 The following questions will help us better understand the partnership between investigators and resource managers / decision makers. Your candid responses are appreciated. Responses will be kept confidential.

Q4.2 How do you feel about your experiences with this investigator team? (select one)

• Extremely satisfied (1)

• Somewhat satisfied (2)

• Neither satisfied nor dissatisfied (3)

• Somewhat dissatisfied (4)

• Extremely dissatisfied (5)

Q4.3 How do you feel about your experiences with this project? (select one)

• Extremely satisfied (1)

• Somewhat satisfied (2)

• Neither satisfied nor dissatisfied (3)

• Somewhat dissatisfied (4)

• Extremely dissatisfied (5)

Q4.4 Please rate how strongly you agree or disagree with the following statements. (select one for each)

	Strongly agree (1)	Somewhat agree (2)	Neither agree nor disagree (3)	Somewhat disagree (4)	Strongly disagree (5)
The investigator team appreciated and respected what I brought to the project. (1)
The investigator team took my opinion seriously in the course of discussions. (2)
The investigator team was committed to the process. (3)
The investigator team was honest and sincere. (4)
The investigator team was trustworthy. (5)

Q4.5 Based on your experiences with this investigator team and project, in the future, would you: (select one for each)

	Extremely likely (1)	Somewhat likely (2)	Neither likely nor unlikely (3)	Somewhat unlikely (4)	Extremely unlikely (5)
Utilize additional results generated by this investigator team (1)
Work with this investigator team again on another project (2)
Work with another investigator team on an NC CSC-funded project (3)
Request climate-relevant adaptation support directly from the NC CSC (4)

Display Q4.6 ONLY if they select “Somewhat unlikely” or “Extremely unlikely” to Q4.5

Q4.6 From your perspective, what reason(s) would make it unlikely for you to work with this investigator team or the NC CSC again in the future?

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Display Q4.7 ONLY if they select “Somewhat likely” or “Extremely likely” to Q4.5

Q4.7 From your perspective, what reason(s) would make it likely for you to work with this investigator team or the NC CSC again in the future?

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

End of Block: Relationship building / impacts (internal title)

Start of Block: Demographics (internal title)

Q5.1 In which of our region's states do you primarily work? (select all that apply)

• Montana (1)

• Wyoming (2)

• Colorado (3)

• North Dakota (4)

• South Dakota (5)

• Nebraska (6)

• Kansas (7)

• Other (8) ________________________________________________

Q5.2 What is the primary focus area of your work? (select all that apply)

• Wildlife (1)

• Fish (2)

• Water (3)

• Plants/vegetation (4)

• Cultural resources (5)

• Infrastructure (6)

• Agriculture (7)

• Other (8) ________________________________________________

Q5.3 Would you consider your role primarily to be? (select one)

• Resource manager/decision maker/planner (1)

• Scientist/technician/researcher (2)

• Equally both (3)

Q5.4 What type of organization do you work for? (select one)

• Local government (1)

• State agency (2)

• Tribe or tribal organization (3)

• Federal agency (4)

• University or college (5)

• Nongovernment organization (NGO/nonprofit) (6)

• Private/for profit/landowner (7)

• Other (8) ________________________________________________

Q5.5 Please provide any additional feedback on your experience with this investigator team and project that you feel was not covered in the survey questions:

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Q5.6 If you would like to discuss issues related to this survey or participate in a follow-up interview, please contact Aparna Bamzai, North Central CSC Deputy Director, via email: [email protected]. 
 

End of Block: Demographics (internal title)

Page 16 of 16