NSF Centers for Chemical Innovation Survey Pre-tests

Appendix A: Online Surveys
Survey of Principal Investigators and Co-Investigators
OMB# 3145-0215

Pilot only:
Thank you for agreeing to help pilot test this survey. Your feedback will help us improve the survey
before we send it to other CCI participants.
Please review the instructions on this page and the next before beginning. On each page of the survey
we have included a text box for you to add comments about any questions that are not clear or relevant
to you. You only need to fill in these text boxes for any survey items for which you have specific
comments. At the end of the survey, we have included a brief set of questions on your overall
impressions of the survey’s content, clarity, and length.
All respondents:
This survey solicits responses from all lead Principal Investigators and co-Investigators participating in the National Science
Foundation’s (NSF) Centers for Chemical Innovation (CCI) Program. The goal of the survey is to inform the evaluation of
the program. We are asking for information to supplement your annual reports. All data will be aggregated to not reveal
individual respondents. The survey should take no more than 20 minutes to complete.
INSTRUCTIONS

•

Please use the survey's navigation buttons (“Back” and “Next”) to move through the questionnaire.

•

You may exit the survey at any time by pressing the “save and continue later” button. When you re-open the
survey, you will be able to continue where you left off.

•

Once you reach the last question in the survey, you will see a “submit” button. After clicking this button, your
survey will be complete and you may close your browser. Once submitted, you will not be able to return to
the survey without contacting our team.

•

If you have any questions regarding the survey, you can contact us at [email protected].

[Verification]

1. Is your name: [First Name, Middle Name/Initial, Last Name]?
□ Yes, this is correct
□ No, my name has changed or is misspelled. My name is: _________
□ No, I’m not the person named above  Exit the survey
2. All of the questions in the survey refer to your participation in the following NSF Center for
Chemical Innovation (CCI): [CCI name].
Please note that your center may have been funded under a predecessor to the NSF CCI
program such as Chemical Bonding Centers (CBC).
□ I have participated in this CCI  3
□ I have participated in a different CCI  A
□ I have not participated or I am not sure whether I participated in the CCI program  Exit
A. Please select from the list of options.
If you participated in more than one CCI, please select the CCI with which you have
been most extensively involved.
[pull-down menu of CCIs]
3. What is your role in this CCI? Select one.
□ Principal Investigator (PI)
□ Co-Investigator
□ Other. Please specify: ______________
4. For how many years have you been or were you associated with the CCI program? 1
□ This is the first year
□ 1 year
□ 2 to 4 years
□ 5 to 6 years
□ 7 or more years
[Project characteristics]

5. On a scale of 1-5, where 1 indicates “not at all” and 5 indicates “very,” to what extent does the
research conducted by your CCI have the following characteristics? Enter one response per
row. 2
1-5
Addresses important societal problem
Focuses on major scientific challenge(s) in fundamental chemistry
Has the potential to radically change our understanding of an important scientific or engineering concepts
Interdisciplinary
1
2

AAAS STC Faculty survey item 3
Includes options from Abt INSPIRE survey

1-5
High-risk
Requires a coordinated effort from diverse experts
Requires large investment of funds

[Pre-existing collaboration and role of CCI in collaboration]

6. Had you collaborated with any or all of the CCI partners before the center was established?
Select one.
□ None of the partners  9
□ Some partners  7
□ Half or more of the partners  7
7. For how many years had you collaborated with any of the CCI partners before the center was
established?
Years: ______________
8. To what extent has participation in CCI influenced these pre-existing collaborations? Select
one.
□ Not at all
□ To some extent
□ To a considerable extent
9. Have you continued or do you expect to continue collaborating with CCI partners? Select all
that apply.
□ No
□ I maintained or expect to maintain some CCI collaborations after the grant ends
□ I maintained or expect to maintain most or all CCI collaborations after the grant ends
□ Uncertain
[Benefits of CCI participation: research, personal visibility, and infrastructure for the community]

10. Please indicate whether participation in the CCI has benefited your research program. Select
one response per row. 3
No benefit
due to CCI
Use of new theoretical models
Use of new/additional instrumentation or technology
Use of new/additional data sources
Use of new/additional theoretical or experimental models
Ability to generate new and/or better ideas
Ability to more quickly/effectively respond to scientific
developments
Ability to take your research in a new direction
Access to resources at partner institutions
Access to resources at your institution
3

STC faculty survey item 9 modified

Some benefit
due to CCI

Large benefit
due to CCI

Not
applicable

Ability to attract better qualified or more diverse students
and postdocs to your research group
Ability to obtain additional funding to support your
research

No benefit
due to CCI

Some benefit
due to CCI

Large benefit
due to CCI

Not
applicable

11. Have any of the following changes occurred in your publication patterns, research interests,
and/or professional visibility since you began participating in CCI? Select one response per
row.
Decreased
due to
CCI
participation

Decreased, but
not due to CCI
participation

Has not
changed

Increased,
but
not due to
CCI
participation

Increased
due to
CCI
participation

Number of papers published
per year
Journal quality
Publishing in a broader range
of journals
Publishing with industry
Funding from industry
Interest in commercialization
Participation in new
professional conferences,
associations or societies
Number of speaking invitations
Requests to serve on
dissertation committees
outside your home institution
Requests to serve on advisory
panels
Requests to serve on editorial
board of journals
Requests to serve as a peer
reviewer
Requests to provide policy
advice or testimony
Receipt of awards, fellowships,
or chaired positions
Diversity of research problems
on which you work

12. Which of the following resources/infrastructure created or improved by CCI, if any, are being
used by researchers not affiliated with the center? Select all that apply.
□ Methods or instruments, research practice
□ Reagents

□
□
□
□
□
□
□

Data
Communication infrastructure
Educational or outreach materials
Facilities or equipment
Lessons learned for how to run a large center
New partnerships or alliances
Other resources. Please specify: __________

[Benefits of CCI participation: broader impacts]

13. Please indicate whether your CCI developed or improved the following infrastructure to
develop workforce, broaden participation of underrepresented groups in chemistry, and
improve public outreach, and if you expect to be able to sustain it after the grant ends.
Developed/
Enhanced

Expect to sustain after
CCI grant ends
(current grants) or
Still in place
(grants that ended)

Courses/seminars in chemistry
Training programs in chemistry
Research and teaching experiences for students and postdocs
Programs for educating the public about chemistry
Mechanisms for recruitment and/or retention of individuals
from underrepresented groups [hyperlink defining URGs]
Mechanisms for mentorship of individuals from
underrepresented groups
Engagement with organizations with expertise supporting
underrepresented groups in the scientific community
Engagement with organizations focused on outreach and
advocacy to pre-college, public, or policymaker audiences
[URGs include women, members of racial and ethnic minorities, persons with disabilities, and persons with low socioeconomic status]

14. Please indicate whether the following improvements have occurred as a result of CCI funding.
Select one response per row.
No improvement
due to CCI
Increased participation of underrepresented groups in your
lab
Increased participation of underrepresented groups at your
institution
Improved ability of students and postdocs to obtain a
position after leaving CCI
Improved quality of education in chemistry

Some
improvement
due to CCI

Large
improvement
due to CCI

No improvement
due to CCI

Some
improvement
due to CCI

Large
improvement
due to CCI

Increased interest in/understanding of chemistry among
the public you were able to reach

15. Please indicate whether the CCI delivered any of the following benefits to industry. Select one
response per row.
No benefit due to
CCI

Some benefit
due to CCI

Large benefit
due to CCI

New or improved ideas for commercial product or process
New or improved product or process
Reduction in environmental impact
Ability to meet regulatory requirements
Access to personnel
Increase in sales
Cost savings
Other benefits. Please specify: ______________

[Center organization and function]

16. On a scale of 1-5, where 1=least and 5=most, how satisfied are you with the following elements
of CCI? To what extent have these elements contributed to the success of your center?

N/A

Requirement to develop a strategic plan
Distribution of resources among partners
Sharing of credit among partners
Policies for conflict resolution
Intellectual contribution of partners
Productivity of meetings among partners
Frequency of meeting among partners
Communication tools
Data sharing tools
Professional development opportunities for students/postdocs
Broadening participation activities and programs
Public outreach activities and programs
Leadership of the center
Overall direction of the center
Funding level for Phase I (Display for Phase I only)
Funding level for Phase II (Display for Phase II only)

Satisfaction:
1-5

Contribution
to success:
None/Some/
Considerable

Duration of Phase I (Display for Phase I only)
Duration of Phase II (Display for Phase II only)

17. Overall, how would you rate your experience in the CCI? Select one. 4
□ Very satisfied
□ Satisfied
□ Somewhat satisfied
□ Dissatisfied
□ Very dissatisfied
18. To what extent has your CCI experienced the following challenges? Have these been resolved?
Challenge

Experienced or not
Yes/No

Resolved
[only if experienced]
No/To some extent/Fully

Technical or experimental challenges
Meeting administrative requirements
Sharing of credit for discovery
Communication between partners
Coordination of activities between partners
Contributions by partners to the center
Delays in progress of research
Staffing of the center
Access to needed resources
Seeding new projects
Terminating unsuccessful projects
Other challenge. Please specify:

19. Phase II only: To what extent participation in Phase I contributed to the success of your
Phase II center? Select one.
□ Not at all
□ To some extent
□ To a considerable extent
20. Have any of the following occurred as a result of your participation in Phase I? Would these
outcomes be more challenging to achieve under a 1-phase center model?
Occurred
Developed new ideas
Advanced research program
Changed direction of research
Formed new or cemented old collaborations

4

AAAS STC Faculty survey item 14

More challenging under a
1-phase center model?

Recruited staff
Obtained additional funding to support this research
Gained experience of participating in or running a center
Developed a center culture across the team
Other benefits. Please describe:
No benefits of participation in Phase I

21. Phase I only: Have you been able to continue working on the CCI-funded projects? Select
one.
□ Yes
□ No
22. In your view, which of the following are the advantages of the 2-phase model? Select all that
apply.
□ It enables selection of better Phase II centers by NSF
□ It allows participants to determine whether they like the experience
□ It allows selection of the right partners
□ It allows the centers to refine their research goals and approach
□ It allows the centers to pilot activities and programs
□ It allows the centers to develop and test policies and procedures
□ No advantages
□ Other advantages. Please specify:____________
23. In your view, what are the disadvantages of the 2-phase model? Select all that apply.
□ Some strong applicants may be discouraged from applying
□ Insufficient resources for Phase I grantees not selected for Phase II to continue their research
□ Time burden to submit a Phase II application
□ Delay in tacking time-sensitive topics
□ Other disadvantages. Please specify:_____________

□

No disadvantages

24. On balance, is a 2-phase center model preferable to a single phase? Select one.
□ Yes, 2-phase model is preferable
□ No, 1-phase model is preferable
□ Uncertain

Pilot only:
In addition to any feedback that you provided in the comment boxes on each page, please answer the
following questions. You may use the survey's navigation buttons (“Back” and “Next”) to remind
yourself of specific questions. Please be honest and critical in your feedback. The study team will use
your feedback to revise and improve the survey before administering it to other CCI participants.
Are the survey items easy to answer?
□ Yes
□ No  Please explain. _____________
Are the survey instructions easy to follow?
□ Yes
□ No  Please explain. _____________
Are the questions in an order that makes logical sense?
□ Yes
□ No  Please explain. _____________
Are the questions clearly worded?
□ Yes
□ No  Please explain. _____________
Does the survey:
□ Have too few questions about each topic?  Please explain. _____________
□ Have the right amount of questions about each topic?
□ Have too many questions about each topic?  Please explain. _____________
Do you have any other comments about the survey?
□ Yes
□ No  Please explain. _____________