Advancing Informal STEM Learning (AISL) Monitoring System, formerly Informal Science Education (ISE)

Attachment D1

Advancing Information STEM Learning (AISL)

Monitoring System,

Formerly Informal Science Education (ISE)

Table of Contents

AISL: Crosswalk – OPMS Baseline Survey 3

AISL: Crosswalk – OPMS Annual Survey 8

AISL: Crosswalk – OPMS Closeout Survey 17

AISL: Number of Respondents, Frequency of Response, and Annual Hour Burden 29

AISL: Hour Burden Estimates by Each Form and Aggregate Hour Burdens 31

AISL: Estimates of Annualized Cost to Respondents for the Hour Burdens 32

AISL: Estimates of Costs to the Federal Government 33

AISL: Questions – OPMS Baseline Survey 34

AISL: Questions – OPMS Annual Survey 39

AISL: Questions – OPMS Closeout Survey 48

AISL: Crosswalk – OPMS Baseline Survey

Baseline Report—Section A: Project and the Lead Organization

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics1	Project Implementation Characteristics2	Project Outputs3
A1	Project Title		X	X
A2	Project Abstract		X	X
A3	Lead organization name	X	X
A4	Lead organization division/department name	X	X
A5a	Lead organization type	X	X
A5b	Division/department type	X	X
A6a1	PI served on previous NSF projects	X
A6a2	PI served on previous AISL projects	X
A6b1	Co-PI served on previous NSF projects	X
A6b2	Co-PI served on previous AISL projects	X
A7a	Public venue name		X
A7b	Public venue zip code		X
A7c	Public venue type		X
A8	Primary point of contact – contact information	X

Baseline Report—Section B: Key Members of the Project Team

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
B	Name of key personnel	X
B1	Affiliated organization name		X
B2a	Organization type		X
B2b	Division/department type		X
B3a	Primary role	X
B3b	Secondary role(s)	X
B4	Contact information	X

Baseline Report—Section C: Organizational Partners

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
C1	Partner organization name		X
C2	Division/department name		X
C3a	Partner organization type		X
C3b	Division/department type		X
C4	Project role	X
C5	Partner organization contact information	X

Baseline Report—Section D: Products, Programs, or Experiences for Public Audiences

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
D1	Does project have public audience deliverables		X	X
D	Deliverable name		X	X
D2	Deliverable type		X	X
D2a	Website components (may not apply)		X	X
D3a-h	Delivery methods		X	X
D3a-h	Anticipated audience count (duplicated, unduplicated, cannot determine)		X	X
D4	Deliverable description		X	X
D5	Primary or secondary project focus		X	X
D6a	Primary STEM content		X	X
D6b	Secondary STEM content		X	X
D7	Interdisciplinary status of deliverable		X	X
D8	Geographic reach		X	X
D9	International reach		X	X
D10	Whether deliverable targets specific public audiences		X	X
D10a	Target audiences		X	X
D10b	Description of steps to target audiences		X	X
D10c	Plans to collect counts of target audiences		X	X
D11	Impact name			X
D11a	Impact category			X
D11b	Indicators			X
D11c	Evaluation design			X
D11d	Data collection methods			X

Baseline Report—Section E: Products, Programs, and Experiences for Professional Audiences

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
E1	Does project have professional audience deliverables		X	X
E	Deliverable name		X	X
E2	Deliverable type		X	X
E3	Deliverable description		X	X
E4	Primary or secondary project focus		X	X
E5a	Primary STEM content		X	X
E5b	Secondary STEM content		X	X
E6	Interdisciplinary status of deliverable		X	X
E7	Geographic reach		X	X
E8	International reach		X	X
E9a	Professional audiences targeted by deliverable		X	X
E9b	Anticipated audience count (duplicated, unduplicated, cannot determine)		X	X
E10	Impacted organizations		X	X
E11	Impact name			X
E11a	Impact category			X
E11b	Indicators			X
E11c	Evaluation design			X
E11d	Data collection methods			X

Baseline Report—Section F: Formative and Summative Evaluation Questions

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
F1	Formative evaluator contact information	X
F2a	Formative evaluation questions			X
F2b	Formative evaluation study designs			X
F3	Summative evaluator contact information	X
F4a	Summative evaluation questions			X
F4b	Summative evaluation study designs			X

AISL: Crosswalk – OPMS Annual Survey

Annual Report—Section A: Project and the Lead Organization

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
A1	Review and update contact information	X
A2a	Public venue name		X
A2b	Public venue zip code		X
A2c	Public venue type		X
A2d	Public venue use in past year		X	X

Annual Report—Section B: Key Members of the Project Team

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
Previously entered key personnel
B1	Update affiliated organization name		X
B2a	Update organization type		X
B2b	Update division/department type		X
B3a	Update primary role	X
B3b	Update secondary role(s)	X
B4	Update contact information	X
B	New key personnel name	X
B5	Affiliated organization name		X
B6a	Organization type		X
B6b	Division/department type		X
B7a	Primary role	X
B7b	Secondary role(s)	X
B8	Contact information	X

Annual Report—Section C: Organizational Partners

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
New partner organization
C1	Partner organization name		X
C2	Division/department name		X
C3a	Partner organization type		X
C3b	Division/department type		X
C4	Project role	X
C5	Partner organization contact information	X
Previously entered partner organization
C6	Update partner contact information	X

Annual Report—Section D: Products, Programs, or Experiences for Public Audiences

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
D1	Does project have public audience deliverables		X	X
Previously Entered Deliverables
D2	Significant changes to deliverable in past year		X	X
D3	Did deliverable reach public audiences in past year		X	X
D4a-f	Delivery methods used in past year		X	X
D4a-f	Audience count for past year (duplicated, unduplicated, cannot determine)		X	X
D5	Geographic reach in past year		X	X
D6	International reach in past year		X	X
D7	Whether deliverable targeted specific public audiences in past year		X	X
D7a	Audiences targeted in past year		X	X
D7b	Description of steps to target audiences		X	X
D8a	Is impact still applicable			X
D8b	Why impact is no longer applicable			X
D9a	Update evaluation design			X
D9b	Update data collection methods			X
D8	Impact name			X
D9c	Impact category			X
D9d	Indicators			X
D9e	Evaluation design			X
D9f	Data collection methods			X
New Deliverable
D	New deliverable name		X	X
D10	Deliverable type		X	X
D10a	Website components (may not apply)		X	X
D11a-h	Delivery methods		X	X
D11a-h	Anticipated audience count (duplicated, unduplicated, cannot determine)		X	X
D12	Deliverable description		X	X
D13	Primary or secondary project focus		X	X
D14a	Primary STEM content		X	X
D14b	Secondary STEM content		X	X
D15	Interdisciplinary status of deliverable		X	X
D16	Geographic reach		X	X
D17	International reach		X	X
D18	Whether deliverable targets specific public audiences		X	X
D18a	Target audiences		X	X
D18b	Description of steps to target audiences		X	X
D18c	Plans to collect counts of target audiences		X	X
D19	Impact name			X
D19a	Impact category			X
D19b	Indicators			X
D19c	Evaluation design			X
D19d	Data collection methods			X
D20	Did deliverable reach public audiences in past year		X	X
D21a-h	Delivery methods used in past year		X	X
D21a-h	Audience count for past year (duplicated, unduplicated, cannot determine)		X	X
D22	Geographic reach in past year		X	X
D23	International reach in past year		X	X
D24	Whether deliverable targeted specific public audiences in past year		X	X
D24a	Audiences targeted in past year		X	X
D24b	Description of steps to target audiences		X	X

Annual Report—Section E: Products, Programs, and Experiences for Professional Audiences

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
E1	Does project have professional audience deliverables		X	X
Previously entered deliverable
E2	Significant changes to deliverable in past year		X	X
E3	Did deliverable reach professional audiences in past year		X	X
E4	Geographic reach in past year		X	X
E5	International reach in past year		X	X
E6a	Professional audiences targeted by deliverable in past year		X	X
E6b	Past year audience count (duplicated, unduplicated, cannot determine)		X	X
E7a	Is impact still applicable			X
E7a	Why impact is no longer applicable			X
E8a	Update evaluation design			X
E8b	Update data collection methods			X
E7	New impact name			X
E8c	Impact category			X
E8d	Indicators			X
E8e	Evaluation design			X
E8f	Data collection methods			X
New Deliverable
E	Deliverable name		X	X
E9	Deliverable type		X	X
E10	Deliverable description		X	X
E11	Primary or secondary project focus		X	X
E12a	Primary STEM content		X	X
E12b	Secondary STEM content		X	X
E13	Interdisciplinary status of deliverable		X	X
E14	Geographic reach		X	X
E15	International reach		X	X
E16a	Professional audiences targeted by deliverable		X	X
E16b	Anticipated audience count (duplicated, unduplicated, cannot determine)		X	X
E17	Impacted organizations		X	X
E18	Impact name			X
E18a	Impact category			X
E18b	Indicators			X
E18c	Evaluation design			X
E18d	Data collection methods			X
E19	Did deliverable reach professional audiences in past year		X	X
E20	Geographic reach in past year		X	X
E21	International reach in past year		X	X
E22a	Professional audiences targeted by deliverable in past year		X	X
E22b	Past year audience count (duplicated, unduplicated, cannot determine)		X	X

Annual Report—Section F: Formative and Summative Evaluation Questions

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
F1	Update formative evaluator contact information	X
F2a	Add new formative evaluation questions			X
F2b	Add new formative evaluation study designs			X
F3	Update summative evaluator contact information	X
F4a	Add new summative evaluation questions			X
F4b	Add new summative evaluation study designs			X

Annual Report—Section G: Project Accomplishments and Lessons Learned

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
G1	Significant accomplishment			X
G2	Progress developing partnerships with organizational partners			X
G3	Progress developing partnerships with formal educational organizations			X
G4	Steps taken to use findings from formative evaluation			X
G5	Lessons learned			X

AISL: Crosswalk – OPMS Closeout Survey

Closeout Report—Section A: Project and the Lead Organization

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
A1	Review and update contact information	X
A2a	Public venue name		X	X
A2b	Public venue zip code		X	X
A2c	Public venue type		X	X
A2d	Public venue use over grant period		X	X

Closeout Report—Section B: Key Members of the Project Team

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
Previously entered key personnel
B1	Update affiliated organization name		X
B2a	Update organization type		X
B2b	Update division/department type		X
B3a	Update primary role	X
B3b	Update secondary role(s)	X
B4	Update contact information	X
New key personnel
B	New key personnel name	X
B5	Affiliated organization name		X
B6a	Organization type		X
B6b	Division/department type		X
B7a	Primary role	X
B7b	Secondary role(s)	X
B8	Contact information	X

Closeout Report—Section C: Organizational Partners

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
New partner organization
C1	Partner organization name		X
C2	Division/department name		X
C3a	Partner organization type		X
C3b	Division/department type		X
C4	Project role	X
C5	Partner organization contact information	X
Previously entered partner organization
C6	Update partner contact information	X

Closeout Report—Section D: Products, Programs, or Experiences for Public Audiences

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
D1	Does project have public audience deliverables		X	X
Previously Entered Deliverables
D2	Status of deliverable			X
D2a	If deliverable complete: Describe accomplishments, innovations, and lessons learned for deliverable			X
D2b	If incomplete: Describe why not complete, future plans for deliverable			X
D2c	If work on deliverable suspended or cancelled: Describe why suspended or cancelled, future plans for deliverable			X
D3	Did deliverable reach public audiences during grant period		X	X
D4a-f	Delivery methods used during grant period		X	X
D4a-f	Audience count during grant period (duplicated, unduplicated, cannot determine)		X	X
D5	Geographic reach during grant period		X	X
D6	International reach during grant period		X	X
D7	Whether deliverable targeted specific public audiences during grant period		X	X
D7a	Audiences targeted during grant period		X	X
D7b	Description of steps to target audiences during grant period		X	X
D8	Did project collect counts of target audience during grant period		X	X
D10a	Update evaluation design			X
D10b	Update data collection methods			X
D9	Impact name			X
D12a	Impact category			X
D12b	Indicators			X
D12c	Evaluation design			X
D12d	Data collection methods			X
D11	Extent to which impact was achieved (applies to new and previously entered impacts)			X
D11a	If met and major accomplishment: Provide evidence impact was met and explain why major accomplishment			X
D11a	If impact met: Provide evidence impact was met			X
D11c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment			X
D11d	If progress toward impact not measured: Describe why progress was not measured			X
D11e	If impact no longer applicable: Describe why impact no longer applies to deliverable			X
New Deliverable
D	New deliverable name		X	X
D13	Deliverable type		X	X
D13a	Website components (may not apply)		X	X
D14	Deliverable description		X	X
D15	Primary or secondary project focus		X	X
D16a	Primary STEM content		X	X
D16b	Secondary STEM content		X	X
D17	Interdisciplinary status of deliverable		X	X
D19	Impact name			X
D19a	Impact category			X
D19b	Indicators			X
D19c	Evaluation design			X
D19d	Data collection methods			X
D19	Extent to which impact was achieved (applies to new and previously entered impacts)			X
D19a	If met and major accomplishment: Provide evidence impact was met and explain why major accomplishment			X
D19a	If impact met: Provide evidence impact was met			X
D19c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment			X
D19d	If progress toward impact not measured: Describe why progress was not measured			X
D19e	If impact no longer applicable: Describe why impact no longer applies to deliverable			X
D20	Did deliverable reach public audiences during grant period		X	X
D21a-h	Delivery methods used during grant period		X	X
D21a-h	Audience count during grant period (duplicated, unduplicated, cannot determine)		X	X
D22	Geographic reach during grant period		X	X
D23	International reach during grant period		X	X
D24	Whether deliverable targeted specific public audiences during grant period		X	X
D24a	Audiences targeted during grant period		X	X
D24b	Description of steps to target audiences		X	X
D25	Did project obtain counts of target audiences during grant period		X	X

Closeout Report—Section E: Products, Programs, and Experiences for Professional Audiences

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
E1	Does project have professional audience deliverables		X	X
Previously entered deliverable
E2	Status of deliverable			X
E2a	If deliverable complete: Describe accomplishments, innovations, and lessons learned for deliverable			X
E2b	If incomplete: Describe why not complete, future plans for deliverable			X
E2b	If work on deliverable suspended or cancelled: Describe why suspended or cancelled, future plans for deliverable			X
E3	Did deliverable reach professional audiences during grant period		X	X
E4	Geographic reach during grant period		X	X
E5	International reach during grant period		X	X
E6a	Professional audiences targeted by deliverable during grant period		X	X
E6b	Grant period audience count (duplicated, unduplicated, cannot determine)		X	X
E8a	Update evaluation design			X
E8b	Update data collection methods			X
E7	New impact name			X
E9a	New impact category			X
E9b	New impact indicators			X
E9c	New impact evaluation design			X
E9d	New impact data collection methods			X
E10	Extent to which impact was achieved (applies to new and previously entered impacts)			X
E10a	If met and major accomplishment: Provide evidence impact was met/explain why major accomplishment			X
E10b	If impact met: Provide evidence impact was met			X
E10c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment			X
E10d	If progress toward impact not measured: Describe why progress was not measured			X
E10e	If impact no longer applicable: Describe why impact no longer applies to deliverable			X
New Deliverable
E	Deliverable name		X	X
E11	Deliverable type		X	X
E12	Deliverable description		X	X
E13	Primary or secondary project focus		X	X
E14a	Primary STEM content		X	X
E14b	Secondary STEM content		X	X
E15	Interdisciplinary status of deliverable		X	X
E17	Impacted organizations during grant period		X	X
E17	Impact name			X
E18a	Impact category			X
E18b	Indicators			X
E18c	Evaluation design			X
E18d	Data collection methods			X
E19	Extent to which impact was achieved (applies to new and previously entered impacts)			X
E19a	If met and major accomplishment: Provide evidence impact was met and explain why major accomplishment			X
E19a	If impact met: Provide evidence impact was met			X
E19c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment			X
E19d	If progress toward impact not measured: Describe why progress was not measured			X
E19e	If impact no longer applicable: Describe why impact no longer applies to deliverable			X
E20	Did deliverable reach professional audiences during grant period?		X	X
E21	Geographic reach during grant period		X	X
E22	International reach during grant period		X	X
E23a	Professional audiences targeted by deliverable during grant period		X	X
E23b	Grant period audience count (duplicated, unduplicated, cannot determine)		X	X

Closeout Report—Section F: Formative and Summative Evaluation Questions

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Outputs
F1	Update formative evaluator contact information	X
F2	Add new formative evaluation questions			X
F2	Add new formative evaluation study designs			X
F2a	Progress toward answering formative evaluation question			X
F2b	If formative question no longer applicable: Explain why no longer applicable			X
F3	Update summative evaluator contact information	X
F4	Add new summative evaluation questions			X
F4	Add new summative evaluation study designs			X
F4a	Progress toward answering summative evaluation question			X
F4b	If summative question no longer applicable: Explain why no longer applicable			X

Closeout Report—Section G: Project Accomplishments and Lessons Learned

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Output
G1	Significant accomplishment			X
G2	How did project advance knowledge			X
G3	Unanticipated benefits or impacts			X
G4	Significant innovations			X
G5	Lessons learned			X
G6	Steps taken to use findings from formative evaluation			X
G7	Approaches to disseminate information to AISL field			X
G8	Honors, awards, or recognitions			X
G9	Print and/or media coverage about project			X
G10	Upload project products			X
G11	Enter project websites or sites with project materials			X

Closeout Report—Section H: Future Directions

Item Number	Item	Common Collection Categories
		Staff and Project Participant Characteristics	Project Implementation Characteristics	Project Output
H1	Will project continue beyond grant period			X
H2a	Future plans			X
H2b	Why not planning to continue			X

AISL: Number of Respondents, Frequency of Response, and Annual Hour Burden

The estimated annual response burden is approximately 2,047 person-hours.

Table 2 provides an overview of the types of projects that will enter data directly into the OPMS and the project types that will have a subset of the survey items entered by Westat.

Table 2. Number of anticipated new awards and the methods that will be used to obtain OPMS data

Project Type	Anticipated number of new awards per year	How data will be entered into the OPMS
Research	5	Initial, annual, and closeout data will be entered by projects directly into the AISL OPMS
CRPA	8
Full-Scale Development	25
Broad Implementation	3
Pathways	5	Initial data will be entered into the AISL OPMS by Westat using information from proposals
Conferences	8
RAPID
EAGER

In all cases, the respondents are PIs on AISL-supported projects. Burden hours per response are estimated on the basis of discussions with NSF program officers, a sample of AISL PIs, and experience in administrating similar surveys. The calculations used to determine the overall response burden of 2,047 hours are shown in Table 3.

Table 3. Calculations used to estimate annual response burden for the OPMS during a typical collection cycle

Respondent Type	Number of Annual Respondents	Hours per Respondent Type	Annual Person-Hour Total
Initial Survey for Research, CRPA, Full-Scale Development, and Broad Implementation Projects	41	24	984
Initial Survey for Pathways, Conferences, RAPID, and EAGER Projects	13	0.5	6.5
Annual Survey Research, CRPA, Full-Scale Development, and Broad Implementation Projects	74	8	592
Closeout Survey Research, CRPA, Full-Scale Development, and Broad Implementation Projects	29	16	464
Total	157		2,046.5

Note: Pathways, Conferences, RAPID, and EAGER projects do not complete an Annual or Closeout Survey.

AISL: Hour Burden Estimates by Each Form and Aggregate Hour Burdens

The hour burden estimates by type of module are presented in Table 3 above.

AISL: Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall annualized cost to the respondents is estimated to be $102,182. The hourly wage rate is based on information found in the U.S. Department of Labor, Bureau of Labor Statistics, National Compensation Survey (Table 3, National Compensation Survey: Occupational Wages in the United States, Bulletin 2753, May 2011), adjusted for inflation. Calculations are shown in Table 4. Projects are encouraged to include evaluators in the OPMS completion process. Westat estimates that evaluators’ wages are approximately equivalent to those of the PI respondents. Further, any time evaluators devote to completing the OPMS would reduce the time that PIs spend completing the system, so the number of hours and total dollar burden would not vary due to evaluator involvement.

Table 4. Calculations used to estimate cost burden for the OPMS for the 2012 collection cycle

Respondent Type	Calculation (# of respondents x hourly wage x hours to complete)	Total Dollars
Initial Submission
PIs: Full-Scale, Research, CRPA, and Broad Implementation	41 respondents x $49.93/hour x 24 hours to complete	$49,131.12
PIs: Conferences, Pathways, EAGER, RAPID	13 respondents x $49.93/hour x 0.5 hours to complete	$324.55
Annual Report
PIs: Full-Scale, Research, CRPA, and Broad Implementation	74 respondents x $49.93/hour x 8 hours to complete	$29,558.56
Closeout Report
PIs: Full-Scale, Research, CRPA, and Broad Implementation	29 respondents x $49.93/hour x 16 hours to complete	$23,167.52
Total		$102,181.75

AISL: Estimates of Costs to the Federal Government

The total estimated cost to the government of all data collection, analysis, and reporting activities for this study is approximately $256,725. This estimate is based on the actual annual cost of the collection in 2011 (shown in Table 5).

Table 5. Estimated annual cost to the Federal government of collection (based on 2011 expenditures)

Personnel	$81,172.59
Travel and field expenses	$269.84
Computing	$11,641.87
Overhead	$88,413.94
G&A and Fee	$75,226.32
Total Costs	$256,724.56

Note: Excludes costs expended for significant revisions to the OPMS system architecture.

AISL: Questions – OPMS Baseline Survey

Baseline Report—Section A: Project and the Lead Organization

Item Number	Item
A1	Project Title
A2	Project Abstract
A3	Lead organization name
A4	Lead organization division/department name
A5a	Lead organization type
A5b	Division/department type
A6a1	PI served on previous NSF projects
A6a2	PI served on previous AISL projects
A6b1	Co-PI served on previous NSF projects
A6b2	Co-PI served on previous AISL projects
A7a	Public venue name
A7b	Public venue zip code
A7c	Public venue type
A8	Primary point of contact – contact information

Baseline Report—Section B: Key Members of the Project Team

Item Number	Item
B	Name of key personnel
B1	Affiliated organization name
B2a	Organization type
B2b	Division/department type
B3a	Primary role
B3b	Secondary role(s)
B4	Contact information

Baseline Report—Section C: Organizational Partners

Item Number	Item
C1	Partner organization name
C2	Division/department name
C3a	Partner organization type
C3b	Division/department type
C4	Project role
C5	Partner organization contact information

Baseline Report—Section D: Products, Programs, or Experiences for Public Audiences

Item Number	Item
D1	Does project have public audience deliverables
D	Deliverable name
D2	Deliverable type
D2a	Website components (may not apply)
D3a-h	Delivery methods
D3a-h	Anticipated audience count (duplicated, unduplicated, cannot determine)
D4	Deliverable description
D5	Primary or secondary project focus
D6a	Primary STEM content
D6b	Secondary STEM content
D7	Interdisciplinary status of deliverable
D8	Geographic reach
D9	International reach
D10	Whether deliverable targets specific public audiences
D10a	Target audiences
D10b	Description of steps to target audiences
D10c	Plans to collect counts of target audiences
D11	Impact name
D11a	Impact category
D11b	Indicators
D11c	Evaluation design
D11d	Data collection methods

Baseline Report—Section E: Products, Programs, and Experiences for Professional Audiences

Item Number	Item
E1	Does project have professional audience deliverables
E	Deliverable name
E2	Deliverable type
E3	Deliverable description
E4	Primary or secondary project focus
E5a	Primary STEM content
E5b	Secondary STEM content
E6	Interdisciplinary status of deliverable
E7	Geographic reach
E8	International reach
E9a	Professional audiences targeted by deliverable
E9b	Anticipated audience count (duplicated, unduplicated, cannot determine)
E10	Impacted organizations
E11	Impact name
E11a	Impact category
E11b	Indicators
E11c	Evaluation design
E11d	Data collection methods

Baseline Report—Section F: Formative and Summative Evaluation Questions

Item Number	Item
F1	Formative evaluator contact information
F2a	Formative evaluation questions
F2b	Formative evaluation study designs
F3	Summative evaluator contact information
F4a	Summative evaluation questions
F4b	Summative evaluation study designs

AISL: Questions – OPMS Annual Survey

Annual Report—Section A: Project and the Lead Organization

Item Number	Item
A1	Review and update contact information
A2a	Public venue name
A2b	Public venue zip code
A2c	Public venue type
A2d	Public venue use in past year

Annual Report—Section B: Key Members of the Project Team

Item Number	Item
Previously Entered Key Personnel
B1	Update affiliated organization name
B2a	Update organization type
B2b	Update division/department type
B3a	Update primary role
B3b	Update secondary role(s)
B4	Update contact information
B	New key personnel name
B5	Affiliated organization name
B6a	Organization type
B6b	Division/department type
B7a	Primary role
B7b	Secondary role(s)
B8	Contact information

Annual Report—Section C: Organizational Partners

Item Number	Item
New Partner Organization
C1	Partner organization name
C2	Division/department name
C3a	Partner organization type
C3b	Division/department type
C4	Project role
C5	Partner organization contact information
Previously Entered Partner Organization
C6	Update partner contact information

Annual Report—Section D: Products, Programs, or Experiences for Public Audiences

Item Number	Item
D1	Does project have public audience deliverables
Previously Entered Deliverables
D2	Significant changes to deliverable in past year
D3	Did deliverable reach public audiences in past year
D4a-f	Delivery methods used in past year
D4a-f	Audience count for past year (duplicated, unduplicated, cannot determine)
D5	Geographic reach in past year
D6	International reach in past year
D7	Whether deliverable targeted specific public audiences in past year
D7a	Audiences targeted in past year
D7b	Description of steps to target audiences
D8a	Is impact still applicable
D8b	Why impact is no longer applicable
D9a	Update evaluation design
D9b	Update data collection methods
D8	Impact name
D9c	Impact category
D9d	Indicators
D9e	Evaluation design
D9f	Data collection methods
New Deliverable
D	New deliverable name
D10	Deliverable type
D10a	Website components (may not apply)
D11a-h	Delivery methods
D11a-h	Anticipated audience count (duplicated, unduplicated, cannot determine)
D12	Deliverable description
D13	Primary or secondary project focus
D14a	Primary STEM content
D14b	Secondary STEM content
D15	Interdisciplinary status of deliverable
D16	Geographic reach
D17	International reach
D18	Whether deliverable targets specific public audiences
D18a	Target audiences
D18b	Description of steps to target audiences
D18c	Plans to collect counts of target audiences
D19	Impact name
D19a	Impact category
D19b	Indicators
D19c	Evaluation design
D19d	Data collection methods
D20	Did deliverable reach public audiences in past year
D21a-h	Delivery methods used in past year
D21a-h	Audience count for past year (duplicated, unduplicated, cannot determine)
D22	Geographic reach in past year
D23	International reach in past year
D24	Whether deliverable targeted specific public audiences in past year
D24a	Audiences targeted in past year
D24b	Description of steps to target audiences

Annual Report—Section E: Products, Programs, and Experiences for Professional Audiences

Item Number	Item
E1	Does project have professional audience deliverables
Previously Entered Deliverable
E2	Significant changes to deliverable in past year
E3	Did deliverable reach professional audiences in past year
E4	Geographic reach in past year
E5	International reach in past year
E6a	Professional audiences targeted by deliverable in past year
E6b	Past year audience count (duplicated, unduplicated, cannot determine)
E7a	Is impact still applicable
E7a	Why impact is no longer applicable
E8a	Update evaluation design
E8b	Update data collection methods
E7	New impact name
E8c	Impact category
E8d	Indicators
E8e	Evaluation design
E8f	Data collection methods
New Deliverable
E	Deliverable name
E9	Deliverable type
E10	Deliverable description
E11	Primary or secondary project focus
E12a	Primary STEM content
E12b	Secondary STEM content
E13	Interdisciplinary status of deliverable
E14	Geographic reach
E15	International reach
E16a	Professional audiences targeted by deliverable
E16b	Anticipated audience count (duplicated, unduplicated, cannot determine)
E17	Impacted organizations
E18	Impact name
E18a	Impact category
E18b	Indicators
E18c	Evaluation design
E18d	Data collection methods
E19	Did deliverable reach professional audiences in past year
E20	Geographic reach in past year
E21	International reach in past year
E22a	Professional audiences targeted by deliverable in past year
E22b	Past year audience count (duplicated, unduplicated, cannot determine)

Annual Report—Section F: Formative and Summative Evaluation Questions

Item Number	Item
F1	Update formative evaluator contact information
F2a	Add new formative evaluation questions
F2b	Add new formative evaluation study designs
F3	Update summative evaluator contact information
F4a	Add new summative evaluation questions
F4b	Add new summative evaluation study designs

Annual Report—Section G: Project Accomplishments and Lessons Learned

Item Number	Item
G1	Significant accomplishment
G2	Progress developing partnerships with organizational partners
G3	Progress developing partnerships with formal educational organizations
G4	Steps taken to use findings from formative evaluation
G5	Lessons learned

AISL: Questions – OPMS Closeout Survey

Closeout Report—Section A: Project and the Lead Organization

Item Number	Item
A1	Review and update contact information
A2a	Public venue name
A2b	Public venue zip code
A2c	Public venue type
A2d	Public venue use over grant period

Closeout Report—Section B: Key Members of the Project Team

Item Number	Item
Previously Entered Key Personnel
B1	Update affiliated organization name
B2a	Update organization type
B2b	Update division/department type
B3a	Update primary role
B3b	Update secondary role(s)
B4	Update contact information
New Key Personnel
B	New key personnel name
B5	Affiliated organization name
B6a	Organization type
B6b	Division/department type
B7a	Primary role
B7b	Secondary role(s)
B8	Contact information

Closeout Report—Section C: Organizational Partners

Item Number	Item
New Partner Organization
C1	Partner organization name
C2	Division/department name
C3a	Partner organization type
C3b	Division/department type
C4	Project role
C5	Partner organization contact information
Previously Entered Partner Organization
C6	Update partner contact information

Closeout Report—Section D: Products, Programs, or Experiences for Public Audiences

Item Number	Item
D1	Does project have public audience deliverables
Previously Entered Deliverables
D2	Status of deliverable
D2a	If deliverable complete: Describe accomplishments, innovations, and lessons learned for deliverable
D2b	If incomplete: Describe why not complete, future plans for deliverable
D2c	If work on deliverable suspended or cancelled: Describe why suspended or cancelled, future plans for deliverable
D3	Did deliverable reach public audiences during grant period
D4a-f	Delivery methods used during grant period
D4a-f	Audience count during grant period (duplicated, unduplicated, cannot determine)
D5	Geographic reach during grant period
D6	International reach during grant period
D7	Whether deliverable targeted specific public audiences during grant period
D7a	Audiences targeted during grant period
D7b	Description of steps to target audiences during grant period
D8	Did project collect counts of target audience during grant period
D10a	Update evaluation design
D10b	Update data collection methods
D9	Impact name
D12a	Impact category
D12b	Indicators
D12c	Evaluation design
D12d	Data collection methods
D11	Extent to which impact was achieved (applies to new and previously entered impacts)
D11a	If met and major accomplishment: Provide evidence impact was met and explain why major accomplishment
D11a	If impact met: Provide evidence impact was met
D11c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment
D11d	If progress toward impact not measured: Describe why progress was not measured
D11e	If impact no longer applicable: Describe why impact no longer applies to deliverable
New Deliverables
D	New deliverable name
D13	Deliverable type
D13a	Website components (may not apply)
D14	Deliverable description
D15	Primary or secondary project focus
D16a	Primary STEM content
D16b	Secondary STEM content
D17	Interdisciplinary status of deliverable
D19	Impact name
D19a	Impact category
D19b	Indicators
D19c	Evaluation design
D19d	Data collection methods
D19	Extent to which impact was achieved (applies to new and previously entered impacts)
D19a	If met and major accomplishment: Provide evidence impact was met and explain why major accomplishment
D19a	If impact met: Provide evidence impact was met
D19c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment
D19d	If progress toward impact not measured: Describe why progress was not measured
D19e	If impact no longer applicable: Describe why impact no longer applies to deliverable
D20	Did deliverable reach public audiences during grant period
D21a-h	Delivery methods used during grant period
D21a-h	Audience count during grant period (duplicated, unduplicated, cannot determine)
D22	Geographic reach during grant period
D23	International reach during grant period
D24	Whether deliverable targeted specific public audiences during grant period
D24a	Audiences targeted during grant period
D24b	Description of steps to target audiences
D25	Did project obtain counts of target audiences during grant period

Closeout Report—Section E: Products, Programs, and Experiences for Professional Audiences

Item Number	Item
E1	Does project have professional audience deliverables
Previously Entered Deliverables
E2	Status of deliverable
E2a	If deliverable complete: Describe accomplishments, innovations, and lessons learned for deliverable
E2b	If incomplete: Describe why not complete, future plans for deliverable
E2b	If work on deliverable suspended or cancelled: Describe why suspended or cancelled, future plans for deliverable
E3	Did deliverable reach professional audiences during grant period
E4	Geographic reach during grant period
E5	International reach during grant period
E6a	Professional audiences targeted by deliverable during grant period
E6b	Grant period audience count (duplicated, unduplicated, cannot determine)
E8a	Update evaluation design
E8b	Update data collection methods
E7	New impact name
E9a	New impact category
E9b	New impact indicators
E9c	New impact evaluation design
E9d	New impact data collection methods
E10	Extent to which impact was achieved (applies to new and previously entered impacts)
E10a	If met and major accomplishment: Provide evidence impact was met/explain why major accomplishment
E10b	If impact met: Provide evidence impact was met
E10c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment
E10d	If progress toward impact not measured: Describe why progress was not measured
E10e	If impact no longer applicable: Describe why impact no longer applies to deliverable
New Deliverables
E	Deliverable name
E11	Deliverable type
E12	Deliverable description
E13	Primary or secondary project focus
E14a	Primary STEM content
E14b	Secondary STEM content
E15	Interdisciplinary status of deliverable
E17	Impacted organizations during grant period
E17	Impact name
E18a	Impact category
E18b	Indicators
E18c	Evaluation design
E18d	Data collection methods
E19	Extent to which impact was achieved (applies to new and previously entered impacts)
E19a	If met and major accomplishment: Provide evidence impact was met and explain why major accomplishment
E19a	If impact met: Provide evidence impact was met
E19c	If impact was not met: Describe factors that hindered projects, future steps to achieve impact, methods to document impact attainment
E19d	If progress toward impact not measured: Describe why progress was not measured
E19e	If impact no longer applicable: Describe why impact no longer applies to deliverable
E20	Did deliverable reach professional audiences during grant period?
E21	Geographic reach during grant period
E22	International reach during grant period
E23a	Professional audiences targeted by deliverable during grant period
E23b	Grant period audience count (duplicated, unduplicated, cannot determine)

Closeout Report—Section F: Formative and Summative Evaluation Questions

Item Number	Item
F1	Update formative evaluator contact information
F2	Add new formative evaluation questions
F2	Add new formative evaluation study designs
F2a	Progress toward answering formative evaluation question
F2b	If formative question no longer applicable: Explain why no longer applicable
F3	Update summative evaluator contact information
F4	Add new summative evaluation questions
F4	Add new summative evaluation study designs
F4a	Progress toward answering summative evaluation question
F4b	If summative question no longer applicable: Explain why no longer applicable

Closeout Report—Section G: Project Accomplishments and Lessons Learned

Item Number	Item
G1	Significant accomplishment
G2	How did project advance knowledge
G3	Unanticipated benefits or impacts
G4	Significant innovations
G5	Lessons learned
G6	Steps taken to use findings from formative evaluation
G7	Approaches to disseminate information to AISL field
G8	Honors, awards, or recognitions
G9	Print and/or media coverage about project
G10	Upload project products
G11	Enter project websites or sites with project materials

Closeout Report—Section H: Future Directions

Item Number	Item
H1	Will project continue beyond grant period
H2a	Future plans
H2b	Why not planning to continue

1 Elements of these characteristics include: Name, address, date of birth, gender, ethnicity, race, disability status, class, major, grade point average, yearly fellowship or stipend amount, or project role.

2 Elements of these characteristics include: Sources and amount of funds, fellowships, scholarships, traineeships, partnerships, training, and research methods.

3 Elements of these characteristics include: Research findings, publications, presentations, degrees granted, and educational materials.

Page 58 of 58