GPS-STEM Supp and Part B

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

This survey will be conducted by contractors acting on behalf of the NOAA Office of Space Commerce and the National Coordination Office for Space-Based Positioning, Navigation, and Timing. These program offices have developed a new Science, Technology, Engineering, and Mathematics (STEM) curriculum for middle school students and made it available to teachers at www.gps.gov/students/curriculum. The survey will help the government evaluate adoption rates and marketing channels for the new curriculum. The customers are Middle School teachers, who are able to download the lesson plans from the website at no charge.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

The survey was developed with support of the creators of the GPS-Based STEM Curriculum, who consulted with Sarah Brabson, PRA Clearance Officer for NOAA. Suggestions for improving the survey were provided by the project team and subject matter experts who formerly worked for NOAA Education. The survey intentionally requests as little information as possible.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate?

The survey will be conducted passively online on the same website as the free STEM lessons offered to school teachers. Customers filling out the survey will be sampled against the current subscriber list to an email newsletter. This newsletter is opt-in for teachers that want to know about GPS-STEM Curriculum updates, including free new lessons and materials. Every two to three weeks a newsletter is sent out. This newsletter will include a reminder that the teachers can obtain a completion certificate for their class if they fill out the survey. We expect that less than half of subscribing users will verify using the lessons by filling out the survey but have no means of testing these expectations until the survey is released on the website.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

Sampling will use simple stochastic flow model of site visits, subscribers, downloads, and those filling out the survey. This model will use binomial distributions to predict the likelihood that those who subscribe to the website will complete a lesson. A simple linear correlation study between site visits, subscribers, downloads, and survey completion in total numbers to help approximate the number of teachers actively using the GPS-Based STEM Curriculum nationwide. Information on specific lessons identified in the survey will be compared with lesson site page visits. Information on geographic distribution will be evaluated to understand the reach of distribution channels.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The respondents to this survey will be a subset of the science teachers that have completed one or more lessons from the GPS-STEM lesson curriculum. These teachers are usually working in either elementary or middle school and are based throughout the United States, predominantly.

As of October 3, 2018, there are approximately 1200 users per month, averaging 240 per week, and about 40 users per day. There are also 240 subscribers, mostly STEM teachers, who receive email notifications when updates (e.g. new material) is added to the website and curriculum.

In the last month, 57 additional subscribers have been added, coinciding with ongoing efforts to promote the curriculum to science teachers. This means about 25% of those who visit subscribe. We expect about 25% of those who subscribe will also choose to download the certificate.

This means over the next two months (the period of performance for collecting data) we expect to gain about 120 new subscribers. Since the lessons take a day to a week to complete, we expect that there will be about a one-month lag in those subscribing and those who complete the lesson and download the certificate. Therefore, the final population of subscribers for this study is estimated to be approximately 300 total science teachers (240 + 60 additional teachers in October). The number of expected downloads of the certificate is therefore approximately 75 unique science teachers (25% * 300 subscribers).

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The sampling method is entirely passive and random from within the subscriber community of science teachers using the GPS-STEM curriculum. The certificate used for sampling will be offered as a new opportunity for science teachers adopting the GPS-Based STEM curriculum via a newsletter sent to the subscribing teachers. The estimate procedure for how these compare to the total population of teachers using the curriculum will be made post-hoc using conditional probabilities by comparing the general website analytics for users of the website and the respondents ZIP code locality. This is a highly generalized comparison using conditional probabilities, with added power through binomial statistical analysis.

The periodic sampling will be on a bimonthly basis (every two months) per the period of performance of the survey and the end of the contract for the contractors supporting the study.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The response rate will be amplified through ongoing distribution of the GPS-STEM curriculum to science and math teachers. This will be conducted through the channels available, including through professional organizations (e.g. National Science Teacher Association, “NSTA”), the NOAA Education website, email newsletters managed by supporting government agencies, search engine optimization (SEO), search engine marketing (SEM), and newsletters to subscribing teachers. This serves the purpose of distributing these highly valuable materials to science and math teachers who need them, and to promote the available certificate to award to students and classrooms that complete the lessons within the curriculum.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

No test of the procedures are possible or available at this time, or plan to be taken.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

John Johnson, PMP CSM SPC

Chief Technology Officer

SOFTEK ENTERPRISES

[email protected] | http://www.softekenterprises.com

412.608.2654 (c) | 443.741.8715 (o) | 267.575.4150 (f)

9250 Bendix Road N., Suite 620

Columbia, MD 21045