1_2017 ECDS OMB Supporting Statement_Part A (Updated 25Sept2017)

SUPPORTING STATEMENT
for the
2017 Early Career Doctorates Survey

Section
Page
A. Justification ......................................................................................................................... 1
A.1

Necessity for Information Collection ....................................................................... 3

A.2

Uses of Information .................................................................................................. 3

A.3

Consideration of Using Improved Technology ........................................................ 4

A.4

Efforts to Identify Duplication ................................................................................. 5

A.5

Efforts to Minimize Burden on Small Business ....................................................... 6

A.6

Consequences of Less Frequent Data Collection ..................................................... 6

A.7

Special Circumstances .............................................................................................. 6

A.8

Federal Register Announcement and Consultations outside the Agency................. 6

A.9

Payment or Gifts to Respondents ............................................................................. 8

A.10 Assurance of Confidentiality .................................................................................... 8
A.11 Justification for Sensitive Questions ........................................................................ 9
A.12 Estimate of Respondent Burden ............................................................................... 9
A.13 Cost Burden to Respondents .................................................................................. 11
A.14 Cost Burden to the Federal Government ................................................................ 11
A.15 Reason for Change in Burden ................................................................................ 11
A.16 Schedule for Information Collection and Publication ............................................ 11
A.17 Display of OMB Expiration Date ........................................................................... 12
A.18 Exception to the Certification Statement................................................................ 12
B. Collection of Information Employing Statistical Methods ............................................... 13
B.1

Universe and Sampling Procedure ......................................................................... 13

B.2

Description of Survey Methodology and Statistical Procedures ............................ 17
B.2.1 Imputation for Item Nonresponse in the 2017 ECDS ................................18

B.3

Methods Used To Maximize Response Rate ......................................................... 18

B.4

Testing of Procedures ............................................................................................. 20
B.4.1
B.4.2
B.4.3
B.4.4
B.4.5

B.5

Test of Building Sample Frame .................................................................20
Experimental Results for the Pilot ECDS ..................................................21
Experiments for the 2017 ECDS................................................................23
Changes to the ECDS Questionnaire .........................................................24
Evaluation to Determine the Future of the ECD Project ...........................25

Names and Telephone Numbers of Individuals Consulted .................................... 26

Early Career Doctorates Survey

i

TABLES
Number

Page

Table 1.

Estimated Burden by Stage and Respondent Type: 2017 ECDS ........................10

Table 2.

ECDS Data Collection and Publication Schedule ..............................................11

Table 3.

Anticipated Rates of Stage 1 Nonparticipation and Stage 2 Ineligibility
and Nonresponse .................................................................................................14

Table 4.

ECDS Sampling Strata and Estimated Sample Sizes .........................................14

Table 5.

Minimum Sample Size and Coefficient Variation (CV) by Key Domains
of Interest ...........................................................................................................15

Table 6.

E-mail Subject Line Experimental and Control Conditions ...............................21

Table 7.

Response by E-mail Subject Line Treatment......................................................22

Table 8.

Response by Mail Class Treatment.....................................................................22

Table 9.

Individual Consulted on ECDS Technical and Statistical Issues .......................26

LIST OF ATTACHMENTS
Attachment A. Contact Strategy Narrative
Attachment B. Institutional Contact Materials
Attachment C. ECD Contact Materials
Attachment D. Confidentiality and Security Measures
Attachment E. ECDS Questionnaire
Attachment F.

Federal Register Announcements

Attachment G. HA / LC Web Tools and Information
Attachment H. Sample Design and Sampling Plan
Attachment I.

References

Early Career Doctorates Survey

ii

A.

JUSTIFICATION

The National Center for Science and Engineering Statistics (NCSES) within the National Science
Foundation (NSF) serves as a central Federal clearinghouse for the collection, interpretation,
analysis, and dissemination of objective data on science, engineering, technology, and research
and development for use by practitioners, researchers, policymakers, and the public. The Early
Career Doctorates Survey (ECDS), co-sponsored by the National Institutes of Health (NIH), is
the newest part of an integrated survey system that meets the human resources part of this
mission.
The Early Career Doctorates Project was established to gather in-depth information about early
career doctorates (ECD), including postdoctoral researchers (postdocs). ECD are critical to the
success of the U.S. scientific enterprise and will influence U.S. and global scientific markets for
years to come. Despite their importance, extant surveys of this population have significant
coverage issues and are limited by the lack of individual level data on the ECD work
experiences. The NSF’s Survey of Earned Doctorates (SED) and the Survey of Doctorate
Recipients (SDR) are limited to individuals who received their research doctorates from U.S.
academic institutions, thereby excluding individuals who earned professional doctoral degrees,
as well as those who earned research doctorates from institutions outside the United States but
are currently employed in the United States. The NSF’s Survey of Graduate Students and
Postdoctorates in Science and Engineering (GSS) provides data on postdocs and non-faculty
researchers working in science, engineering, and selected health (SEH) fields regardless of where
they earned the degree, but is limited to U.S. academic institutions and collects aggregate count
data at the program or research unit level. The 2005 Sigma Xi Postdoc Survey and current
University of Chicago National Postdoc Survey1 had (have) significant sample design issues
with respect to identifying and recruiting postdocs.
Through its multi-year Postdoc Data Project (PDP), NCSES determined the need for and the
feasibility of gathering information about postdocs and other ECD working in the United States.
The PDP aimed to increase the coverage of postdocs working in non-academic sectors. NCSES
assessed various options for filling the coverage and content gaps across all sectors. Several
recommendations for list sources emerged to address these gaps in the government sector, and
quasi-government/non-profit sector. Efforts were unsuccessful in making substantial inroads into
providing good coverage of postdocs in the private sector. Additionally, efforts to reliably
identify and gather information about postdocs at the individual level proved difficult due to
substantial variation in how institutions characterize and report postdoc appointments. As a
result, NCSES decided to expand the target population to include all individuals who earned
their first doctoral degree within the past 10 years. With this expansion of the target population,
the Postdoc Data Project was transformed into the Early Career Doctorates Project (ECDP).
Unique in scope, the key goals of the ECDP are:

1

For more information on these surveys, please see http://www.sigmaxi.org/docs/default-source/ProgramsDocuments/Critical-Issues-in-Science/postdoc-survey/highlights and www.postdocsurvey.org, respectively.

Early Career Doctorates Survey

1

• To broaden the scope and depth of national statistics on the ECD population working in the
United States (both U.S. degreed and non-U.S. degreed) across various employment sectors
and fields of discipline,
• To collect nationally representative data from ECD that can be used by funding agencies,
policy makers, and other researchers to better understand the labor markets for and work
experiences of recent doctorate recipients, and
• To develop common definitions for postdocs and other types of ECD (e.g., junior faculty and
non-faculty researchers) that can be applied across and within employment sectors.
The current focus of the ECDP is to conduct a survey of ECD working in three areas of
employment: U.S. academic institutions in the GSS, Federally Funded Research and
Development Centers (FFRDCs), and the National Institutes of Health (NIH) Intramural
Research Programs (IRP). Attempts to collect data from ECD working across all sectors are
limited by the lack of a frame source that provide the necessary coverage, privacy, and
confidentiality requirements to adequately measure the ECD population. 2 Thus, NCSES
leverages our experiences and contacts from the GSS which covers these three employment areas
(U.S. academic institutions, FFRDCs, and NIH IRP) as a conduit for frame building. Under
generic clearance OMB control #3145-0174, NCSES conducted the Early Career Doctorates
Study to test the feasibility of 1) obtaining lists of ECD from organizations in these three
employment sectors and 2) using these lists to sample and survey ECD. Completed in February
2013, this methodological study confirmed that institutions in these sectors could build sampling
frames for their ECD and that NCSES could successfully contact and survey these individuals
through the information on these frames. This study also tested several contacting protocols and
showed that sample members were more likely to respond when a high authority figure (HA)
from their institution notified them of their selection and supported their participation.
Under OMB control #3145-0235, NCSES subsequently conducted the Pilot ECDS to refine the
data collection protocols and determine whether these data could be used to develop national
estimates of ECD working in U.S. academic institutions, FFRDCs, and the NIH IRP. Overall,
149 out of 176 sampled institutions (84.7%) provided lists of ECD in stage one of the Pilot
ECDS, and 4,179 of 6,827 sample members were confirmed eligible respondents (66.3%
response rate). 3 Estimated counts of ECD subpopulations based on the Pilot ECDS, such as the
number of postdocs working in each sector, were consistent with other NCSES data suggesting
that the ECDS will be able to produce reliable national estimates for this population. Results
from the Pilot ECDS have been used to fine-tune plans for the full implementation of the ECDS
(hereafter called the 2017 ECDS). The 2017 ECDS will be a two-stage data collection. In the
first stage, the sampled institutions will be contacted and asked to provide a listing of ECD
working at the institution and, in the second stage, ECD will be sampled and surveyed.

2

National Science Foundation, National Center for Science and Engineering Statistics, Phase 2 of the
Postdoc Data Project, Catalog of Lists for Building a Postdoc Sampling Frame (June 7, 2007) and
Postdoc Data Project Phase 2 Summary Report (February 15, 2011).

3

Calculated according to American Association for Public Opinion Research (AAPOR) response rate definition 2.

Early Career Doctorates Survey

2

A.1

Necessity for Information Collection

The authority to collect information for the 2017 ECDS is established under the National Science
Foundation Act of 1950, as amended, Public Law 507 (42 U.S.C. 1862), Section 3(a) (6), which
directs NSF “…to provide a central clearinghouse for the collection, interpretation, and analysis
of data on scientific and engineering resources and to provide a source of information for policy
formation by other agencies of the federal government…”
Furthermore, Executive Order 10521 (March 17, 1954) states: “The Foundation shall continue to
make comprehensive studies and recommendations regarding the Nation’s scientific research
effort and its resources for scientific activities, including facilities and scientific personnel, and
its foreseeable scientific needs, with particular attention to the extent of the federal government’s
activities and the resulting effects upon trained scientific personnel.” More recently, NCSES was
established within the National Science Foundation by Section 505 of the America COMPETES
Reauthorization Act of 2010 and given a broader mandate to collect data related to STEM
education, the science and engineering workforce, and U.S. competitiveness in science,
engineering, technology, and research and development (R&D).
The 2017 ECDS will greatly enhance the center’s current postdoc-related data collections and
provide critical information that will be used to meet NSF’s responsibilities under the Acts and
the Executive Order.
A.2

Uses of Information

The proposed data collection is the first full implementation of the ECDS. The goal is to produce
population estimates for the total ECD working in U.S. academic institutions, FFRDC, and NIH
IRP (the sectors established by the Pilot ECDS), thereby substantially improve the reliability of
estimates for this population by gender, citizenship, origin of doctoral degree, and field of study.
At the conclusion of the 2017 ECDS, NCSES plans to conduct an evaluation of the work
conducted to date in order to determine the future of the Early Career Doctorates Project.
The goal for the evaluation is to identify the role of the ECDS in the context of NCSES’s suite of
science and engineering (S&E) workforce surveys. The ECDS was designed to address the
coverage issues and lack of data on work experiences that has limited the understanding of the
ECD population. However, there is overlap in the target population and survey content for the
ECDS with other NCSES S&E workforce surveys including the Survey of Doctorate Recipients
(SDR), Survey of Earned Doctorates (SED), and National Survey of College Graduates (NSCG).
This evaluation should assess the unique purpose, population coverage, and data utility for each
survey, and provide insight on how the suite of S&E workforce surveys jointly support NCSES’s
role in the collection, interpretation, analysis, and dissemination of objective data on the S&E
enterprise. The findings from this evaluation will guide the future decisions for the ECDS
including the timing, population coverage, survey content, and sample design for any future
surveys of the ECD population.
Pending the outcome of the evaluation, NCSES might consider conducting this survey every
other year survey, or alternating with and complementing the SDR data collection. NCSES will
Early Career Doctorates Survey

3

also consider the need for and value of a longitudinal component with follow-up interviews
occurring approximately 5 and 8 years from doctorate award, and with the longitudinal
respondents entering the SDR sample when they age out of ECDS target population
NSF will publish statistics from the 2017 ECDS in NCSES InfoBriefs and reports, including the
Science and Engineering Indicators and the Women, Minorities and Persons with Disabilities in
Science and Engineering reports. NSF will generate data extracts and other analyses at NIH’s
requests. Restricted-use and public use data files will also be developed. Public-use data will be
available through the NSF website; access to restricted-use data files will be assessed and
granted on a case-by-case basis with licensees signing affidavits of nondisclosure. Likely users
of the data include the National Postdoctoral Association (NPA), the American Association for
the Advancement of Science (AAAS), the Council of Graduate Schools (CGS), the American
Association of Medical Colleges (AAMC), the Association of American Universities (AAU),
and the Organisation for Economic Co-operation and Development (OECD). Possible uses of the
data include descriptive statistics on the employment and productivity of postdocs and other
ECD, the pathways followed by ECD, and their future career plans, including plans to work
outside the United States.
A.3

Consideration of Using Improved Technology

At the institutional level, communications with high authority figures (HAs), list coordinators
(LCs), and communication coordinators (CCs) will be primarily conducted through e-mail, apart
from introductory conversation via telephone. A new secure application accessible through the
ECDS website (www.earlycareerdoctoratessurvey.org) will be created to enable HAs to approve
the survey and identify LCs and CCs online and provide a mechanism for LCs to upload their
ECD lists (see Attachment G). HAs will also be able to access the HA pre-notification tool used
in the methodological study and the Pilot ECDS to assist HAs with sending individualized emails to each sample member. In the Pilot ECDS, this tool was enhanced to allow HAs to modify
the body of the message within the application rather than within each e-mail. LCs will also be
able to log into this secure section of the ECDS website to review instructions for preparing and
uploading their ECD lists.
During the stage two data collection, communications will also be primarily through e-mail, with
seven of ten possible contacts being e-mails. Two of the reminders for non-respondents will be
completed by phone and one will be a hard-copy mail (see Attachment C). Sample members will
have two methods by which to participate in the survey. The primary method will be selfadministration of the Web questionnaire, which can be completed on personal computers and
mobile devices. The secondary method is completing the questionnaire through a telephone
interview, in which a project staff member would access the web questionnaire, read the
questions, and record the responses into the web instrument. In the Pilot ECDS, less than 1
percent of respondents completed some or all of their survey by telephone. Using a web
questionnaire for all responses reduces the costs of creating multiple versions of the
questionnaire, improves data consistency and quality, eliminates costs associated with data entry,
and reduces post completion processing time.

Early Career Doctorates Survey

4

The data collection system used for the 2017 ECDS is designed so that respondents are able to
change response modes should the need arise. For example, respondents may partially complete
the survey on their own and, upon receiving a reminder call, finish the survey with the telephone
interviewer picking up where they broke off. The survey will have a real-time monitoring
system, allowing NSF to monitor response status by mode, system problems, and comments
from respondents. The survey will also perform checks on inconsistent and unlikely answers and
prompt respondents to correct or confirm the data. Respondents will also be informed of their
progress through the instrument and, upon completion, receive an automated e-mail confirming
their completion and thanking them for their participation.
A.4

Efforts to Identify Duplication

Some overlap in terms of target population and content exists with the Survey of Doctorate
Recipients (SDR) and National Survey of College Graduates (NSCG), however the ECDS will
provide more comprehensive coverage of its target population (individuals working in U.S.
academic institutions, FFRDC, and NIH IRP who earned their first doctorate within the last 10
years) than either the SDR or NSCG. The ECDS will also provide more complete data on foreign
degreed doctorate holders.
While both the SDR and NSCG gather basic data about current employment and work
experience, the ECDS collects substantially more information about work experiences, especially
with regard to postdoc experiences, professional activities and achievements, funding, and future
career plans. As a result, the ECDS is the only comprehensive survey of recent U.S. and nonU.S. doctorate recipients working in the United States in these three areas of employment.
Within a sample drawn from the Survey of Earned Doctorates, the SDR is a survey of
individuals through age 75 that earned their doctorate in a research field from a U.S. academic
institution. Due to its frame, the SDR does not include doctorate holders currently employed in
the United States who earned professional doctorates or earned doctorates from institutions
outside the United States.
Less than 2 percent (n=48) of the individuals sampled in the methodological study were also in
the 2013 SDR sample; 4 percent (n=263) of the Pilot ECDS sample members were also sampled
for the 2015 SDR. The increase in overlap between 2013 and 2015 was due to the increased
sample size of the Pilot ECDS and the 2015 SDR. The ECDS and SDR staff will work together
to identify sample members selected for both surveys. Because the ECDS collects substantially
more information than the SDR, the 2017 ECDS will attempt to contact all sample members,
even those identified as having been in the 2017 SDR. Given the expansion of the ECDS sample,
we expect the sample overlap between the 2017 ECDS and 2017 SDR to be approximately 900
cases (4% of the ECDS sample and 0.7% of the SDR sample). The 2017 ECDS sample members
identified as having been in the 2017 SDR will receive a different version of the survey
invitation e-mail/letter that clarifies that this is a different survey and asks these ECD to
participate in both surveys.
Sample overlap with the NSCG is expected to be minimal as only a small fraction of their
sample will be ECD.
Early Career Doctorates Survey

5

A.5

Efforts to Minimize Burden on Small Business

Not applicable. Small businesses will not be included in this round of data collection.
A.6

Consequences of Less Frequent Data Collection

NCSES is seeking clearance for a single data collection cycle only.
The ECDS data are valuable to the analysis presented in a pair of congressionally mandated
reports published by NSF – Science and Engineering Indicators and Women, Minorities, and
Persons with Disabilities in Science and Engineering. Since these reports are published on a
biennial schedule, they rely on the availability of updated data on the science and engineering
resources every two years. Conducting the ECDS on a less frequent basis would prohibit NSF
from meeting its congressional mandate to produce a report that contains an accurate accounting
and comparison, by sex, race, and ethnic group and by discipline, of the participation of women
and men in scientific and engineering positions. The impact of not being able to meet this
congressional mandate is that government, business, industry, and universities would have less
recent data to use as a basis for formulating the nation's science and engineering policies.
A.7

Special Circumstances

Not applicable. This data collection does not require any of the reporting requirements listed.
A.8

Federal Register Announcement and Consultations outside the Agency

Federal Register Notice
The Federal Register notice was published on June 3, 2016 (see Attachment F). NSF received no
public comment in response to the announcement by the closing date of August 5, 2016.
Consultations outside the Agency
NCSES regularly consults with the Department of Education’s National Center for Education
Statistics (NCES) and other federal agencies, such as the NIH and the Department of Education,
professional societies, such as the National Postdoctoral Association, and university staff.
NCSES staff members maintain frequent contact with members of the data-using community as
well as with major academic data providers through attendance at professional society meetings
and consultation with institutional and agency officials.
NCSES has sought advice and guidance of survey methodologists, statisticians, demographers,
researchers, data analysts, and policymakers to examine numerous issues related to the
development of the ECDS.
•

Evaluation of the Pilot ECDS questionnaire for the 2017 ECDS

Early Career Doctorates Survey

6

NCSES convened a Human Resources Expert Panel (HREP) to review methodological
findings related to the Pilot ECDS and review the ECDS questionnaire section by section to
solicit feedback on the survey content for the 2017 ECDS questionnaire.
HREP Panel Members
Catherine Didion, Olin College
Cary Funk, Pew Research Center
Howard Garrison, Federation of American Societies for Experimental Biology
Beverly Karplus Hartline, Montana Tech
David Laurence, Modern Language Association
Cheryl Leggon, Georgia Institute of Technology
Sean McConnell, University of Chicago
Earnestine Psalmonds, National Science Foundation
Yvette Seger, National Postdoctoral Association
Karen Stamm, American Psychological Association
Jodi Yellin, Association of American Medical Colleges
Outside Agency Participant
Jennifer Sutton, National Institutes of Health
•

Evaluation of the NCSES Effort to Measure the Science and Engineering (S&E) Workforce
Population
The National Academies of Science, Engineering, and Medicine’s Committee on National
Statistics (CNSTAT), at the request of NCSES has convened an expert panel to review,
assess, and provide guidance on NCSES’s effort to measure the S&E workforce population
in the United States. Given the evolving data needs of NCSES stakeholders and the budget
climate uncertainty under which NCSES operates, NCSES would like to develop a
framework for measuring the S&E workforce that will enable the flexibility to examine
emerging issues related to this unique population while at the same time allowing for stability
in the estimation of trend data. This framework would provide direction for numerous issues
related to measuring the S&E workforce population including content, data sources, survey
design, survey methodology, data collection, data processing, data integration, data
dissemination, and data promotion.
At the end of its review, the panel will issue a report with findings, recommendations, and
priorities for improving the relevance, accuracy, timeliness, and cost-effectiveness of S&E
workforce data for the next decade and beyond. The information included in this report will
provide the details, direction, and guidance necessary for NCSES to develop a robust and
flexible framework for measuring the S&E workforce over the coming decades.
CNSTAT Panel Chairs and Panel Members
CNSTAT Panel Chairs
Rita Colwell, University of Maryland and Johns Hopkins University
James House, University of Michigan

Early Career Doctorates Survey

7

CNSTAT Panel Members
Jennifer Sue Bond, Council on Competitiveness
Geoff Davis, Verily
Don Dillman, Washington State University
Richard Freeman, Harvard University
Jack Gambino, Statistics Canada
Maresi Nerad, University of Washington
Randy Olsen, Ohio State University
Willie Pearson, Georgia Institute of Technology
Keith Rust, Westat
Nora Cate Schaeffer, University of Wisconsin
James Wagner, University of Michigan
Yu Xie, Princeton University
A.9

Payment or Gifts to Respondents

Not applicable. There will be no payments to the 2017 ECDS respondents.
A.10

Assurance of Confidentiality

NSF and its contractors are fully committed to protecting the confidentiality of all survey
respondents. The ECDS data will be collected under the authority of the National Science Act of
1950, as amended, the Confidential Information Protection and Statistical Efficiency Act
(CIPSEA) of 2002, and the Federal Cybersecurity Enhancement Act of 2015. Institutional
contacts (HAs, CCs and LCs) and prospective respondents are assured in contact materials that
personally identifiable information (PII) and all responses will be kept confidential. Prospective
respondents will also provide informed consent before beginning the survey. Attachment D
(Confidentiality and Security Measures) outlines the other multiple steps during each stage of
data collection that will be taken in order to protect both PII and survey responses.
Cover letters and survey questionnaire sent to each selected sample member advise them that the
information they provide is confidential (see Attachment B – Institutional Contact Materials and
Attachment C – ECD Contact Materials). The same notice of confidentiality (Attachment D) will
be used in the introduction to the computer-assisted telephone interview (CATI) as will be
displayed prior to the start of the survey in the Web instrument. The specific confidentiality
wording that will be used in the CATI and Web instruments is as follows:
Per the Federal Cybersecurity Enhancement Act of 2015, your data are protected from
cybersecurity risks through screening of the federal information systems that transmit your data.
Standard data collection procedures incorporate numerous safeguards for the data and must
conform to a detailed security plan approved by NSF. All data collected and processed will be
stored in a secure internal data network with access available only to authorized users. While
collecting ECDS data, the PII data are separated from the survey response data and are not
included in the analytic data sent to NSF. All project staff at the survey contractor and NSF will
receive annual CIPSEA training to reinforce their legal obligations to protect the privacy and
Early Career Doctorates Survey

8

confidentiality of ECDS and sign data use agreements annually to acknowledge this legal
obligation.
The contractor takes special steps to ensure that data collected via the Web questionnaire are
secure. First, access to the Web instrument is only allowed with a valid Personal Identification
Number (PIN) and password correctly entered in combination. Second, data are transmitted by
the Secure Sockets Layer (SSL) protocol that uses powerful encryption during transmission
through the Internet. If a respondent keeps a Web survey open without any activity, the Web
server closes it after a short period of inactivity, thus preserving the data up to the break-off point
and securely closing the connection. The database architecture underlying the Web and CATI
survey systems is designed such that authentication information and response data are
maintained in separate tables with randomly generated identification numbers assigned for data
collection purposes, and new, random identification numbers generated for the analytic files.
This strategy provides an extra layer of security to protect respondents’ data. Both development
and production servers are backed up nightly, as required by the contractor’s disaster recovery
plan.
All published findings, analyses, and restricted databases are in formats which preclude
individual identification and measures are taken so that the identity of individuals or
organizations is not disclosed.
A.11

Justification for Sensitive Questions

No questions of a sensitive nature are asked in this data collection.
A.12

Estimate of Respondent Burden

There are four types of respondents to the ECDS. At the first stage of sampling, we will select
approximately 350 institutional sampling units. For each of these sampling units, a high authority
figure (HA) will authorize the institution’s participation in the survey and designate a list
coordinator (LC) and a communication coordinator (CC). The LC will provide a list of all
individuals working at their institution who earned their first doctorate or doctorate-equivalent
degree within the past 10 years, including postdocs, nonfaculty researchers, and faculty
members. The CC will be the point of contact for interaction with the HA’s office, and will assist
the HA with developing a letter of support.
In the methodological study and Pilot ECDS, these stage one tasks were typically completed in
less than an hour with administrative personnel in the HA’s office playing an important role in
completing these tasks. As a result, NCSES is adding a formal CC role and is estimating that
these tasks will take an average of 80 minutes in the 2017 ECDS – 20 minutes for the HA and up
to 60 minutes for the CC. Assuming that deciding not to participate takes a similar amount of
time as deciding to participate, NCSES estimates the total HA burden for stage one to be 117
hours and the total CC burden for stage one to be 350 hours.
Based on the methodological study and Pilot ECDS, it is expected that 15% of the institutions
and 21% of individual sample members will not participate (2% will be ineligible and 19% will
Early Career Doctorates Survey

9

not respond). Thus, in order to yield 18,000 eligible ECD respondents, NCSES will need to
sample 22,855 individuals from the approximately 300 participating sampling units.
The primary responsibility of the LC is to prepare a list of ECD employed at the institution. In
the Pilot ECDS, LCs required an average of 8 hours to fulfill their duties. Assuming 300
institutions will participate, we estimate the total burden for LCs to be 2,400 hours.
In stage two, the HAs and CCs will send a pre-notification e-mail to sample members within
their institution. In the methodological study and Pilot ECDS, most HAs were able to complete
this task in less than 30 minutes. To maximize response rates, NCSES will ask the HA to identify
a CC who can send e-mails for the HA or to designate another recognizable authority (e.g., Vice
Chancellor for Research) to send the pre-notification e-mails. In the Pilot ECDS, 23% of
participating institutions chose to have NSF send the pre-notification or did not send the prenotification e-mail within the specified time period, so for purposes of estimating burden for the
2017 ECDS, NCSES assumes 80% of participating institutions will send the stage two prenotification. At 10 minutes per HA and 30 minutes per CC (more sample members per institution
than in the Pilot ECDS), NCSES estimates a total burden of 40 hours for HAs and 120 hours for
CCs to send the pre-notification e-mail.
Based on survey timing data from the methodological study and planned changes to the
questionnaire, NCSES estimates that ECD will require approximately 32 minutes on average to
complete the questionnaire for eligible ECD and less than 2 minutes for ineligible respondents.
Assuming 18,000 eligible ECD respond and 410 ineligible respondents, NCSES estimates the
total respondent burden will be approximately 9,614 hours.
Table 1. Estimated Burden by Stage and Respondent Type: 2017 ECDS
Number of
Respondents

Minutes Per
Respondent

Estimated
Total Hours

Stage 1: Frame Creation
High Authorities (HA)
Communication Coordinators (CC)
List Coordinators (LC)
Subtotal

350
350
300

20
60
480

117
350
2,400
2,867

Stage 2: Individual Survey
High Authorities (HA)
Communication Coordinators (CC)
Early Career Doctorates (ECD)
Ineligible Respondents
Subtotal

240
240
18,000
410

10
30
32
2

40
120
9,600
14
9,774

Respondent Type

Total

12,641

Taking into account all respondent types across both stages of data collection, NCSES estimates
the total respondent burden to be just over 12,600 hours. Table 1 shows detailed estimates of
burden across respondent types.

Early Career Doctorates Survey

10

A.13

Cost Burden to Respondents

Not applicable. This survey does not require respondents to purchase equipment, software, or
services beyond those normally used in universities as part of customary and usual business.
A.14

Cost Burden to the Federal Government

The total estimated cost to the Government for the 2017 ECDS is $6.13 million for survey costs,
associated methodological work, and for NSF staff costs to provide oversight and coordination
with other surveys. The cost estimated for the survey cycle is $6.08 million, which is based on
sample size; length of questionnaire; CATI and Web data collection technology; administrative
overhead, design, printing mail, and telephone data collection costs; data keying, editing, and
processing; data quality control; imputation for missing item responses; weighting and
estimating sampling error; file preparation and delivery; methodological work, preparation of
documentation and final reports; and survey evaluation. NSF staff costs are estimated at
$675,000 ($150,000 annual salary of 1.5 FTE for 3 years).
A.15

Reason for Change in Burden

NCSES expects that per respondent burden for the 2017 ECDS to be slightly less than that
observed in the Pilot ECDS. The NCSES Human Resources Expert Panel (HREP) advised that
the questionnaire was too long, so NCSES shortened the survey by 20% from 41 to 32 minutes.
However, the total burden for the 2017 ECDS is expected to increase from 4,803 hours in the
Pilot ECDS to 12,533 hours primarily due to the increase in sample size (see table 1 above).
A.16

Schedule for Information Collection and Publication

The 2017 ECDS will be a two-stage data collection. In the first stage, sampled institutions are
contacted to begin the frame building process. Because ECD listings from sampled institutions
will be received at various times, the second stage of the data collection will be a rolling process
beginning approximately one month after the receipt of the ECD listing from the institution and
ending two to three months after the survey invitation. The InfoBrief, detailed statistical tables,
and public use data files will be published and made available on the NCSES website. The time
schedule for the ECDS data collection and publication is currently estimated as follows:
Table 2. ECDS Data Collection and Publication Schedule
Activities
Stage 1 (Institutional) Data Collection
Stage 2 (Individual ECD) Data Collection
Final Edited/Weighted/Imputed data file
Public Use Data File and Documentation
Survey Evaluation Report

Early Career Doctorates Survey

Dates
October 2017 – January 2017
October 2017 – May 2018
July 2018
December 2018
January 2019

11

A.17

Display of OMB Expiration Date

The OMB expiration date will appear on all ECDS webpages, including the welcome page, login
page, confidentiality page, and survey questions pages.
A.18

Exceptions to the Certification Statement

Not applicable. There are no exceptions.

Early Career Doctorates Survey

12