Download:
pdf |
pdfDOE Office of Science
Technical Manpower Survey Sampling Plan
Supporting Statement Part B: Sampling Plan
This sampling plan is being submitted to comply with the OMB PRA. The following
describes how the sampling plan provides each of the following items required under Part
B: Collections of Information Employing Statistical Methods.
1. Description of the potential respondent universe and any sampling or other
respondents selection methods.
Section 4: Sampling Frame and Population (pg. 4) describes the sampling frame.
The sampling frame is based on a DOE database of grants, and the selection of
respondents is through a stratified random sampling procedure, as described in
Section 5: Method of Sampling Selection (pg. 10).
2. Procedures for the collection of information.
Information will be collected by means of an online survey, as described in
Section 6: Design of an Online Survey (pg. 11).
3. Methods to maximize response rates:
Section 7: Implementation Feasibility (pg. 12) describes the follow-up procedures
intended to increase response rates. These include the sending of an introductory
letter, and follow up via email and phone calls.
4. Tests of procedures or methods
The online survey was pretested to three Principal Investigators (see pg. 12) to
ensure the clarity of the survey.
5. Consultant’s Contact information:
This sampling plan was developed by Dr. Albert Lee, who will also oversee the
analysis of the sample.
Dr. Albert Lee
Summit Consulting, LLC
1310 12th Street NW, Suite 1
Washington DC 20005
Phone: 202-588-1092
Additional expertise was provided by Dr. Frauke Kreuter of the Joint Program in
Survey Research at the University of Maryland at College Park, and Dr. Robert
Gould of the Department of Statistics at the University of California at Los
Angeles.
Prepared by Summit Consulting, LLC
1
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
1. Introduction
This is a sampling plan for the Department of Energy (DOE) Technical Manpower
Survey for the 2005 academic year.1 This plan provides:
1. An overview of sampling theory in the context of the survey’s objectives,
2. A description of the sampling methodology,
3. Method of implementation, and
4. A random sample for survey.2
This sampling plan consists of six subsections. These subsections are:
1.
2.
3.
4.
5.
6.
Background and objectives,
Stratified random sampling,
Sampling frame and population,
Method of sampling selection,
Online Survey design, and
Feasibility implementation.
2. Background and Objectives
DOE Office of Science has tasked Summit Consulting, LLC, (Summit Consulting) to
design and implement an online survey to estimate the total technical manpower that was
supported by DOE basic science grants in the 2005 academic year. The total technical
manpower refers to the full time equivalent (FTE) and head count of undergraduate and
graduate students, post-doctoral fellows (post-docs), principal investigators (PIs), nontechnical personnel and administrative staff employed by DOE grants. In addition to
estimating these quantities, the online survey is also to estimate the number of Ph.D.
graduates in June 2005 that have received DOE grant support in the 2005 academic year.
Precise estimates of these quantities are consistent with the mission of DOE Office of
Science. Department of Energy Organization Act (Public Law 95-91, as amended) Sec.
209 defines the duty and the responsibilities of the Director of Office of Science to
include:
1. Advising the Secretary with respect to the physical research program transferred
to the Department from the Energy Research and Development Administration;
2. Monitoring the Department’s energy research and development programs in order
to advise the Secretary with respect to any undesirable duplication or gaps in such
programs;
3. Advising the Secretary with respect to the well-being and management of the
multipurpose laboratories under the jurisdiction of the Department, excluding
laboratories that constitute part of the nuclear weapons complex;
1
2
For the purpose of this sampling plan, the 2005 academic year is between July 1, 2004 and June 30, 2005.
See Appendix A.
Prepared by Summit Consulting, LLC
2
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
4. Advising the Secretary with respect to education and training activities required
for effective short- and long-term basic and applied research activities of the
Department;
5. Advising the Secretary with respect to grants and other forms of financial
assistance required for effective short- and long-term basic and applied research
activities of the Department; and
6. Carrying out such additional duties assigned to the Office by the Secretary
relating to basic and applied research, including but not limited to supervision or
support of research activities carried out by any of the Assistant Secretaries
designated by section 203 of this Act, as the Secretary considers advantageous.
[42 U.S.C. 7139]
Items 4, 5, and 6 indicate that the Office of Science has the responsibility to manage
grants and financial assistance concerning education and training activities in basic
sciences. Part of the evaluation of the performance of these grants and other forms of
financial assistance is the number of technical manpower supported by them. Estimates
of the number of graduate students, post-doctoral fellows and doctoral graduates
supported by these grants have been a perennial feature in the department’s annual
budget requests to Congress.3 Thus, a precise quantification of technical manpower
supported by the Department’s grants is consistent with the management objectives of the
Office of Science.
Currently, no statistically valid estimates of technical manpower supported by the
department’s grants exist. The total number of technical manpower supported by the
department’s grants is based on forecasts by the grant recipients (i.e., PIs) at the time
when the grants were initially funded. This estimate is unreliable because it is based on
the best guess of the PIs at the time of funding. While the PI’s initial estimate could be
accurate at the time of the grant’s funding, the reliability of the initial estimate decreases
as the grant matures. Further, the forecasts by the PIs are subjective. Therefore, it is not
possible to quantify the inaccuracies with any confidence.
Moreover, the unit of accounting is inconsistent across grant proposals. While some PIs
reported their estimates of technical manpower supported by the department’s grants in
full time equivalent (FTE), other PIs reported their estimates of technical manpower by
head count without taking into consideration share of support, which are frequently less
than full-time. This inconsistency of unit of accounting could cause double counting and
other inaccuracies in the department’s annual budget requests to Congress. Thus, the
current estimate of technical manpower supported by the department’s grants is only a
best guess. This estimate lacks the rigor of an estimate produced by a survey based on an
appropriately designed sampling plan.
Therefore, the purpose of this survey is to
1. Produce a statistical valid estimate of technical manpower support by the
department’s grants in the 2005 academic year, and
2. Quantify the uncertainty associate with this estimate due to sampling error.
3
For example, see page 23 of Basis Energy Science’s FY 2006 Budget Request to Congress.
Prepared by Summit Consulting, LLC
3
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
3. Stratified Random Sampling
In principle we could achieve these objectives by questioning all of the principal
investigators of all active awards. However, this is laborious and expensive. Instead we
survey a random sample of principal investigators and use their responses to estimate the
desire quantities for the entire population. A simple random sample, in which every
same-size subset of the population is equally likely to be included in the sample,
produces estimates with quantifiable precisions.
Simple random samples are themselves too expensive because they require a relatively
large sample size to achieve the desired level of precision. For this reason, modifications
of this basic sampling scheme are implemented to produce equally accurate estimates
with a smaller sample. One method to achieve efficiency gains is to partition the
population into strata, and to select a simple random sample from each of these strata. A
stratified random sample produces a more precise estimate about the population because
it is frequently possible to divide a heterogeneous population into strata that are internally
homogenous.4 Thus, the goal of stratification is to partition the population in such a way
that the units within a stratum are as similar as possible, even though one stratum may
differ markedly from another.
The implementation of a stratified random sample predicates on some known structures
within the population. Ideally, a successful stratification would require prior knowledge
about some auxiliary variables that are correlated with the quantities of interests. In the
absence of perfect prior knowledge, one could still rely on features of data, to partition
the population into strata that are likely to correlate with the quantities of interests.
Thus, in keeping with best practices, we propose to implement a stratified random
sampling in two phases. The initial phase involves the selection of a feasibility sample,
in which the stratification and sample allocation are based on the judgments guided by
data analyses. Based on the results of this feasibility sample, the second phase
implementation will either re-stratified or increase the sample size to achieve the
prescribed precision requirement.
4. Sampling Frame and Population
A sampling frame is a list of items in the population from which we select a sample. Our
sampling frame is based on the IMFC database, retrieved in September 2004. This
database contains an exhaustive list of all active awards granted by the DOE. This
database also contains auxiliary information, such as the program, funding amount and
reported number of graduate students etc.
4
Although, it is not universally true that any stratified random sample will yield a more precise estimate. If
appropriately applied, stratification nearly always results in a more precise estimate than by a comparable
simple random sample. See “Sampling Techniques,” by William Cochran.
Prepared by Summit Consulting, LLC
4
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Specifically, the retrieved database contains funding information of 2,716 distinct
awards.5 These 2,716 awards amount to about $292,109,312 in total funding. Each of
these distinct awards forms a sampling unit within the sampling frame.6
These 2,716 awards are classified into six program types, which are:7
1. Advanced Science Computing;
2. Basic Energy;
3. Biological and Environmental Sciences;
4. Fusion Energy Sciences;
5. High Energy Physics; and
6. Lab Ops and ES&H.
Table 1: Distribution of Awards and Funding by Program
Program
Advanced Sci Computing
Basic Energy
Biological and Environmental Sciences
Fusion Energy Sciences
High Energy Physics
Lab Ops and ES&H
Total
Funding Amount
$1,262,155
$99,323,251
$45,654,905
$55,022,963
$90,846,038
$292,109,312
Number of Awards
217
1,229
821
256
190
3
2,716
Table 1 shows that the distribution of awards and funding is uneven across these six
programs. For instance, in terms of number of awards, the largest program is Basic
Energy, which represents about 46 percent of the awards, and about 34 percent of the
total funding. The second largest program is Biological and Environmental Sciences,
which represents about 23 percent of the awards, and 16 percent of the total funding.
Figure 1 shows that the top two program types dominate the number of awards.
5
Awards are identified by “awardno” in the database.
Some of these awards contain subcontracts and include funding for off-site research projects, which
require separate surveys. Since this sampling plan envisions a 100 percent sample from these subunits,
their presence does not alter the substance of the sampling methodology, i.e., no second stage sampling is
required.
7
Excluded from the sampling frame are 199 awards in Nuclear Physics, which amount to about
$62,500,964.
6
Prepared by Summit Consulting, LLC
5
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Figure 1: Distribution of Awards by Program
The distribution of funding by program is likewise uneven. Figure 2 depicts the
proportions of funding by program. It shows that Basic Energy and High Energy Physics
account for about 65 percent of the total funding. Biological and Environmental Sciences
and Fusion Energy Sciences divide the rest of the funding in roughly equal proportion.
Advanced Scientific Computing represents less than one percent of the total funding.
Lab Ops and ES&H did not receive funding as of the extraction of the database.
Prepared by Summit Consulting, LLC
6
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Figure 2: Distribution of Funding by Program
We also note that 1,839, or about 68 percent of the total awards, appear to have received
no funding in 2005. These 1,839 awards probably represent active grants that are in their
second or third year of funding. It is likely that these multi-year awards had reported the
entire amount of their funding in the academic year in which they were first funded.
Therefore, these otherwise active awards reported zero funding in years subsequent to
initial funding. Moreover, these 1,839 awards did not report the number of graduate
students funded by the grants. That is, beyond the initial estimate of the number of
graduate students supported by the grants, the database does not contain any update in the
subsequent years after initial funding.
There are 877 awards, or 32 percent of the total awards, that reported non-zero funding in
the database. These 877 awards are all in their first year of funding (in 2004), and have
reported estimates of the number of graduate students supported by the grants. Table 2
shows that awards that were funded in 2004 reported non-zero funding amounts and
number of graduate students. Awards that were funded prior to 2004 reported neither the
funding amount nor the number of graduate students.
Table 2: Reported Amount of Funding, Number of Awards and Report number of Graduate
Students by Program
Program
Advanced Sci Computing
Basic Energy
Funding Amount
Reported zero funding
0
0
Prepared by Summit Consulting, LLC
7
Number of Awards
214
714
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Program
Funding Amount
Biological and Environmental Sciences
0
Fusion Energy Sciences
0
High Energy Physics
0
Lab Ops and ES&H
0
Total
0
Reported non-zero funding
Advanced Sci Computing
$1,262,155
Basic Energy
$99,323,251
Biological and Environmental Sciences
$45,654,905
Fusion Energy Sciences
$55,022,963
High Energy Physics
$90,846,038
Lab Ops and ES&H
Total
$292,109,312
Number of Awards
668
162
78
3
1,839
3
515
153
94
112
877
Among the awards that were funded in 2004, a noticeable relationship emerges between
the reported number of graduate students and program. Figure 3 is a bar chart of the
mean of reported number of graduate students by program. It shows that different
programs, on average, supported different number of graduate students. For example, on
average, each award in High Energy Physics supports about six graduate students. Each
award in Advance Scientific Computing supports about five graduate students. On
average, an award in Fusion Energy Sciences, Lab Ops and ES&H, and Biological and
Environmental Sciences support about only two graduate students.
Prepared by Summit Consulting, LLC
8
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Figure 3: Mean number of Reported Graduate Student by Program
5.7
5.0
2.3
2.0
1.8
0
2
4
Mean of Reported Number of Graduate Student
6
Advanced Sci Computing
Basic Energy
Biological and Environmental Sciences
Fusion Energy Sciences
High Energy Physics
Lab Ops and ES&H
Table 3 presents summary statistics of the reported number of graduate students for 877
awards that have received funding in 2004. In addition to variations by the mean, the
standard deviations vary noticeably by program as well.
Table 3: Summary Statistics for the Reported Number of Graduate Students by Program
Program
Mean
Advanced Sci Computing
Basic Energy
Biological and Environmental Sciences
Fusion Energy Sciences
High Energy Physics
Graduate Students
Std. Dev.
5
3
2
4.8
1.8
1.4
2.3
2.6
5.7
5.3
Lab Ops and ES&H
Total
2.5
4.4
Count
3
515
153
94
112
0
877
The above analyses provide three observations that have an impact on stratification:
1. The database does not provide information on the funding amount and the
reported number of graduate students for awards that were funded in years prior
to 2004.
2. The number of awards and the size of the funding vary significantly by program.
3. The distributions of the reported number of graduate students differ noticeably by
program.
Prepared by Summit Consulting, LLC
9
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
5. Method of Sampling Selection
Because the reported number of graduate students in the database is likely to correlate
with the actual quantities of technical manpower employed by the active awards, the
results of the above analyses give support to stratify the population by program to achieve
an efficient estimate. In additional to stratifying the awards by program, we further
stratified the population by a binary indicator that indicates whether an award reported
number of graduate students. As discussed earlier, the database captures both the
reported number of graduate students and the funding amount only for awards in their
initial year of funding. And because about 2/3 of the active awards are in their second
and third year of funding, it is important to ensure these awards’ representation in the
sample. We therefore partitioned the population into 12 strata, two strata per program.
Within each program, awards are differentiated by whether an award reported number of
graduate students.
For the initial feasibility sample, we choose a sample size of 366 from these 12 strata.
Table 4 summarizes the distribution of the sampled items by program and by whether an
award reported graduate students.
Table 4: Distribution of Sampled Items by Program and Whether an Award Reported Graduate
Students
Program
Sample Size
Reported Zero Students
Advanced Sci Computing
12
Basic Energy
38
Biological and Environmental Sciences
35
Fusion Energy Sciences
9
High Energy Physics
5
Lab Ops and ES&H
1
Total
100
Reported Non-Zero Students
Advanced Sci Computing
3
Basic Energy
154
Biological and Environmental Sciences
46
Fusion Energy Sciences
29
High Energy Physics
34
Lab Ops and ES&H
Total
266
Population Size
214
714
668
162
78
3
1,839
3
515
153
94
112
877
To select a sample of 366 items from the population, we first sort the database by
program and award number, which is unique in the database. We than assigned a random
Prepared by Summit Consulting, LLC
10
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
number based on a uniform distribution.8 This number randomized the items in sampling
frame. Based on the sample allocation scheme in Table 4, we selected a sample of 366
items.
6. Design of an Online Survey
The surveying literature has identified several strengths and weakness about online and
telephone surveys. This section summarizes our research results and the specifics of the
DOE Technical Manpower survey that mitigate and amplify these limitations.
Specifically, the survey literature has indicated that online surveys are likely to achieve a
lower response rate (incomplete and non-responses) compared to telephone surveys
because online surveys
1. Are not administrated by interviewers, human assistance or probing for difficult
questions is not possible.
2. Have to be administrated via Internet portals, and therefore exclude the portion of
population without Internet access.
Given the target population of this survey, many of the aforementioned limitations are
mitigated. The target respondents of this survey are scientists at universities who have
received DOE grants. These scientists are likely to have high-speed Internet access and,
presumably, knowledgeable in using Internet.
Also, human assistance is not without its problems. On one hand, interviewer assistance
can solicit otherwise difficult to obtain information. On the other hand, it can also
introduce unintended interviewer influences, and thus skewing the response. Online
surveys can utilize pop-up windows, links and other visual aids to assist the respondents.
We can also provide a help desk during usual office hours to assist the respondents.
The benefits associated with online surveying include:
1. Online surveys are more cost-effective:
a. Beyond the fixed start up costs, the marginal costs for an additional survey
is negligible. These savings apply to follow-up and next year’s studies.
2. Online surveys can provide and retrieve more information than telephone survey
because:
a. Online surveys can allow pop-up windows and links for definition and
example;
b. Online survey can provide visual clues;
c. Online survey can detect pauses, the number of times the respondent
return to the survey after interruptions, and the duration necessary to
complete the survey.
3. Unlike telephone surveys, online surveys do not require pre-designed
appointments.
4. Unlike telephone surveys, online surveys can be completed in intervals.
8
We implemented the randomization using STATA, a statistical software program. To ensure
reproducibility, we set a seed at 9162004, the date when the sample was selected.
Prepared by Summit Consulting, LLC
11
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
5. Online survey can send out e-mail reminder to non-respondents.
6. Automated skip-patterns and response validation to minimize response time and
response consistency.
We have implemented a number of steps to ensure the Technical Manpower survey is as
clear as possible.
1. The DOE Technical Manpower survey is designed by experts in survey design.9
The language and the organization of the survey are intended to achieve clarity
and cohesion.
2. We have pre-tested the survey through telephone interviews with three actual
principal investigators. The results of the pre-test telephone interviews have
further clarified ambiguities and assisted in anticipating contexts that could lead
to misinterpretation.
3. The online survey allows hyperlinks and pop-up windows that provide precision
definitions of terms to further eliminate ambiguity.
4. A helpdesk is established during the fielding of the survey to answer any
remaining questions.
Because a copy of the survey questionnaire is attached as an appendix, suffice it to note
that the proposed questionnaire enquires about the numbers of graduate students,
undergraduates, (co-) principal investigators, (non-) technical personnel etc. In addition
to the quantities of technical manpower, the proposed questionnaire also enquires about
the full- or part-time status of these categories of technical manpower. In case the
selected grants also fund off-site research and subcontracts, the proposed questionnaire
differentiates off-site research and subcontracts, isolating the survey responses to
research activities that respondents exercise immediate control over.
We also note that the proposed online survey questionnaire is imbedded with skip-pattern
logic to route the respondents to relevant questions depending on their responses to prior
questions. Also, the survey questionnaire imposes constraints on response format to
further mitigate response errors. A copy of a question flow chart is collected in Appendix
B: Questionnaire Flowchart.
7. Implementation Feasibility
The implementation of a feasibility sample begins with a letter to send to the selected
principal investigators. This letter will be issued by the DOE Office of Science, which
states the purpose of the survey and ensures respondents’ confidentiality. The online
survey literature indicates that pre-notification by a different medium could greatly
enhance the likelihood of success of the survey.
Following the pre-notification letter is an e-mail that invites the selected principal
investigators to log onto the online survey. This e-mail invitation provides a login
9
Specifically, Professor Frauke Kreuter of the Joint Program of Survey Research of University of
Maryland at College Park, and Professor Robert Gould of the Department of Statistics at University of
California at Los Angeles are the principal architects of the DOE manpower survey.
Prepared by Summit Consulting, LLC
12
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
identification number and a password. This login identification number and password
permit the respondents to save and edit their responses prior to final submission. Once a
response is successfully submitted, an acknowledgment and a copy of their responses will
be e-mailed. We expect the fielding periods for the feasibility phase to last a month after
the mailing of the invitation e-mail.
Follow-up reminders will be e-mailed to those principal investigators that fail to log on to
the online survey after one week of the receipt of the e-mail invitation. Follow-up phone
calls will be placed to those principal investigators that fail to log on to the online survey
after two weeks of the receipt of the initial e-mail invitation. At the conclusion of the
field period, e-mail reminders will also be sent to those respondents that have logged in
the online survey but have failed to complete and submit the survey. To increase the
response rate, we will host a help desk during regular office hours to answer questions
and provide clarifications.
Prepared by Summit Consulting, LLC
13
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Appendix A: Feasibility Sample
Sample No
Unique ID
1
8
Advanced Sci Computing
Program
0
25047
2
12
Advanced Sci Computing
0
25164
3
31
Advanced Sci Computing
0
25452
4
64
Advanced Sci Computing
0
25488
5
66
Advanced Sci Computing
0
25490
6
73
Advanced Sci Computing
0
25497
7
89
Advanced Sci Computing
0
25515
8
99
Advanced Sci Computing
0
25533
9
155
Advanced Sci Computing
0
25592
10
179
Advanced Sci Computing
0
25621
11
189
Advanced Sci Computing
0
25631
12
192
Advanced Sci Computing
0
25634
13
215
Advanced Sci Computing
1
25053
14
216
Advanced Sci Computing
1
25341
15
217
Advanced Sci Computing
1
25517
16
260
Basic Energy
0
14187
17
281
Basic Energy
0
14485
18
284
Basic Energy
0
14534
19
309
Basic Energy
0
14771
20
318
Basic Energy
0
14853
21
328
Basic Energy
0
14907
22
352
Basic Energy
0
15097
23
375
Basic Energy
0
15161
24
377
Basic Energy
0
15169
25
384
Basic Energy
0
15181
26
396
Basic Energy
0
15204
27
410
Basic Energy
0
15227
28
431
Basic Energy
0
15267
29
433
Basic Energy
0
15270
30
436
Basic Energy
0
15282
31
445
Basic Energy
0
15313
32
463
Basic Energy
0
15351
33
466
Basic Energy
0
15354
34
476
Basic Energy
0
15375
35
493
Basic Energy
0
15411
36
518
Basic Energy
0
15443
37
554
Basic Energy
0
15511
38
562
Basic Energy
0
15529
39
581
Basic Energy
0
15556
40
617
Basic Energy
0
15598
41
647
Basic Energy
0
20015
42
650
Basic Energy
0
20033
43
705
Basic Energy
0
20312
44
730
Basic Energy
0
45438
Prepared by Summit Consulting, LLC
Reported Student (=1)
14
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
45
739
Basic Energy
Program
0
45579
46
794
Basic Energy
0
45891
47
825
Basic Energy
0
45942
48
844
Basic Energy
0
45996
49
849
Basic Energy
0
46011
50
851
Basic Energy
0
46013
51
876
Basic Energy
0
46078
52
892
Basic Energy
0
46096
53
924
Basic Energy
0
46173
54
935
Basic Energy
1
13251
55
936
Basic Energy
1
13289
56
947
Basic Energy
1
13500
57
951
Basic Energy
1
13537
58
954
Basic Energy
1
13579
59
956
Basic Energy
1
13622
60
960
Basic Energy
1
13690
61
961
Basic Energy
1
13714
62
962
Basic Energy
1
13742
63
966
Basic Energy
1
13850
64
970
Basic Energy
1
13880
65
973
Basic Energy
1
13941
66
976
Basic Energy
1
14020
67
978
Basic Energy
1
14048
68
981
Basic Energy
1
14125
69
997
Basic Energy
1
14303
70
998
Basic Energy
1
14305
71
1000
Basic Energy
1
14320
72
1001
Basic Energy
1
14327
73
1003
Basic Energy
1
14333
74
1007
Basic Energy
1
14359
75
1008
Basic Energy
1
14363
76
1014
Basic Energy
1
14467
77
1017
Basic Energy
1
14500
78
1022
Basic Energy
1
14546
79
1023
Basic Energy
1
14549
80
1025
Basic Energy
1
14568
81
1026
Basic Energy
1
14589
82
1028
Basic Energy
1
14600
83
1030
Basic Energy
1
14619
84
1033
Basic Energy
1
14626
85
1035
Basic Energy
1
14641
86
1046
Basic Energy
1
14678
87
1047
Basic Energy
1
14680
88
1049
Basic Energy
1
14684
89
1050
Basic Energy
1
14685
90
1059
Basic Energy
1
14814
91
1060
Basic Energy
1
14847
92
1063
Basic Energy
1
14879
Prepared by Summit Consulting, LLC
Reported Student (=1)
15
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
93
1066
Basic Energy
Program
Reported Student (=1)
1
Award No
14890
94
1070
Basic Energy
1
14908
95
1075
Basic Energy
1
14943
96
1076
Basic Energy
1
14949
97
1080
Basic Energy
1
14982
98
1082
Basic Energy
1
14988
99
1087
Basic Energy
1
14996
100
1088
Basic Energy
1
14998
101
1089
Basic Energy
1
14999
102
1096
Basic Energy
1
15035
103
1106
Basic Energy
1
15121
104
1107
Basic Energy
1
15129
105
1113
Basic Energy
1
15176
106
1120
Basic Energy
1
15244
107
1132
Basic Energy
1
15287
108
1133
Basic Energy
1
15296
109
1136
Basic Energy
1
15317
110
1137
Basic Energy
1
15318
111
1140
Basic Energy
1
15328
112
1141
Basic Energy
1
15331
113
1145
Basic Energy
1
15364
114
1148
Basic Energy
1
15372
115
1154
Basic Energy
1
15387
116
1155
Basic Energy
1
15389
117
1156
Basic Energy
1
15390
118
1159
Basic Energy
1
15404
119
1165
Basic Energy
1
15422
120
1167
Basic Energy
1
15424
121
1172
Basic Energy
1
15457
122
1174
Basic Energy
1
15460
123
1176
Basic Energy
1
15463
124
1177
Basic Energy
1
15464
125
1179
Basic Energy
1
15466
126
1182
Basic Energy
1
15472
127
1184
Basic Energy
1
15474
128
1187
Basic Energy
1
15481
129
1188
Basic Energy
1
15490
130
1189
Basic Energy
1
15496
131
1191
Basic Energy
1
15503
132
1192
Basic Energy
1
15505
133
1201
Basic Energy
1
15520
134
1205
Basic Energy
1
15530
135
1214
Basic Energy
1
15568
136
1216
Basic Energy
1
20021
137
1224
Basic Energy
1
20175
138
1226
Basic Energy
1
20206
139
1228
Basic Energy
1
20221
140
1235
Basic Energy
1
45118
Prepared by Summit Consulting, LLC
16
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
141
1236
Basic Energy
Program
1
45125
142
1237
Basic Energy
1
45170
143
1239
Basic Energy
1
45237
144
1244
Basic Energy
1
45331
145
1245
Basic Energy
1
45332
146
1247
Basic Energy
1
45405
147
1251
Basic Energy
1
45435
148
1253
Basic Energy
1
45471
149
1259
Basic Energy
1
45504
150
1260
Basic Energy
1
45538
151
1261
Basic Energy
1
45543
152
1262
Basic Energy
1
45557
153
1264
Basic Energy
1
45576
154
1272
Basic Energy
1
45628
155
1273
Basic Energy
1
45632
156
1280
Basic Energy
1
45685
157
1281
Basic Energy
1
45686
158
1284
Basic Energy
1
45702
159
1292
Basic Energy
1
45770
160
1296
Basic Energy
1
45778
161
1304
Basic Energy
1
45805
162
1305
Basic Energy
1
45810
163
1307
Basic Energy
1
45818
164
1310
Basic Energy
1
45827
165
1316
Basic Energy
1
45869
166
1317
Basic Energy
1
45871
167
1318
Basic Energy
1
45872
168
1320
Basic Energy
1
45885
169
1322
Basic Energy
1
45906
170
1323
Basic Energy
1
45907
171
1327
Basic Energy
1
45913
172
1329
Basic Energy
1
45923
173
1330
Basic Energy
1
45927
174
1334
Basic Energy
1
45933
175
1336
Basic Energy
1
45940
176
1341
Basic Energy
1
45957
177
1342
Basic Energy
1
45958
178
1343
Basic Energy
1
45959
179
1345
Basic Energy
1
45961
180
1346
Basic Energy
1
45963
181
1349
Basic Energy
1
45967
182
1352
Basic Energy
1
45974
183
1354
Basic Energy
1
45976
184
1368
Basic Energy
1
46010
185
1369
Basic Energy
1
46014
186
1370
Basic Energy
1
46015
187
1375
Basic Energy
1
46023
188
1377
Basic Energy
1
46027
Prepared by Summit Consulting, LLC
Reported Student (=1)
17
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
189
1378
Basic Energy
Program
1
46028
190
1380
Basic Energy
1
46031
191
1381
Basic Energy
1
46033
192
1385
Basic Energy
1
46038
193
1387
Basic Energy
1
46040
194
1389
Basic Energy
1
46042
195
1391
Basic Energy
1
46044
196
1394
Basic Energy
1
46053
197
1400
Basic Energy
1
46066
198
1401
Basic Energy
1
46072
199
1402
Basic Energy
1
46076
200
1404
Basic Energy
1
46086
201
1410
Basic Energy
1
46103
202
1415
Basic Energy
1
46109
203
1419
Basic Energy
1
46114
204
1423
Basic Energy
1
46119
205
1425
Basic Energy
1
46121
206
1439
Basic Energy
1
46142
207
1446
Basic Energy
1
46155
208
1447
Biological and Environmental Sciences
0
10926
209
1458
Biological and Environmental Sciences
0
11114
210
1461
Biological and Environmental Sciences
0
11118
211
1465
Biological and Environmental Sciences
0
11127
212
1513
Biological and Environmental Sciences
0
62028
213
1523
Biological and Environmental Sciences
0
62210
214
1542
Biological and Environmental Sciences
0
62472
215
1557
Biological and Environmental Sciences
0
62687
216
1609
Biological and Environmental Sciences
0
63025
217
1638
Biological and Environmental Sciences
0
63157
218
1653
Biological and Environmental Sciences
0
63195
219
1680
Biological and Environmental Sciences
0
63242
220
1685
Biological and Environmental Sciences
0
63248
221
1708
Biological and Environmental Sciences
0
63280
222
1736
Biological and Environmental Sciences
0
63333
223
1745
Biological and Environmental Sciences
0
63365
224
1747
Biological and Environmental Sciences
0
63373
225
1767
Biological and Environmental Sciences
0
63442
226
1775
Biological and Environmental Sciences
0
63456
227
1816
Biological and Environmental Sciences
0
63512
228
1851
Biological and Environmental Sciences
0
63586
229
1860
Biological and Environmental Sciences
0
63598
230
1869
Biological and Environmental Sciences
0
63609
231
1876
Biological and Environmental Sciences
0
63621
232
1917
Biological and Environmental Sciences
0
63671
233
1932
Biological and Environmental Sciences
0
63688
234
1937
Biological and Environmental Sciences
0
63693
235
1978
Biological and Environmental Sciences
0
63774
236
2017
Biological and Environmental Sciences
0
63839
Prepared by Summit Consulting, LLC
Reported Student (=1)
18
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
237
2019
Biological and Environmental Sciences
Program
0
63841
238
2024
Biological and Environmental Sciences
0
63850
239
2048
Biological and Environmental Sciences
0
63874
240
2050
Biological and Environmental Sciences
0
63876
241
2082
Biological and Environmental Sciences
0
63909
242
2104
Biological and Environmental Sciences
0
63932
243
2115
Biological and Environmental Sciences
1
15012
244
2127
Biological and Environmental Sciences
1
61768
245
2129
Biological and Environmental Sciences
1
61937
246
2132
Biological and Environmental Sciences
1
62370
247
2133
Biological and Environmental Sciences
1
62443
248
2134
Biological and Environmental Sciences
1
62452
249
2135
Biological and Environmental Sciences
1
62469
250
2139
Biological and Environmental Sciences
1
62570
251
2140
Biological and Environmental Sciences
1
62571
252
2148
Biological and Environmental Sciences
1
62809
253
2149
Biological and Environmental Sciences
1
62844
254
2150
Biological and Environmental Sciences
1
62882
255
2162
Biological and Environmental Sciences
1
63144
256
2163
Biological and Environmental Sciences
1
63147
257
2173
Biological and Environmental Sciences
1
63224
258
2178
Biological and Environmental Sciences
1
63324
259
2180
Biological and Environmental Sciences
1
63332
260
2182
Biological and Environmental Sciences
1
63339
261
2197
Biological and Environmental Sciences
1
63445
262
2198
Biological and Environmental Sciences
1
63446
263
2201
Biological and Environmental Sciences
1
63483
264
2203
Biological and Environmental Sciences
1
63493
265
2204
Biological and Environmental Sciences
1
63502
266
2205
Biological and Environmental Sciences
1
63507
267
2209
Biological and Environmental Sciences
1
63516
268
2211
Biological and Environmental Sciences
1
63530
269
2220
Biological and Environmental Sciences
1
63655
270
2224
Biological and Environmental Sciences
1
63663
271
2225
Biological and Environmental Sciences
1
63704
272
2227
Biological and Environmental Sciences
1
63712
273
2232
Biological and Environmental Sciences
1
63727
274
2233
Biological and Environmental Sciences
1
63729
275
2235
Biological and Environmental Sciences
1
63732
276
2236
Biological and Environmental Sciences
1
63733
277
2238
Biological and Environmental Sciences
1
63737
278
2239
Biological and Environmental Sciences
1
63741
279
2242
Biological and Environmental Sciences
1
63746
280
2245
Biological and Environmental Sciences
1
63752
281
2248
Biological and Environmental Sciences
1
63757
282
2249
Biological and Environmental Sciences
1
63763
283
2251
Biological and Environmental Sciences
1
63770
284
2253
Biological and Environmental Sciences
1
63784
Prepared by Summit Consulting, LLC
Reported Student (=1)
19
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
285
2254
Biological and Environmental Sciences
Program
1
63785
286
2258
Biological and Environmental Sciences
1
63803
287
2262
Biological and Environmental Sciences
1
63823
288
2264
Biological and Environmental Sciences
1
63842
289
2332
Fusion Energy Sciences
0
54628
290
2346
Fusion Energy Sciences
0
54659
291
2358
Fusion Energy Sciences
0
54685
292
2380
Fusion Energy Sciences
0
54727
293
2389
Fusion Energy Sciences
0
54744
294
2398
Fusion Energy Sciences
0
54764
295
2406
Fusion Energy Sciences
0
54783
296
2411
Fusion Energy Sciences
0
54788
297
2421
Fusion Energy Sciences
0
54798
298
2430
Fusion Energy Sciences
1
52118
299
2432
Fusion Energy Sciences
1
52131
300
2438
Fusion Energy Sciences
1
53222
301
2441
Fusion Energy Sciences
1
53296
302
2442
Fusion Energy Sciences
1
54109
303
2453
Fusion Energy Sciences
1
54344
304
2455
Fusion Energy Sciences
1
54350
305
2460
Fusion Energy Sciences
1
54376
306
2461
Fusion Energy Sciences
1
54413
307
2462
Fusion Energy Sciences
1
54437
308
2464
Fusion Energy Sciences
1
54458
309
2465
Fusion Energy Sciences
1
54459
310
2467
Fusion Energy Sciences
1
54464
311
2468
Fusion Energy Sciences
1
54475
312
2470
Fusion Energy Sciences
1
54513
313
2478
Fusion Energy Sciences
1
54538
314
2480
Fusion Energy Sciences
1
54546
315
2493
Fusion Energy Sciences
1
54692
316
2497
Fusion Energy Sciences
1
54708
317
2500
Fusion Energy Sciences
1
54715
318
2504
Fusion Energy Sciences
1
54737
319
2505
Fusion Energy Sciences
1
54738
320
2510
Fusion Energy Sciences
1
54747
321
2511
Fusion Energy Sciences
1
54750
322
2513
Fusion Energy Sciences
1
54753
323
2516
Fusion Energy Sciences
1
54760
324
2517
Fusion Energy Sciences
1
54762
325
2519
Fusion Energy Sciences
1
54765
326
2521
Fusion Energy Sciences
1
54768
327
2526
High Energy Physics
0
40226
328
2527
High Energy Physics
0
40291
329
2533
High Energy Physics
0
40797
330
2550
High Energy Physics
0
41184
331
2575
High Energy Physics
0
41283
332
2602
High Energy Physics
1
40153
Prepared by Summit Consulting, LLC
Reported Student (=1)
20
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Sample No
Unique ID
333
2605
High Energy Physics
Program
1
40231
334
2606
High Energy Physics
1
40546
335
2615
High Energy Physics
1
40631
336
2617
High Energy Physics
1
40643
337
2622
High Energy Physics
1
40661
338
2631
High Energy Physics
1
40679
339
2635
High Energy Physics
1
40685
340
2639
High Energy Physics
1
40695
341
2646
High Energy Physics
1
40709
342
2647
High Energy Physics
1
40715
343
2648
High Energy Physics
1
40716
344
2652
High Energy Physics
1
40788
345
2653
High Energy Physics
1
40823
346
2656
High Energy Physics
1
40854
347
2658
High Energy Physics
1
40893
348
2664
High Energy Physics
1
40917
349
2665
High Energy Physics
1
40919
350
2667
High Energy Physics
1
40938
351
2671
High Energy Physics
1
40954
352
2675
High Energy Physics
1
40969
353
2677
High Energy Physics
1
41005
354
2680
High Energy Physics
1
41027
355
2685
High Energy Physics
1
41045
356
2687
High Energy Physics
1
41104
357
2689
High Energy Physics
1
41117
358
2694
High Energy Physics
1
41262
359
2698
High Energy Physics
1
41288
360
2700
High Energy Physics
1
41290
361
2703
High Energy Physics
1
41295
362
2705
High Energy Physics
1
41299
363
2707
High Energy Physics
1
41305
364
2709
High Energy Physics
1
41308
365
2710
High Energy Physics
1
41310
366
2716
Lab Ops and ES&H
0
35838
Prepared by Summit Consulting, LLC
Reported Student (=1)
21
Award No
12/28/06
DOE Office of Science
Technical Manpower Survey Sampling Plan
Appendix B: Questionnaire Flowchart
Prepared by Summit Consulting, LLC
22
12/28/06
| File Type | application/pdf |
| File Title | Microsoft Word - SUPPORTING STATEMENT PART B - Sampling Plan 08 24 06.doc |
| Author | Albert Lee |
| File Modified | 2006-12-28 |
| File Created | 2006-12-28 |