DOT Checklist

U.S. Department of Transportation
Checklist for Paperwork Reduction Act Submissions
Before forwarding a Paperwork Reduction Act (PRA) submissions to the Bureau of
Transportation Statistics for review, the OA Manager of the sponsoring program office of
the proposed information collection or the OA CIO should review the package, and then
complete, sign and date the following checklist. Any responses of “No” marked on this
checklist should be accompanied by a written explanation of why the question is not
applicable to the package under consideration. The “Reference Sections” covered in the
final column are provided in the attached “Summary of Administrative and Technical
Features for Paperwork Reduction Act Submissions in the Department of
Transportation.”
Yes No

Topic

The PRA submission and instructions have been reviewed by
the OA Manager or OA CIO, and by the senior technical
representative responsible for the survey. In addition, if the
agency intends to use a contractor to prepare a PRA submission,
the contract must specifically require that the contractor “prepare
all materials necessary for a PRA submission for this study,
with format and content that meets the standards required by the
BTS and OMB.”
NA NA Payments or gifts to respondents, if any, are carefully justified
Confidentiality and disclosure limitation methods are fully
X
specified; including citing the authority for any confidentially
pledge.
OA Manager or OA CIO, and OA statistical technical
X
representative have reviewed and approved all statements of
accounting fact in the proposed package.
X Population, parameters, scope of inference and frames are fully
and explicitly specified, with detailed mathematical formulas
where appropriate.
X

X

1

Reference
Sections
I.1-I.2
and
II.1

II.2
II.3
II.4
II.5

Explanation: This is a simple questionnaire with no intended
utility other than gaining insight on the opinions & experience of
the UAS Operator community. No statistical analysis or
inference will be performed, other than tabulation of responses.
The population of interest, in this case, is the community of UAS
Operators who currently hold an FAA waiver, are applying for
an FAA waiver, or intend to apply (i.e. current or prospective
“customers” of the FAA).
Sample design, sample sizes and degree of accuracy are fully and II.6
explicitly specified, with detailed mathematical formulas where
appropriate.

X

X

X

Explanation: This is a simple questionnaire with no intended
utility other than gaining insight on the opinions & experience of
the UAS Operator community. No statistical analysis or
inference will be performed, other than tabulation of responses.
There is no target sample size in this case; we intend to survey
the entire population to the greatest extent possible.
Probability weights and design-based point estimators are fully
specified; or model-based estimation methods (including all
model validation procedures to be used) are fully specified
Explanation: This is a simple questionnaire with no intended
utility other than gaining insight on the opinions & experience of
the UAS Operator community. No statistical analysis or
inference will be performed, other than tabulation of responses.
No statistical model is needed, in this case, because we will not
be drawing any substantive conclusions from the data (Note that
results are only intended to justify and refine prospective FAA
research).
Design-based variance estimators and inference methods fully
specified; or model-based variance estimators and inference
methods are fully specified.
Explanation: This is a simple questionnaire with no intended
utility other than gaining insight on the opinions & experience of
the UAS Operator community. No statistical analysis or
inference will be performed, other than tabulation of responses.
We will not be making inferences or establishing correlations
from the data, so variance estimators and inference methods are
not needed.
Explicit justifications of anticipated response rates are included

II.7

II.8

II.9

Explanation: This is a simple questionnaire with no intended
utility other than gaining insight on the opinions & experience of
the UAS Operator community. No anticipation of response-rate
has been made.
X Methods of nonresponse weighting adjustment and/or imputation II.10
are explicitly defined and justified
The proposed package includes one copy of each technical report II.11
X
or reference cited in the package, except for materials widely
available in the public domain.
NA NA (For reviews after the initial review):
II.12
The package submitted to the BTS includes an appendix with
detailed point-by-point responses to each question raised by the
reviewer in the previous review.

2

OA Statistical Technical Representative:
_Philip Maloney__
Name

Digitally signed by PHILIP
PHILIP
MALONEY
Date: 2020.03.25 11:40:16
MALONEY
-04'00'
______________________

Signature

_01/23/20__ __609-485-7349____
Date
Telephone number

[email protected]__
E-mail address

OA Manager or OA Chief Information Officer:
________________
Name

__________________
Signature

___________________
E-mail address

3

_______
Date

________________
Telephone number

Summary of Administrative and Technical Features for Paperwork Reduction Act
Submissions in the Department of Transportation
I. Administrative Features of the PRA Process in the Department of Transportation
I.1. Background
Under the Paperwork Reduction Act (44 U.S.C. chapter 35) and related regulations
issued by the Office of Management and Budget (5 CFR §1320), OMB approval is
required for essentially all surveys conducted for federal agencies. The OMB has
established a process, for Paperwork Reduction Act submissions and related
questionnaires (“Request for OMB Clearance of Information Collection under the
Paperwork Reduction Act of 1995. Part A: Justification. Part B: Collections of
Information Involving Statistical Methods”) to be completed by the agency that proposes
to conduct the survey. This includes all surveys used for statistical purposes, and also
includes any census (100% sample) that collects data used to make inferences to a larger
population. Copies of OMB’s instructions for PRA submissions are available at:
http://www.whitehouse.gov/omb/inforeg/83i-fill.pdf1
Somewhat shorter questionnaires are provided for customer satisfaction surveys.
In addition, pursuant to 5 CFR § 1320, the Department of Transportation has established
a policy that before a PRA submission is transmitted to the OMB, it must first be
reviewed and approved by the DOT Clearance Officer. For data collection work that will
involve statistical methods (see section I.2.c below), the PRA submission must also be
reviewed and approved by the Bureau of Transportation Statistics.
The materials provided in this summary are intended for studies in which the lead agency
is an agency in the Department of Transportation other than the Bureau of Transportation
Statistics.
I.2. Assignment of Responsibilities.
I.2.a. Lead Agency. The lead agency is responsible for specification of the purpose and
scope of the study; general management of the study; qualifications, selection,
management and performance of the government or contractor personnel assigned to the
study; budgeting and statements of fact regarding accounting or respondent burden; and
inferences, interpretations or policy statements based on results of the study. Much of
this work requires a high degree of specialized technical and managerial training and
experience, and it is the responsibility of the lead agency to ensure that the personnel
assigned to this work have the necessary training and experience. Important evaluation
1

Please note, the Form OMB 83-I is obsolete as of July 2006; however, OMB instructions for completing
both Parts A and B of the Justification for a Paperwork Reduction Act Submission are attached to the Form
OMB 83-I and remain in effect.

4

criteria include a record of strong training and performance in similar previous studies,
and demonstrated capacity for continued strong performance in the proposed study. This
includes work in the following areas.
i.

Construction of probability sample designs appropriate for the proposed study.
This generally involves definition of target populations and parameters; frame
development; stratification; multistage selection and unequal-probability
sampling.

ii.

If the agency proposes to use a census (100% sample), justification of the decision
not to collect data on a sample basis.

iii.

Development and implementation of appropriate statistical formulas for point
estimation, variance estimation and inference.

iv.

Design, testing and implementation of data collection instruments and field
procedures.

v.

Data collection and management of field staff.

vi.

Management, cleaning, editing and nonresponse adjustment of data; and
production of valid survey estimates.

vii.

Interpretation of inferential results and production of appropriate reports.

viii.

Management of projects, personnel, information technology and finances; and
compliance with all applicable laws, regulations and contractual requirements.

ix.

In addition, if the agency intends to use a contractor to prepare a PRA submission,
the contract must specifically require that the contractor “prepare all materials
necessary for a PRA submission for this study, with format and content that meets
the standards required by the BTS and OMB.”

I.2.b. DOT Clearance Officer. The DOT Clearance Officer manages the review and
submission process for PRA submissions; and is the primary point of contact in the DOT
for questions regarding the said submissions.
I.2.c. Bureau of Transportation Statistics. For studies in the Department of
Transportation for which the Bureau of Transportation Statistics is not the lead agency,
the role of the BTS centers on review of technical material in the PRA submission, such
as specification of the sample design, estimation methods, and related statistical topics.
For the PRA submissions, there are two distinct cases.
i.

5

Studies based on a 100% sample and limited inferential goals. In some cases,
data collection by a DOT agency will satisfy both of the following conditions.
(A) Data collection is attempted for all units in the inference population. (This

often is referred to as a census or a 100% sample.) (B) No attempt will be made
to draw inferences to any population other than the set of units that responded to
the data collection effort in (A). (C) The response rate is expected to be close to
100%. If each of (A), (B) and (C) are true, then there is no need for a review by
the BTS. Instead, the lead agency should forward the PRA submission to the
DOT Clearance Officer, and should state in the Justification Part B, “No statistical
methods will be used. All data collection will be based on a 100% sample of the
inference population. In all reports and other publications and statements
resulting from this work, no attempt will be made to draw inferences to any
population other than the set of units that responded to the data collection effort.”
ii.

All other studies involving statistical methods. In keeping with the Paperwork
Reduction Act and related regulations for the PRA submission process, the
Bureau of Transportation Statistics is asked to review technical features of the
sample design, and related statistical estimation and inference methods, covered
in a proposed PRA submission package. Issues regarding the purpose and scope
of the study; general management of the study; qualifications, selection,
management and performance of the government or contractor personnel assigned
to the study; budgeting and statements of fact regarding accounting or respondent
burden; inferences, interpretations or policy statements based on results of the
study; or decisions by the DOT Clearance Officer, are outside the scope of BTS
responsibility.

I.2.d. Office of Management and Budget. The Office of Management and Budget has
final authority over approval or disapproval of PRA submission packages submitted to it
for review.

6

II. Technical Features of the PRA Submission Process
A copy of the “Request for OMB Clearance of Information Collection Under the
Paperwork Reduction Act of 1995. Part A: Justification. Part B: Collections of
Information Involving Statistical Methods” is attached to the end of this summary. Parts
A and B cover several important statistical topics.
For PRA submission packages handled through the Department of Transportation, the
Bureau of Transportation Statistics has developed a “Checklist for PRA Submissions”
(attached).
II.1. Before the start of work on the study, the PRA submission; including, Parts A and B
should be reviewed by the OA Manager or OA CIO, and the OA statistical technical
representative in the lead agency, and (if applicable) by the responsible senior
management and senior technical personnel in each contractor and subcontractor.
Questions A.1-A.8, A.11, A.15-A.18, B.4 and B.5 are largely self-explanatory. The
remaining questions in Parts A and B are covered in additional lines of the “Checklist for
PRA Submissions” as follows.
II.2. Incentives. Question A.9 covers payments or gifts to respondents, other than
remuneration of contractors or grantees. The OMB generally discourages the use of gifts
or payments to respondents in federal surveys. If the lead agency considers such gifts or
payments to be necessary, it is advisable to respond to Question A.9 with a careful
justification of these gifts of payments, with appropriate citations of relevant empirical
literature on respondent incentives, response rates and related methodological issues.
II.3. Respondent confidentiality. Question A.10 covers assurances of confidentiality
given to respondents. The agency must state whether the respondents will receive an
explicit pledge of confidentiality in the collection instrument or in related field
procedures. If the respondents are given assurances of confidentiality, then the answer to
Question A.10 question should include explanation of these assurances, and the basis of
these assurances in statute, regulation, or agency policy. As part of the “agency policy”
component, the response also should include an explanation of disclosure-limitation
procedures that will be used in dissemination of results from this survey. This generally
will involve a description of the level of aggregation at which results will be published;
and a description of specific methods that will be used to ensure that inadvertent
disclosure will not take place through, e.g., publication of tables at excessively fine levels
of aggregation. For any cases involving public-use microdata files, the answer to
Question A.10 should also include an explanation of the microdata disclosure-limitation
procedures that will be implemented.
II.4. Statements of accounting fact. Items A.12, A.13 and A.14 cover, respectively, the
respondent hour burden, the respondent cost burden and the annualized cost to the federal
government. As with any statement of accounting fact, it is very important for the lead
agency’s responses to items A.12, A.13 and A.14 be complete, accurate and transparent.
The OMB has stated that under the “total cost to the government” response to item A.14,
7

the lead agency should include full cost of the entire study, including the cost of sample
and questionnaire design; data collection; data management; editing, processing,
analyzing, reviewing, and interpreting the information; preparation of reports and
documentation; reviewing reports; and (if applicable) preparing public use files. If a
contractor is doing some or all of the abovementioned tasks, then the cost of the contract
should be included, but agency staff time devoted to managing or doing the project
should be included as well. Some components, such as contractor costs, will be relatively
easy to know precisely. Other components, such as the costs of government employee
time, will need to be a good-faith estimate. The costs reported in A.14 should be broken
down into major categories of expense.
II.5. Population, parameters, scope of inference and frames. Question B.1 covers several
important statistical issues. The response to Question B.1 should cover all of the issues
raised in this question, either through direct answers or through referral to a more detailed
answer under Questions B.2 or B.3. In addition, the initial part of the response to
Question B.1 should include a clear and explicit statement of the inference population; an
explicit mathematical definition of the parameters of primary importance for the study;
and a clear and explicit statement of the frame(s) that will be used in the survey.
II.6. Specification of the sample design, sample sizes and degree of accuracy. The
response to Question B.2 should cover all of the issues raised in this question. This
should include explicit statements (with detailed mathematical formulas as appropriate)
of the specific methods that will be used for formation of strata, definition of units at each
stage of sample selection, calculation of selection probabilities. In addition, the response
to Question B.2 should include clear and explicit statements (including mathematical
formulas where appropriate) regarding: the degree of accuracy needed for the purpose
described in Part A; assumed design effects (with citation of appropriate technical reports
or other references to support these design-effect assumptions); and the resulting
calculations of sample sizes that will ensure that the needed degree of accuracy is
satisfied.
II.7. Point estimation methods. The response to Question B.2 should also state clearly
and explicitly whether the proposed work will use design-based or model-based
estimation and inference methods, and give a clear justification for this choice. For
design-based work, the response to Question B.2 should include clear and explicit
statements (including detailed mathematical formulas) for probability weights and
design-based point estimators for each of the principal parameters of interest, as specified
in the response to Question B.1. For model-based work, the response to Question B.2
should include clear and explicit statements (including detailed mathematical formulas)
of the statistical models being used; point estimators for each of the principal parameters
of interest, as specified in the response to Question B.1; and specific empirical
procedures that will be used to check the validity of the models under consideration.
II.8. Variance estimation and inference methods. The response to Question B.2 should
also include clear and explicit statements (including detailed mathematical formulas) for
each of variance estimators and inference methods (e.g., test statistics or confidence
8

intervals) that will be used for the parameter estimators specified under (II.7). The
statements on variance estimation and inference methods should be clearly and explicitly
justified within the framework of the design-based or model-based approaches specified
under (II.7). Any additional approximations employed in this variance estimation work
should be clearly and explicitly stated and justified with detailed mathematical formulas
and literature references, as appropriate.
II.9. Justification of anticipated response rates. The response to Question B.3 should
cover all of the issues raised in this question. In addition, the anticipated response rates
in B.1, and the methods to maximize response rates should be justified through careful
citation of empirical and methodological literature that is relevant to the proposed study.
II.10. Nonresponse adjustments. The response to Question B.3 also should include clear
and explicit statements (including mathematical formulas as appropriate) of the specific
weighting adjustment or imputation procedures that will be used. In addition, the
underlying assumptions (e.g., homogeneity of response rates or mean values) for these
weighting adjustment and imputation procedures should be stated explicitly.
II.11. Reference materials. PRA submission packages generally include a bibliography
that lists references cited in the text of the package. Some of these references are
textbooks, journal articles, web pages or other materials that are widely accessible to the
public. However, some of the references may include agency technical reports or other
materials that are not readily available to the BTS and OMB reviewers. For any cited
references in the latter category, one paper or electronic copy of the reference should be
included with the PRA submission package submitted to the BTS.
II.12. Point-by-point responses to issues raised in previous reviews. If the BTS does not
approve a PRA submission package on its first review, then the BTS generally will
provide a report that lists specific questions or areas in which the lead agency will need to
do additional work before the package is ready for submission to the OMB. When the
lead agency submits a revised package to the BTS for further review, the lead agency
should include an appendix that provides point-by-point responses to the issues identified
by the BTS in its previous review.

9