OSHRC Supporting Statement B

Supporting Statement B for
Paperwork Reduction Act Submission
Survey of Participants in OSHRC Conventional Proceedings
where between $50,000 and $99,999 is at issue

1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of
entities (e.g., establishments, State and local government units, households, or persons)
in the universe covered by the collection and in the corresponding sample are to be
provided in tabular form for the universe as a whole and for each of the strata in the
proposed sample. Indicate expected response rates for the collection as a whole. If the
collection had been conducted previously, include the actual response rate achieved
during the last collection.
This is a one-time data collection that will be conducted by Indiana University (IU) pursuant to
its contract with OSHRC. The universe for sampling is approximately 300 participants
comprised of employers and Department of Labor personnel, Authorized Employee
Representatives and their representatives, including attorneys, who have personally participated
in OSHRC conventional proceedings cases from February 15, 2011 through June 30, 2012,
where a total proposed penalty between $50,000 and $99,999 was involved and where OSHRC
Settlement Part Process procedures were not used. The cases would include those settled by the
parties without an OSHRC judge conducting a face-to-face settlement proceeding, as well as any
cases within the above dollar range that went to a trial on the merits. No sampling or other
respondent selection method will be used. Because this is a new information collection, we do
not have direct estimates of response rates for this survey.
IU has recently taken steps to “clean the data” in the data provided by OSHRC to reconcile
OSHRC’s estimate that up to 500 individuals participated in conventional proceedings. After
separating prospective survey participants into separate mailings and removing duplicates
(several cases involved multiple docket numbers that involved the same participants) and making
other data corrections, the conventional proceedings survey mailing list prepared by IU presently
shows about 300 identifiable participants. Any edits to the original list were done to ensure that
all individuals have the same probability of selection. Further data cleaning by IU may cause
some heretofore unidentified participants to be added, but IU does not expect the total mailing of
this survey to exceed 300. This sample size is manageable from the standpoint of pursuing
follow up and maximizing response rates. IU will seek to achieve, but cannot guarantee a 70%
response rate for a voluntary survey.

Estimated Number of All
Participants in Conventional
Proceedings Cases between
February 15, 2011 through June 30,
2012
300

Targeted
Response
Rate1
70%

Number of
Responses
210

2. Describe the procedures for the collection of information including:
* Statistical methodology for stratification and sample selection,
* Estimation procedure,
* Degree of accuracy needed for the purpose described in the justification,
* Unusual problems requiring specialized sampling procedures, and
* Any use of periodic (less frequent than annual) data collection cycles to reduce
burden.
Projected Response Rate
The survey will be of the estimated 300 participants in conventional proceedings from February
15, 2011 through June 30, 2012. No stratification or sampling or other respondent selection
method will be used. There is no issue with the degree of accuracy as the survey will be sent to
all identifiable participants. There are no unusual problems requiring specialized sampling
procedures. There will not be any use of a periodic data collection.
IU will make every effort to achieve a higher response rate of 70% for this survey as described in
more detail below. Given the mixed method approach being used by IU in its study, IU is
confident that it can obtain useful information from this survey even if a lower response rate than
70% is achieved. It is also cost effective for OSHRC to conduct this survey. Significantly more
value will be realized by OSHRC through the conduct of this survey, whatever the response rate,
than without it.
Preliminarily, the best way to minimize non-response bias is to achieve the highest possible
response rate. IU will employ the most recent strategies recommended by Dillman.2
Specifically, the initial mailing will include a cover letter on IU letterhead, a brief letter from
OSHRC, the survey instrument printed on paper of a different color, and a postage paid
preaddressed envelope to facilitate returns. The survey cover letter explains the research
objectives, emphasizes the importance and voluntary nature of the survey, and addresses
confidentiality in detail. [See attached sample IU cover letter]. The assurance of confidentiality
1

Every effort will be made to maximize the response rate. Although we are hopeful that our efforts will achieve a 70%
response, there is no way to guarantee this with a voluntary survey.
2
See Dillman, Don, Jolene Smyth, and Leah Melani Christian (2009). Internet, Mail, and Mixed-Mode Surveys:
The Tailored Design Method. Hoboken, NJ: John Wiley & Sons.

2

(to the fullest extent possible) is also considered important to achieving the highest possible
response rate. To further maximize response rate, IU requests that respondents return completed
surveys within one week. The cover letters are addressed to individual participants and include
original signatures in blue ink. A separate brief letter from the agency is also part of the original
mailing to encourage responses. The survey instrument is light blue in color and is designed so
that it is easy to follow. It is as brief as practical to obtain the information relevant to the
analysis. Up to two reminder post cards will be sent. The first reminder post card will be sent if a
response is not received after 7 days. After 10 days a reminder email will be sent in instances
where we have current email addresses.
Non-Response Bias Analyses
Even with IU’s best attempts to maximize returns, we do not expect to achieve a 100% response
rate. Because the unit response rate is likely to be below 80% and the item response rate may be
below 70% for some items on the survey, IU will also conduct a non-response bias analysis in
accordance with OMB guidelines 1.3.4 and 1.3.5 (Office of Management and Budget Standards
and Guidelines for Statistical Surveys, September 2006: 8). IU will report on and address two
distinct non-response categories: unit non-responses and item non-responses.
Unit non-responses arise when entire survey instruments are not returned and item non-responses
arise when survey instruments are returned, but are not entirely completed. For unit nonresponses, IU will attempt to obtain as much information as possible about non-responders. For
example, IU will report on the similarities between late responders, or those who respond only
after follow ups, and unit non-responders as there is some evidence that there is a common
source for reluctance of late responders and non-responders. IU will also compare characteristics
of responders and non-responders to check for evidence of non-response bias. IU has already
obtained information for this purpose including participant type (i.e. attorney for Department of
Labor, labor union representatives, company representative, business owner, etc. ) and case
characteristics (e.g. total dollar amount of penalty, days cases in conventional proceedings,
citation types, region where disputed). Differences in non-response rates across sub-groups of
the population provide possible evidence of bias. This additional information also provides the
basis for statistical tests for bias and bias corrections. Item non-responses are also reported for
each question. In the case of item non-responses, partially completed responses will allow IU to
control for possible differences between responders and item non-responders and to compute
unbiased estimates.
In addition, IU is aware of the various calculations for un-weighted unit and weighted unit
response rates (RRU & RRW). IU is also familiar with the numerous statistical techniques for
non-response bias correction; including weighting, imputing data, and maximum likelihood. In
the final report to OSHRC, we expect that IU will report on overall response rates, unit nonresponse rates and item non-response rates. IU will clarify the related implications and also
report results of the various correction techniques.
3. Describe methods to maximize response rates and to deal with issues of non-response.
The accuracy and reliability of information collected must be shown to be adequate for
intended uses. For collections based on sampling, a special justification must be
3

provided for any collection that will not yield "reliable" data that can be generalized to
the universe studied.
The survey will be distributed by mail. Participation is voluntary. The completed surveys are to
be returned to the contractor by the postage-paid envelopes included with surveys. Up to two
reminder post cards will be sent. The first reminder post card will be sent if a response is not
received after 7 days. After 10 days a reminder email will be sent in instances where we have
current email addresses. See response to question no. 2, above, for additional detail on all steps
to maximize response and techniques to check for, test, report on, and correct any non-response
bias.
Validity
In designing the survey, we worked closely with IU to ensure that empirical measures reflect the
real meaning of concepts. We also helped to design questions to precisely target aspects of the
settlement program we want to better understand. We also vetted the questions with all key
OSHRC personnel to help establish that participants would be likely to interpret the question to
mean what we intended. IU established construct validity mainly by relying on the literature.
Construct validity refers to the degree to which the questions and measures tie to theoretical
relationships. For example, we included several questions on participants’ perception of cases in
conventional proceedings because they have a strong basis in the literature on procedural justice.
Reliability
Reliability in research specifically refers to the quality of measurement method. The main idea is
that the method should yield the same results when repeated. A yes/no- type question in the
survey is expected to yield higher reliability than a question that prompts an answer of “howmany times.” However, the questions about the number of times something occurred will also
potentially provide responses that are more meaningful for improving the Settlement Process. IU
also took a number of steps to achieve reliability in the design of the questions and will continue
to establish reliability in the coding of the responses. Questions were designed to yield responses
that were both mutually exclusive and exhaustive. IU’s efforts to achieve reliability will continue
in the coding of responses. IU will use objective criteria for the coding task itself. Also because
the coding task is by its nature one of clerical recording, IU will take steps to eliminate errors in
recording. IU will also perform random checks of codes that are input using multiple coders. In
the analysis, IU will report coefficients to show IU achieved acceptable levels of inter-coder
reliability.
4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged
as an effective means of refining collections of information to minimize burden and
improve utility. Tests must be approved if they call for answers to identical questions
from 10 or more respondents. A proposed test or set of tests may be submitted for
approval separately or in combination with the main collection of information.
Most survey questions have been pretested in a previous survey by IU. Other questions are
standard in the literature on dispute resolution. All questions have been vetted with key OSHRC
personnel.

4

In addition to the tests for non-response bias described above, IU will also perform tests to refine
collections of information. For example, IU typically uses statistical methods to determine if
responses are internally consistent to see if they can be combined into a single measure (e.g.
Cronbach’s alpha). For tests of inter-coder agreement and reliability (described above) IU also
performs statistical tests to report levels of agreement among coders and also to report the role of
chance agreement among coders. The main test gives a statistic called a Cohen’s kappa.
IU also employs regression methods to answer questions such as: Is the type of case associated
with the participant’s level of satisfaction with the outcome? The specific method will depend
on the level of measurement for each question being examined. Based on the types of questions
in the conventional proceedings survey, IU can expect to use logit analysis (when looking at
factors that lead to yes/no answers) and ordinal logit (when looking at factors that lead to ordinal
type responses/ level of satisfaction or agreement).
5. Provide the name and telephone number of individuals consulted on statistical aspects
of the design and the name of the agency unit, contractor(s), grantee(s), or other
person(s) who will actually collect and/or analyze the information for the agency.
The individuals directly responsible for information collection, analysis, and report writing
are:
Deanna Malatesta
Assistant Professor Public Affairs
801 W. Michigan St. BS 4075
School of Public and Environmental Affairs
Indiana University Purdue University Indianapolis
Indianapolis, IN 46202
(317) 274-0876
[email protected]
CV on SPEA website:
http://spea.iupui.edu/documents/cv_malatesta.pdf
SSRN Author
http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=937720
Lisa Blomgren Bingham
Keller-Runden Professor of Public Service
School of Public and Environmental Affairs
Indiana University
1315 E. 10th Street
Bloomington, IN 47405
Tel. 812-855-4556/1465
[email protected]
CV on SPEA website:
http://www.indiana.edu/~spea/faculty/bingham-lisablomgren%20.shtml
Publications available at SSRN site:
http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=255016
5

Attachments:
Survey Cover Letter
Reminder Post Card

6