OSHRC Supporting Statement B

Supporting Statement B for
Paperwork Reduction Act Submission1
Survey of Participants in OSHRC Settlement Part Program

1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of
entities (e.g., establishments, State and local government units, households, or persons)
in the universe covered by the collection and in the corresponding sample are to be
provided in tabular form for the universe as a whole and for each of the strata in the
proposed sample. Indicate expected response rates for the collection as a whole. If the
collection had been conducted previously, include the actual response rate achieved
during the last collection.
This is a one-time data collection that will be conducted by Indiana University (IU) pursuant to
its contract with OSHRC. The universe for sampling is approximately 300 participants
comprised of employers and Department of Labor personnel, Authorized Employee
Representatives and their representatives, including attorneys, who have personally participated
in the Settlement Part program from February 15, 2011 through February 14, 2012. No sampling
or other respondent selection method will be used. Because this is a new information collection,
we do not have direct estimates of response rates for this survey.
IU has recently taken steps to “clean the data” in the data provided by OSHRC to reconcile
OSHRC’s estimate that up to 750 individuals participated in settlement part cases. After separating
prospective survey participants into separate mailings (Participants in Settlement Part cases will
receive a separate survey from participants in the contemplated second survey of select conventional
proceedings where between $50,000 and $99,999 is involved), IU found that the available data
showed a total of about 491 settlement part case related records. After IU recently removed
duplicates (several cases involved multiple docket numbers that involved the same settlement part
participants) and made other data corrections, the settlement part survey mailing list prepared by IU
presently shows about 251 identifiable participants. Any edits to the original list were done to
ensure that all individuals have the same probability of selection. Further data cleaning by IU may
cause some heretofore unidentified participants to be added, but IU does not expect the total
mailing of this survey to exceed 300. This sample size is manageable from the standpoint of
pursuing follow up and maximizing response rates. IU will seek to achieve, but cannot guarantee a
70% response rate for a voluntary survey.
Estimated number of All
participants in the Settlement Part
Program between February 15, 2011
through February 14, 2012
300
1

Targeted
Response
Rate2
70%

Number of
Responses
210

This submission supersedes OSHRC’s earlier submission of March 13, 2012.
Every effort will be made to maximize the response rate. Although we are hopeful that our efforts will achieve a 70%
response, there is no way to guarantee this with a voluntary survey.

2

2. Describe the procedures for the collection of information including:
* Statistical methodology for stratification and sample selection,
* Estimation procedure,
* Degree of accuracy needed for the purpose described in the justification,
* Unusual problems requiring specialized sampling procedures, and
* Any use of periodic (less frequent than annual) data collection cycles to reduce
burden.
Projected Response Rate
The survey will be of the estimated 300 participants in the Settlement Part program from
February 15, 2011 through February 14, 2012. No stratification or sampling or other respondent
selection method will be used. There is no issue with the degree of accuracy as the survey will be
sent to all identifiable participants. There are no unusual problems requiring specialized
sampling procedures. There will not be any use of a periodic data collection.
IU will make every effort to achieve a higher response rate of 70% for this survey as described in
more detail below. Given the mixed method approach being used by IU in its study, IU is confident
that it can obtain useful information from this survey even if a lower response rate than 70% is
achieved. It is also cost effective for OSHRC to conduct this survey. Significantly more value will be
realized by OSHRC through the conduct of this survey, whatever the response rate, than without it.
Preliminarily, the best way to minimize non-response bias is to achieve the highest possible response
rate. IU will employ the most recent strategies recommended by Dillman.3 Specifically, the initial
mailing will include a cover letter on IU letterhead, a brief letter from OSHRC, the survey
instrument printed on paper of a different color, and a postage paid preaddressed envelope to
facilitate returns. The survey cover letter explains the research objectives, emphasizes the importance
and voluntary nature of the survey, and addresses confidentiality in detail. [See attached sample IU
cover letter]. The assurance of confidentiality (to the fullest extent possible) is also considered
important to achieving the highest possible response rate. To further maximize response rate, IU
requests that respondents return completed surveys within one week. The cover letters are
addressed to individual participants and include original signatures in blue ink. A separate brief letter
from the agency is also part of the original mailing to encourage responses. The survey instrument is
light blue in color and is designed so that it is easy to follow. It is as brief as practical to obtain the
information relevant to the analysis. Up to two reminder post cards will be sent. The first reminder
post card will be sent if a response is not received after 7 days. After 10 days a reminder email will be
sent in instances where we have current email addresses.

3

See Dillman, Don, Jolene Smyth, and Leah Melani Christian (2009). Internet, Mail, and Mixed-Mode Surveys:
The Tailored Design Method. Hoboken, NJ: John Wiley & Sons.

2

Non-Response Bias Analyses
Even with IU’s best attempts to maximize returns, we do not expect to achieve a 100% response
rate. Because the unit response rate is likely to be below 80% and the item response rate may be
below 70% for some items on the survey, IU will also conduct a non-response bias analysis in
accordance with OMB guidelines 1.3.4 and 1.3.5 (Office of Management and Budget Standards and
Guidelines for Statistical Surveys, September 2006: 8). IU will report on and address two distinct
non-response categories: unit non-responses and item non-responses.
Unit non-responses arise when entire survey instruments are not returned and item non-responses
arise when survey instruments are returned, but are not entirely completed. For unit non-responses,
IU will attempt to obtain as much information as possible about non-responders. For example, IU
will report on the similarities between late responders, or those who respond only after follow ups,
and unit non-responders as there is some evidence that there is a common source for reluctance of
late responders and non-responders. IU will also compare characteristics of responders and nonresponders to check for evidence of non-response bias. IU has already obtained information for
this purpose including participant type (i.e. attorney for Department of Labor, labor union
representatives, company representative, business owner, etc. ) and case characteristics (e.g. total
dollar amount of penalty, days in the settlement part process, citation types, region where disputed).
Differences in non-response rates across sub-groups of the population provide possible evidence of
bias. This additional information also provides the basis for statistical tests for bias and bias
corrections. Item non-responses are also reported for each question. In the case of item nonresponses, partially completed responses will allow IU to control for possible differences between
responders and item non-responders and to compute unbiased estimates.
In addition, IU is aware of the various calculations for un-weighted unit and weighted unit response
rates (RRU & RRW). IU is also familiar with the numerous statistical techniques for non-response
bias correction; including weighting, imputing data, and maximum likelihood. In the final report to
OSHRC, we expect that IU will report on overall response rates, unit non-response rates and item
non-response rates. IU will clarify the related implications and also report results of the various
correction techniques.
3. Describe methods to maximize response rates and to deal with issues of non-response.
The accuracy and reliability of information collected must be shown to be adequate for
intended uses. For collections based on sampling, a special justification must be
provided for any collection that will not yield "reliable" data that can be generalized to
the universe studied.
The survey will be distributed by mail. Participation is voluntary. The completed surveys are to
be returned to the contractor by the postage-paid envelopes included with surveys. See response
to question no. 2, above, for additional detail on all steps to maximize response and techniques to
check for, test, report on, and correct any non-response bias.
Validity
In designing the survey, we worked closely with IU to ensure that empirical measures reflect the
real meaning of concepts. We also helped to design questions to precisely target aspects of the
settlement program we want to better understand. We also vetted the questions with all key
3

OSHRC personnel to help establish that participants would be likely to interpret the question to
mean what we intended. IU established construct validity mainly by relying on the literature.
Construct validity refers to the degree to which the questions and measures tie to theoretical
relationships. For example, we included several questions on participants’ perception of the
settlement part process because they have a strong basis in the literature on procedural justice.
Reliability
Reliability in research specifically refers to the quality of measurement method. The main idea is
that the method should yield the same results when repeated. A yes/no- type question in the
survey is expected to yield higher reliability than a question that prompts an answer of “howmany times.” However, the questions about the number of times something occurred will also
potentially provide responses that are more meaningful for improving the Settlement Process. IU
also took a number of steps to achieve reliability in the design of the questions and will continue
to establish reliability in the coding of the responses. Questions were designed to yield responses
that were both mutually exclusive and exhaustive. IU’s efforts to achieve reliability will continue
in the coding of responses. IU will use objective criteria for the coding task itself. Also because
the coding task is by its nature one of clerical recording, IU will take steps to eliminate errors in
recording. IU will also perform random checks of codes that are input using multiple coders. In
the analysis, IU will report coefficients to show IU achieved acceptable levels of inter-coder
reliability.
4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged
as an effective means of refining collections of information to minimize burden and
improve utility. Tests must be approved if they call for answers to identical questions
from 10 or more respondents. A proposed test or set of tests may be submitted for
approval separately or in combination with the main collection of information.
Most survey questions have been pretested in a previous survey by IU. Other questions are
standard in the literature on dispute resolution. All questions have been vetted with key OSHRC
personnel.
In addition to the tests for non-response bias described above, IU will also perform tests to refine
collections of information. For example, IU typically uses statistical methods to determine if
responses are internally consistent to see if they can be combined into a single measure (e.g.
Cronbach’s alpha). For tests of inter-coder agreement and reliability (described above) IU also
performs statistical tests to report levels of agreement among coders and also to report the role of
chance agreement among coders. The main test gives a statistic called a Cohen’s kappa.
IU also employs regression methods to answer questions such as: Is the type of case associated
with the participant’s level of satisfaction with the outcome? The specific method will depend
on the level of measurement for each question being examined. Based on the types of questions
in the settlement part survey, IU can expect to use logit analysis (when looking at factors that
lead to yes/no answers) and ordinal logit (when looking at factors that lead to ordinal type
responses/ level of satisfaction or agreement).

4

5. Provide the name and telephone number of individuals consulted on statistical aspects
of the design and the name of the agency unit, contractor(s), grantee(s), or other
person(s) who will actually collect and/or analyze the information for the agency.
The individuals directly responsible for information collection, analysis, and report writing
are:
Deanna Malatesta
Assistant Professor Public Affairs
801 W. Michigan St. BS 4075
School of Public and Environmental Affairs
Indiana University Purdue University Indianapolis
Indianapolis, IN 46202
(317) 274-0876
[email protected]
CV on SPEA website:
http://spea.iupui.edu/documents/cv_malatesta.pdf
SSRN Author
http://papers.ssrn.com/sol3/cf dev/AbsByAuth.cfm?per id=937720
Lisa Blomgren Bingham
Keller-Runden Professor of Public Service
School of Public and Environmental Affairs
Indiana University
1315 E. 10th Street
Bloomington, IN 47405
Tel. 812-855-4556/1465
[email protected]
CV on SPEA website:
http://www.indiana.edu/~spea/faculty/bingham-lisablomgren%20.shtml
Publications available at SSRN site:
http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=255016
Attachments
Survey IU cover letter
Reminder Post Card

5