2015 Supporting Statement for the Occupational Requirements Survey
Justification, Part A.
This request is for the approval of a nationwide Occupational Requirements Survey (ORS). Under the ORS program, the Bureau of Labor Statistics (BLS) will conduct an ORS survey at the request of the Social Security Administration (SSA). This clearance package covers the first three years of collection starting in 2015 and ending in 2018. The data elements have been evaluated by both BLS and SSA while the sample design has been thoroughly evaluated by BLS.
Estimates produced from the data collected by the ORS will be considered by the SSA to update occupational requirements data used in administering the Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) programs. The collected data will also advance the mission of the BLS by making possible a detailed analysis and expansion of occupational data from several BLS programs, including the National Compensation Survey (NCS), the Occupational Employment Statistics program (OES), and the Occupational Safety and Health Statistics programs (OSHS), promoting the continued effective use of these data, and disseminating these data to a wider audience.
The ORS will collect the following data to meet the needs of SSA’s disability program. These data are not currently collected by any BLS program:
An indicator of “time to proficiency,” defined as the amount of time required by a typical worker to learn the techniques, acquire the information, and develop the facility needed for average job performance. This measure is comparable to the Specific Vocational Preparation (SVP) used in the Dictionary of Occupational Titles (DOT).
Physical Demand characteristics/factors of occupations. These measures are comparable to measures in Appendix C of the Selected Characteristics of Occupations (SCO).
Environmental Conditions. These measures are comparable to measures in Appendix D of the SCO.
Data elements that describe the mental and cognitive demands of work.
Occupational Task lists data similar to those found in the Employment and Training Administration's (ETA’s) O*NET Program in order to validate the key tasks common across establishments and identify other tasks commonly performed.
Some data needed for the ORS are collected by NCS currently from its sample of establishments. These data will be collected with the same methodology for ORS sample establishments that are not in the NCS sample. The general establishment data collected in the survey samples will be the same for the ORS and NCS. For ORS and NCS, these items are employees, occupations, divisions, or sub-units depending upon the application of the sampling procedure being used.
Work leveling data will be collected during the ORS as it is currently collected in NCS. The work level data for each of the four factors has several levels reflecting increasing duties and responsibilities, and there are point values associated with each level.
The four factors are:
Knowledge – the amount of knowledge required for the job
Job controls and complexity – the type of direction received and the nature of the job
Contacts – the nature and purpose of contacts within a job but outside the supervisory chain
Physical environment – risks involved and physical demands
The elements above and the unique ORS data elements will be collected by BLS field economists. “Field economist” is the BLS title for those who collect data from respondents. To collect ORS data, field economists will interview respondents who represent the companies, organizations, and government units within the sample. Field economists conduct these interviews by visiting the company or by phone or e-mail contact. Other communication media, such as faxes, mail, websites, and e-mail are used to assist the process, depending on the wishes of the respondents.
Field economists often initiate contact with companies through personnel or Human Resource staff, but may have multiple respondents within a company providing different information based on expertise. The goal is always to find the best data source while balancing response and burden concerns. Human Resource and management staff are familiar with the requirements of an occupation from the employee hiring and performance perspective. This helps ensure the answers reflect the establishment’s needs and not how a job has been adapted to a particular worker’s skills or personal style. During data collection, many respondents will either consult with supervisors or employees with the field economist present, or will obtain information from more knowledgeable staff and provide it to resolve particular questions.
When asking questions, field economists do not rely on a scripted interview. Instead, they ask probing questions to get the information. Often, conversational interviewing techniques are used. Field economists might ask questions in different ways to different respondents. Some respondents will be experts in the field of human resources, job requirements or compensation, while other respondents merely maintain pay and benefit records. Because of the different levels of respondent knowledge, combined with the scope and complexity of ORS data collection, scripting an interview that covers most situations would be very difficult.
For ORS collection, the respondent does not complete the collection forms. The field economist asks for the needed information and uses the collection forms as a note-taking device. This information will then be entered into two electronic data systems (different parts are entered into each system); one a web-based computer database Compensation Information Entry and Review Application (CIERA) that collects ORS-only data and the other the NCS Integrated Data Capture (IDC) System. Field economists will use the newly developed ORS collection system after the completion of the interview with the respondent. The existing IDC system that NCS currently uses will be used for the entry and capture of general information, and leveling for ORS sample establishments in this survey.
ORS policy is to collect the data in whichever manner is easiest for the respondents to provide and then reformat those data to conform to ORS requirements. This approach could cause some non-sampling error, but new collection training and quality assurance programs are in place to lessen any impact on data collection.
SSA’s regulations require five steps of sequential evaluation to determine whether an adult claimant qualifies for disability benefits. For almost 50 years, SSA has relied on the Department of Labor’s (DOL) Dictionary of Occupational Titles (DOT) as the primary source of occupational information to make medical-vocational decisions. Although DOL did not design the DOT for SSA use, SSA incorporated many of the DOT’s concepts and definitions into SSA’s regulations and policy. The DOL stopped fully updating the DOT in 1991 and, in 1998, replaced it with the Occupational Information Network (O*NET). DOL developed O*NET as a career exploration tool, and it does not measure strength and physical requirements in a manner consistent with SSA’s disability rules.
As occupations and their duties and responsibilities have changed in the last 20 plus years, the relevance of DOT-based information has declined. More than half of the determinations SSA makes at the initial level, and more than 80 percent at the hearing level, are medical-vocational determinations that require current occupational information about work that exists in the national economy. SSA will use the occupational information collected through ORS to develop a new occupational information system to replace the outdated DOT in the disability determination process, as outlined in the Social Security Act Section 223(d)(2)(A) and accompanying regulations.
The authority for the BLS to perform special work or services on a cost basis is 31 United States Code §§ 1535/FAR 17.5 of the Economy Act.
There are multiple stakeholders for the new ORS occupational information, including the SSA and organizations involved in the disability community. The occupational information will be used to update, and improve the operation of, SSA’s disability programs, as described above. Specifically, the SSA will use ORS data in steps four and five of its disability benefits evaluation process, during which SSA must assess the functional impact of a claimants’ impairments and determine whether claimants can perform their past work or other work in the national economy. This process is described by the SSA in the Research and Statistics Note No. 2013-01. The BLS is presenting its work on the ORS to organizations such as the International Association of Rehabilitation Professionals (IARP) and the National Association of Disability Representatives (NADR) to determine the specific needs of this stakeholder community.
BLS field economists obtain data from respondents through personal interview, telephone, e-mail, fax, and web-site contacts. After the interview, BLS field economists will enter collected data into two electronic systems. The first, an existing Integrated Data Capture (IDC) system that NCS currently uses, will be used for the entry and capture of general information and leveling for ORS sample establishments in the survey. The application is designed for use on both laptop and desktop personal computers, and runs in the Microsoft Windows operating system.
For the data elements in ORS, a web-based, computer database system, Compensation Information Entry and Review Application (CIERA), using Oracle will be implemented. Field economists will use this ORS collection system after the completion of the interview with the respondent.
Some general information does not have to be collected from survey respondents as that data is available from the Quarterly Census of Employment and Wages (QCEW). QCEW is a relational database of business establishments linked longitudinally and based on the microdata submitted quarterly by States from Unemployment Insurance (UI) tax files. The QCEW serves as a sampling frame for the ORS and other establishment-based surveys. BLS data elements on these QCEW files include information on monthly employment, quarterly wages, business name and addresses, industry classification, geo codes, and other administrative data. Every business establishment contains a unique identifier that allows for tracking of individual establishments at the micro level across quarters for the United States. The BLS uploads these data into its computer system before the field economist visits the establishment, thereby reducing the burden on respondents to provide this basic information.
For sample establishments that are in the current NCS sample, the ORS will use the NCS data and only collect data on the ORS data elements for those establishments.
The ORS sample is designed to provide occupational requirements data that are representative of the national labor market. Therefore, information is collected from establishments of all sizes. Any establishment with at least one employee is a potential respondent. Respondents will not be asked to provide data more than once every three years for a sampled establishment, as described in Part B, Section 1a.
The aggregate collection burden on small establishments is significantly less than the burden on medium and large establishments. Establishment selection is performed using a systematic probability proportionate to size technique that uses employment as the measure of size. Therefore, larger firms have a greater chance of being selected. Furthermore, small establishments will have a smaller collection burden, because BLS collects data on fewer occupations in small establishments.
The Social Security Administration, Members of Congress, and representatives of the disability community have all identified collection of updated information on the requirements of work in today’s economy as crucial to the equitable and efficient operation of the Social Security disability programs. The information currently available is more than 20 years old.
There are no special circumstances for this collection.
BLS received four public comments on the Federal Register notice published in the Federal Register, 80 FR 8696, on February 18, 2015. Public comments were from two individual businesses within the vocational rehabilitation industry, one professional organization, and one government agency. The comments received concerned the sample design, methodology, existing ORS data elements, suggestions for additional ORS data elements, collection procedures, the manual, and uses of the data obtained by the ORS. Responses below are from BLS in conjunction with the Social Security Administration (SSA).
BLS carefully evaluated all of the comments received from the public. In response, BLS implemented key changes while maintaining a balance between stakeholder needs and respondent burden. The data collected in the ORS are established by SSA and reflect input provided throughout development from multiple stakeholders. It is important to note that a single survey cannot capture an exhaustive list of the dimensions that comprise an occupation. The data collected through the ORS is intended to meet the needs of the Social Security disability adjudication process. However, the ORS design allows it to adapt to the changes in the U.S. economy and to evolving needs for information on the nature of work. For example, the Standard Occupational Classification (SOC) system, which serves as the basis for classifying workers in the ORS, is regularly updated to reflect changes in technology and the occupational composition of the workforce. Similarly, the ORS sample will be regularly refreshed, allowing for improvements in definitions and materials based on survey experience and changes in stakeholder demands.
Two of the commenters expressed concerns with the ORS current sample design. One commenter felt that aggregating 12,761 unique occupations included in the Dictionary of Occupational Titles (DOT) down to 820 SOC codes would not provide useful information due to the variability of Specific Vocational Preparation (SVP) and Strength within specific DOT occupations.
The ORS will classify occupations using the SOC as federal agencies gathering occupational data are mandated to do. Data for the ORS will be coded at an eight digit SOC level defined by the O*NET (O*NET-SOC). As such, we will be coding data to approximately 1,100 individual occupations, not just the 820 SOC codes. Although the ORS samples will be selected using sampling strata defined by North American Industry Classification System (NAICS) economic sector codes (first two digits), all establishments in the sample will be coded into industries using the full NAICS national industry codes (six digits). BLS plans to analyze variability within a given O*NET-SOC code using as many digits of the NAICS codes as possible with the given sample size. Should this analysis show that variability within an O*NET-SOC code can be better explained by generating estimates for an occupation at the more detailed occupation by industry level, BLS will release the data by both occupation and industry whenever sufficient data are present to meet our publication criteria. Decisions about changes to the sample will depend on detailed analysis of the responses from the first several samples as well as the need to balance the cost of the survey with respondent burden.
Additionally, SSA performed a folder study of 5,000 claim folders decided at steps 4 and 5 at the initial and hearings level to inform its Occupational Information System (OIS) development. SSA associated the 15 year work history of each claimant with DOT codes, and found that they were associated with a small percentage – about 10 percent – of the DOT’s titles. SSA and BLS have worked together closely to ensure that the occupational data collected meets the needs of SSA’s disability program. SSA will ensure that the final OIS contains descriptions of a sufficient number of occupations that are common to the disability applicant population. If necessary, SSA and BLS will develop a strategy to disaggregate O*NET-SOC codes for which the data are too spread, by industry or sub-SOC groupings, and to describe occupations which are not captured through sampling.
Two commenters suggested that obtaining NAICS information at the “5-digit level”, as opposed to the “2-digit level”, would “permit a direct link to critical labor market data for numbers of people employed nationally” in targeted industries.
Although the ORS samples will be selected using sampling strata defined by NAICS economic sector codes, all establishments in the sample will be coded into industries using the full NAICS national industry codes (six digits). BLS will release the data by both occupation and industry whenever sufficient data are present to meet our publication criteria.
One commenter felt that due to the wide variety of work history (per DOT coding) of SSA claimants, coding work history at the SOC or O*NET level will often overestimate an individual’s SVP unless it could be “further narrowed to the industry in which the prior job was performed.”
Examination of the data available from the ORS at the occupation level is a topic under continued evaluation. The education and training data collected reflect an establishment’s requirements, not those of individual employees within the occupation. The level of detail published on an occupation or industry basis will be determined by confidentiality and other publication criteria. BLS plans to analyze SVP variability within a given O*NET-SOC code using as many digits of the NAICS codes as possible with the given sample size. Should this analysis show that variability within an O*NET-SOC code can be better explained by generating estimates for an occupation at the more detailed occupation by industry level, BLS will release SVP data by both occupation and industry whenever sufficient data are present to meet our publication criteria.
Two commenters issued concerns on whether Temporary Employment Agencies are “included in the mix of industries surveyed by the NCS.”
The ORS sample includes all economic sectors except agriculture, private households, and the U.S. government. Temporary help services and their employees are represented in the sample.
One commenter posited the ORS would increase efficiency and obtain a “nicely stratified sample of occupations” by using the Occupational Employment Survey’s (OES) survey design as opposed to the current NCS platform.
NCS uses a probability sampling method of selecting occupations within each company in proportion to the number of workers within each occupation. This method, called a Probability Selection of Occupations, has been shown to yield data for a broad range of occupations within each industry and to ensure that all occupations within an industry, even those with low employment, have a chance of being included in the data being collected. While it may be possible to meet these goals using an approach proposed by the commenter, further research needs to be done to explore and test this approach. That research is included in future testing plans but has not been started to date.
Three commenters felt the quality of the data obtained from Human Resource personnel and business owners, without any direct observation or empirical measurement, may be compromised. They stressed the necessity of direct job observation to validate the data provided by typical NCS respondents. It was also suggested that field economists utilize push-pull meters to measure weights lifted or force exerted and that noise levels be measured with decibel meters.
BLS collects information from occupational experts at the companies in the survey. This can include human resources professionals, supervisors, managers, owners, health and safety professionals and other company officials familiar with the selected occupation. Field economists screen potential respondents for an understanding of the surveyed jobs. BLS is beginning to assess the validity of the information collected by interviews with company officials. As part of ORS validity tests, field economists will directly observe previously selected jobs and use meters to measure temperature, humidity and noise levels. The BLS Office of Survey Methods Research (OSMR) will provide guidance for observation training. BLS will continue to evaluate our procedures and survey outputs for validity and reliability as producing accurate and reliable statistics is central to our mission.
One commenter expressed concern regarding the level of training for those collecting data for the ORS. The commenter felt that without consistent training and process knowledge, the responses provided during data collection could be scattered or unreliable.
BLS Field Economists are trained in interview methodology to obtain information from establishments on a voluntary survey basis, how to perceive the attentiveness of the response to the question, and how to evaluate the data collected to determine when additional information is needed or conflicting information is reported. In respect to the ORS collection, staff are provided classroom conceptual training and on-the-job mentoring on the very specific concepts collected. Senior field economists have been collecting data throughout ORS development and have a sound understanding of the data elements. In addition to training, BLS uses a quality assurance program to ensure program consistency. Calibration activities are conducted to improve consistency and reliability across field economists. Work done by all staff is reviewed for consistency of collection. BLS also randomly re-contacts establishments and a second field economist verifies selected data.
All commenters provided feedback on the existing ORS data elements and some commenters suggested further elements should be included into the ORS. Comments were received regarding the Cognitive Elements, Physical Demands, and Environmental Conditions and are summarized in that order in the following sections.
Commenters felt the cognitive questions listed in the 60-day Federal Register Notice did not adequately address job demands and typical work situations in which a worker must be comfortable performing. Another concern was the cognitive elements combined too many discrete constructs.
Commenters felt that the ORS does not currently collect data on typical work situations a worker would typically encounter, such as: exercising judgment to make decisions or recommend courses of action; influencing or directing other people; ability to maintain composure; ability or need to work alone; working under specific instructions.
Commenters suggested that the ORS should collect data on memory, reasoning, concentration, pace, persistence, production rate, time permitted for the worker to be "off task," decision-making, judgment, permissible absences, and frequency of rest or break periods.
One commenter also felt the ORS does not do an adequate job of capturing job demands that relates to language/communication skills and mathematical skills that contribute to the rating of overall task complexity.
The mental cognitive data elements were developed based on disability policy, stakeholder input, the types of limitations most commonly assessed at the initial and hearings level as shown in a claim folder study, and the BLS’s experience leveling jobs for the NCS. In FY 2014, and continuing into FY 2015, BLS has tested these elements and worked with SSA to improve them. The ORS currently captures information related to the frequency and type of communications required. For ORS production collection, questions on work pace were added and the question on task complexity was refined to focus on decision-making. Furthermore, the work controls question was revised to focus on the instruction and review of work provided to the occupation. The revised questions are included on ORS Form 4 (Private and Government). The ORS cannot capture all occupational requirements, but attempts to collect those most important to the SSA disability program. SSA’s new OIS will not eliminate the need for adjudicative judgment or vocational expert input in certain cases. SSA has discussed the elements with the National Institute of Mental Health and the National Institute of Occupational Safety and Health. If SSA and BLS are not confident that the mental-cognitive questions are accurately and reliably measuring the type of occupational information most important to disability adjudication, they will continue to refine these elements.
The DOT defines aptitudes as specific abilities required of an individual to perform a given work activity. SSA does not use aptitudes and temperaments in the disability program because they represent personal interests, natural abilities, and personality characteristics of job incumbents rather than limitations or restrictions resulting from medically determinable impairments. Many of the suggested cognitive elements are aptitudes and beyond the scope of the survey defined by SSA.
Three commenters felt some of the physical demands could be improved by collecting a finer level of detail, such as: distinguishing between time spent standing versus walking, climbing stairs versus ramps, and “duration for the one versus both factors”. Commenters also suggested the following factors should be included: trunk rotation, depth perception, color vision, balancing, touching/feeling, tasting/smelling, and visual accommodation, measure of agility.
The survey must balance the SSA disability program need for the occupational data with the cost of collection and respondent burden. The definitions used are established by SSA to support their disability adjudication needs, and reflect earlier input on SSA’s Occupational Information System project from multiple stakeholders. The ORS measures the elements that are most critical to the disability adjudication, impact the largest number of cases, and where other data sources are insufficient for the program needs.
A commenter suggested revisions to Lifting/Carrying based on the definitions used by the American Conference of Government and Industrial Hygienists (ACGIH).
The ORS definitions are based on the DOT approach. In 2015, BLS investigated the ACGIH definitions and determined that these could not be applied without significant modification. BLS also concluded a revised approach would not reduce respondent burden.
One commenter felt that a question should be included regarding required out-of-town travel as the handling of personal luggage typically dictates the need for lifting and/or carrying an object.
The addition of this element was considered by BLS and SSA. The weight and duration of lifting personal luggage is not captured as this is not controlled by the employer. Whether claimants need to travel out of town is not of high importance to the large proportion of disability applicants. If a case presents where a claimant’s ability to travel overnight for work needs to be considered, an adjudicator will need to rely on the claimant’s job description and adjudicative judgment.
A revision to the definition of Overhead Reaching was sought by multiple commenters who felt that the current degree range of 150 to 180 degrees was too narrow.
This definition of Overhead Reaching was drafted during earlier development when multiple reaching dimensions were collected. BLS and SSA considered how this range compares with what respondents naturally consider for “overhead” reaching and have revised the threshold to be implemented in ORS production collection: Overhead Reaching is now present if the hand goes higher than the head.
One commenter suggested that the following revisions should be made to the environmental conditions currently collected by the ORS: rather than collect Noise Intensity Level, the ORS should collect if an occupation is exposed to high noise that requires hearing protection; if an occupation handles chemicals that require barrier protection as opposed to exposure to toxic, caustic chemicals; exposure to an inhalant that requires respiratory protection rather than exposure to dusts, fumes, and gases; and the addition of exposure to blood borne pathogens. Similarly, a commenter suggested that the ORS collect the length of time a worker may have to wear respiratory protection to guard against exposure to toxic chemicals.
In most cases, the use of protective equipment is mandated by law. Protective equipment may also be provided by employers in cases where it is not mandated. For SSA’s disability adjudication purposes, it is important to know the level of exposure to the chemicals or other environmental conditions despite the use of protective measures. The ORS procedures and forms have been updated to directly capture the presence of any personal protective equipment where appropriate. The environmental condition duration and noise level will continue to be collected based on reported exposure with the use of company required protective equipment.
One commenter suggested the application of set thresholds to the concepts of humidity and noise level.
As part of ORS validity tests, field economists will directly observe previously selected jobs and use meters to measure humidity and noise levels. These measurements will be compared to the data provided by respondents for these elements to determine how respondents perceive various decibel and humidity levels.
One commenter suggested that “Fumes, Noxious Odors, Dust, and Gases” and “Toxic, Caustic Chemicals” are difficult to distinguish from each other and may be difficult to answer as separate components. The commenter stated these terms denote hazards but do not imply a different degree of concern that justifies different categories. The recommendation was to collect the two elements as one item, “Hazardous Contaminants.” The commenter also suggested that the ORS should additionally collect “routes of exposure (e.g., inhalation, eyes, skin, etc.)” and mitigation.
BLS and SSA determined that combining the elements “Fumes, Noxious Odors, Dust, and Gases” and “Toxic, Caustic Chemicals” will decrease confusion by respondents and continue to meet the needs of the Social Security disability adjudication process. The ORS will now collect exposure to “Contaminants” as experienced. The presence of personal protective equipment in the incidence of Contaminants will be collected.
One commenter expressed several concerns about how work schedules are collected and handled within the ORS. In addition, the commenter raised concerns about individualized work schedules and overtime hours.
For occupations with individual employee schedule variability, the ORS collects the average work schedule excluding any scheduled overtime. This information is used in conjunction with durations reported for the ORS data elements. Overtime, minimum and maximum daily hours worked by individual employees are not collected. Employees in the same occupation where shift length varies by production area would be collected as separate jobs.
Suggestions to improve the collection instrument were submitted by one commenter. The commenter felt that grouping the elements the Sitting/Standing/Walking factors with the Postural Elements would improve collection. The commenter also suggested moving the Driving element from its current placement under Auditory/Vision.
BLS considered revisions to the collection instrument. Driving has been relocated to the Task List section. Revisions can be viewed on ORS Form 4 (Private and Government).
One commenter requested clarification on the purpose of the eight columns under the Environmental Conditions section.
The intent of the eight columns is for a multi-quote collection approach in which data for each quote is entered in a column. Field economist collection experience has indicated that the column layout with ample documentation space is the most efficient for collection. The form has been modified to make the intent of the columns clearer.
Two commenters expressed concerns that the ORS data are not sufficient to meet the needs of the SSA disability adjudication process, particularly steps 4 and 5. Their concerns focused on data elements that are not included, challenges in switching to SOC level data from the title based DOT classifications, and concerns about reliability and validity to withstand legal challenges.
SSA worked with BLS to develop the physical, mental, environmental, and vocational preparation data elements that BLS is collecting. These elements were based on what we currently use in the DOT, disability policy, the physical and mental residual functional capacity forms, and the results of a case folder study that showed the most commonly assessed functional limitations in step 4 and 5 decisions and determinations. Since these elements were based on these factors that are relevant to SSA’s needs, they will easily be implementable in adjudication. SSA plans to conduct several tests and assessments of the new data prior to implementing its new Occupational Information System (OIS).
A main area of concern was not collecting information in the ORS that summarizes primary activities and areas of specialization. Specifically, the lack of DOT Work Fields and Materials, Products, Subject Matter, and Services (MPSMS) codes was raised as a fundamental change that would weaken the ability of SSA to perform transferable skills analysis for older workers. Similarly there were concerns that the ORS data collection does not discretely address factors needed by SSA with regard to mental/cognitive impairments. The comment listed very specific characteristics used by SSA in making a disability determination and that should be individual constructs in ORS.
SSA plans to incorporate the tools and technology information from O*NET into the OIS system. SSA believes that ONET’s tools and technology information will serve a purpose similar to the DOT’s MPSMS and Work Fields data. Utilizing the O*NET information instead of including this information directly in ORS will prevent duplication of data collection efforts.
SSA does not use specific characteristics in assessing a claimant’s mental residual functional capacity (MRFC). The checkboxes on the MRFC form do not represent the MRFC. The MRFC is the unstructured medical consultant’s text summary. When SSA developed the mental-cognitive data elements, the Disability Evaluation Constructs (DEC) was our starting point. The DEC contains a list of all of the possible elements that may be useful to disability adjudication that were gathered from external experts and public comment from the Occupational Information Development Advisory Panel. SSA also reviewed the results of an internal study that gathered information from 5,000 disability claims decided at the initial and hearings level at steps 4 and 5 of sequential evaluation. SSA used the information on mental limitations from disability claims folders to examine the DEC and develop a manageable set of data elements that describe the mental demands of work, with a focus on the demands of low skilled work. SSA also leveraged the expertise of BLS regarding the NCS leveling procedures when it was necessary to make refinements and adjustments to the mental-cognitive questions based on the results of testing.
In compliance with SSA policy, many elements do not need to be captured discretely. SSA and BLS have worked diligently to capture elements in a way that meets SSA programmatic needs, and will continue to revise and refine the data elements based on collection results.
Another area of concern was that the collection of information should primarily focus on unskilled positions that are sedentary or have light physical demands, and develop information on employer accommodations for these occupations.
SSA requires occupational information at steps 4 and 5 of sequential evaluation. At step 4, adjudicators must consider whether claimants can perform past work as described or as it is generally performed in the national economy. Because of this obligation to consider whether claimants can perform past work as “generally performed,” SSA must have an OIS that describes a wide-range of occupations in the national economy at all skill and exertion levels. The requirement to consider transferability of skills at step 5 also obligates SSA to obtain current data about higher skilled occupations. To focus on the types of occupations that are most important to our disability applicants, SSA conducted the Occupational, Medical-Vocational claims review study to identify primary occupational, functional, and vocational characteristics of DI and SSI adult applicants whose claims were decided at the initial or hearing levels at steps four or five of SSA’s sequential evaluation process. The results of this study will help SSA and BLS ensure that the types of occupations most common to disability applicants are included in our OIS.
Regarding accommodations, currently under SSA policy, SSA does not consider accommodations when determining whether claimants can work. In accordance with SSA policy, ORS collection is based on an establishment’s requirements for an occupation as the work is performed without accommodation. Not all employers can offer the same accommodations and work performed with an accommodation is not reflective of how a job is generally performed.
One commenter expressed concern regarding how the information is validated to benefit state or federal disability prevention and management programs. The commenter expressed the view that ORS should be collecting information that is relevant to job development and placement services, which is also a component SSA’s Ticket to Work and other disability programs.
The ORS data elements are based on SSA’s policy requirements and disability adjudication needs. The Department of Labor’s O*NET is designed specifically for career exploration and job placement. BLS is exploring options for data dissemination to ensure the data is accessible to the public for a variety of uses.
In addition, two commenters raised questions about the quality and validity of the data from the ORS. Specifically, one commenter stated “The quality, utility and clarity of information collected must be sufficient to withstand a Daubert challenge if it is to be incorporated into the disability determination process. Occupational definitions must be constructed based on sound methodology in order to assure a high level of validity when there is controversy.”
The Bureau of Labor Statistics is a federal principal statistical agency and embraces professional and operational standards designed to ensure the quality, integrity, and credibility of its outputs. Research to identify appropriate approaches for validating ORS elements began in FY 2014 and is on-going. Current research initiatives include the development of a methodological guide, evaluation of benchmarks for data collection, and future testing of inter-rater reliability. This work, as well as the previous refinements of the collection procedures, the data review process, and the validation techniques developed to date, will ensure the ORS produces quality occupational data.
Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), provides the standard for admitting expert scientific testimony in a federal trial. The Daubert standard does not apply in Social Security Disability hearings. In Richardson v. Perales, 402 U.S. 389 (1971), the Supreme Court considered whether the requirements of expert testimony as outlined in the Federal Rule of Evidence Rule 702 applied to experts testifying at Social Security disability hearings, and determined that Rule 702 and its requirements do not govern the admissibility of evidence in disability hearings. Furthermore, under 42 U.S.C. § 405(b)(1) and 20 C.F.R. §§ 404.950(c), 416.1450(c) it is clear that Rule 702, and thus Daubert’s interpretation of Rule 702 criteria, does not apply to the admission of evidence in Social Security disability hearings. The reason for this approach is described by the Richardson Court: the “strict rules of evidence, applicable in the courtroom, are not to operate at Social Security hearings so as to bar the admission of evidence otherwise pertinent[.]” Richardson, 402 U.S. at 400. The Richardson Court held that evidence that would be inadmissible in a court proceeding could nonetheless constitute substantial evidence supporting a Social Security disability determination. Id. The Court reasoned that with regard to Social Security disability proceedings, “[t]here emerges an emphasis upon the informal rather than the formal.” Id. The Richardson Court emphasized that the Social Security disability proceeding should be comprehensible to a layman claimant and that the proceeding “should be liberal and not strict in tone and operation.” Id. at 400-401. This informal evidentiary approach continued as Daubert's interpretation of Rule 702 and Daubert criteria was used in traditional courtrooms and federal agency hearings.
Even though SSA does not believe that Rule 702 and Daubert apply here, as explained in the previous paragraph, SSA and BLS have taken and are continuing to take measures to ensure the ORS produces quality occupational data that are scientifically valid, which includes the consideration of factors, such as those identified by the Daubert Court, that might be helpful in assessing the reliability of a particular scientific theory or technique.
BLS staff engaged in extensive consultation with staff of the Social Security Administration’s Research, Demonstration, and Employment Support and Office of Disability Policy on all aspects of the survey.
BLS staff consulted with staff of the Department of Labor’s Employment and Training Administration (ETA) to learn more about the Occupational Information Network (O*NET) system.
BLS staff presented papers on the proposed survey design, estimation, and data validation at the 2014 Joint Statistical Meetings (JSM).
BLS staff delivered public presentations on the proposed survey design at the following conferences:
2013 Joint Statistical Meetings (JSM)
2013 Federal Committee on Statistical Methodology Research Conference
BLS Data User’s Advisory Committee (DUAC) meeting in April 2013
Overview Presentation to OMB employees in September 2013
BLS staff attended and sometimes presented at conferences sponsored by the following organizations to keep them informed about the project and to learn more about the needs of the stakeholder community and possible uses of ORS data by them:
National Association of Disability Examiners (NADE)
International Association of Rehabilitation Professionals (IARP)
American Board of Vocational Experts (ABVE)
National Association of Disability Representatives (NADR)
National Organization of Social Security Claimants’ Representatives (NOSSCR)
BLS consulted with an outside contractor to review methodological issues and previous research to ensure that data collected for the ORS meets the needs of SSA and various stakeholders as well as to determine if the occupational requirements measured in ORS are reliable, valid, and accurate. Recommendations from this research will be implemented during the survey. An executive summary can be viewed on the BLS web site www.bls.gov/ors.
No payments or gifts will be provided to any respondents.
The Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA) safeguards the confidentiality of individually identifiable information acquired under a pledge of confidentiality for exclusively statistical purposes by controlling access to, and uses made of, such information. CIPSEA includes fines and penalties for any knowing and willful disclosure of individually identifiable information by an officer, employee, or agent of the BLS.
Based on this law, the BLS provides all non-government respondents with the following confidentiality pledge/informed consent statement:
The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
This statement appears on the private industry collection forms.
For the ORS program, the pledge of confidentiality is not extended to State and local government entities, unless specifically requested.
BLS policy on the confidential nature of respondent identifiable information (RII) states, “RII acquired or maintained by the BLS for exclusively statistical purposes and under a pledge of confidentiality shall be treated in a manner that ensures the information will be used only for statistical purposes and will be accessible only to authorized individuals with a need-to-know.”
Aside from the sensitivity attached to position description and working conditions, no sensitive questions are asked during the survey.
Estimates of respondent burden are provided in this section for all activities associated with the ORS program. For the purposes of the discussion of respondent burden and BLS cost, the ORS is an initiation-only survey. Respondents will not be asked to provide data more than once every three years for a sampled establishment, as described in Part B, Section 1a. For an ORS establishment that is also a sample establishment in the NCS survey, the collection of establishment information, work leveling, and work schedule and the associated time are covered and charged under the OMB Clearance for the National Compensation Survey (OMB Control Number 1220-0164).
Broadly stated, both private industry and State/local government establishments in the ORS collection fall into the following three categories below:
Activity (1a) - Initiation of establishments in the ORS sample where general establishment information, work levels, work schedules, and job requirements are collected.
Activity (1b) - Initiation of establishments in the ORS sample which are in the current NCS sample where ORS only collects the job requirements.
Activity (2) - Re-interview for quality assurance activities of ORS job requirements for initiations. Approximately five percent of the sampled establishments will be re-contacted to confirm the accuracy of coding for selected data elements
Activity (3) – Additional testing of occupational requirement collection.
Estimates of net respondent burden associated with these collection activities in FY 2015, FY 2016, FY 2017, and FY 2018 are broken out by affected sectors (private sector, State and local governments) and provided on the following pages. Collection of each sample occurs over multiple fiscal years; therefore, the total number of units collected during each fiscal year reported in the tables below may differ from the stated sample numbers provided in Part B, Section 1a.
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
321 |
1 |
321 |
120 |
642 |
Activity (1b) Initiation of ORS NCS overlap establishments |
211 |
1 |
211 |
66 |
232 |
Activity (2) Re-interview for quality assurance activities |
27 |
1 |
27 |
15 |
7 |
Activity (3) Testing of occupational requirements collection. |
0 |
1 |
0 |
60 |
0 |
FY 2015 TOTALS |
559 |
|
559 |
|
881 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
5,940 |
1 |
5,940 |
120 |
11,880 |
Activity (1b) Initiation of ORS NCS overlap establishments |
2,225 |
1 |
2,225 |
66 |
2,448 |
Activity (2) Re-interview for quality assurance activities |
408 |
1 |
408 |
15 |
102 |
Activity (3) Testing of occupational requirements collection. |
300 |
1 |
300 |
60 |
300 |
FY 2016 TOTALS |
8,873 |
|
8,873 |
|
14,730 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
8,185 |
1 |
8,185 |
120 |
16,370 |
Activity (1b) Initiation of ORS NCS overlap establishments |
310 |
1 |
310 |
66 |
341 |
Activity (2) Re-interview for quality assurance activities |
425 |
1 |
425 |
15 |
106 |
Activity (3) Testing of occupational requirements collection. |
300 |
1 |
300 |
60 |
300 |
FY 2017 TOTALS |
9,220 |
|
9,220 |
|
17,117 |
Note: The sum of individual items may not equal totals due to rounding.
|
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
7,696 |
1 |
7,696 |
120 |
15,392 |
Activity (1b) Initiation of ORS NCS overlap establishments |
239 |
1 |
239 |
66 |
263 |
Activity (2) Re-interview for quality assurance activities |
397 |
1 |
397 |
15 |
99 |
Activity (3) Testing of occupational requirements collection. |
300 |
1 |
300 |
60 |
300 |
FY 2018 TOTALS |
8,632 |
|
8,632 |
|
16,054 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
47 |
1 |
47 |
120 |
94 |
Activity (1b) Initiation of ORS NCS overlap establishments |
10 |
1 |
10 |
66 |
11 |
Activity (2) Re-interview for quality assurance activities |
3 |
1 |
3 |
15 |
1 |
Activity (3) Testing of occupational requirements collection. |
0 |
1 |
0 |
60 |
0 |
FY 2015 TOTALS |
60 |
|
60 |
|
106 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
941 |
1 |
941 |
120 |
1,882 |
Activity (1b) Initiation of ORS NCS overlap establishments |
135 |
1 |
135 |
66 |
149 |
Activity (2) Re-interview for quality assurance activities |
54 |
1 |
54 |
15 |
13 |
Activity (3) Testing of occupational requirements collection. |
50 |
1 |
50 |
60 |
50 |
FY 2016 TOTALS |
1,180 |
|
1,180 |
|
2,094 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
1,399 |
1 |
1,399 |
120 |
2,798 |
Activity (1b) Initiation of ORS NCS overlap establishments |
101 |
1 |
101 |
66 |
111 |
Activity (2) Re-interview for quality assurance activities |
75 |
1 |
75 |
15 |
19 |
Activity (3) Testing of occupational requirements collection. |
50 |
1 |
50 |
60 |
50 |
FY 2017 TOTALS |
1,625 |
|
1,625 |
|
2,978 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
1,306 |
1 |
1,306 |
120 |
2,612 |
Activity (1b) Initiation of ORS NCS overlap establishments |
95 |
1 |
95 |
66 |
105 |
Activity (2) Re-interview for quality assurance activities |
70 |
1 |
70 |
15 |
18 |
Activity (3) Testing of occupational requirements collection. |
50 |
1 |
50 |
60 |
50 |
FY 2018 TOTALS |
1,521 |
|
1,521 |
|
2,785 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
The table below summarizes the data, including figures on the actual number of respondents to be contacted each fiscal year.
Fiscal Year |
Respondents |
Total # of Responses* |
Average responses per year |
Average minutes per response |
Total hours |
FY 2015 |
532 |
559 |
1.05 |
95 |
881 |
FY 2016 |
8,465 |
8,873 |
1.05 |
100 |
14,730 |
FY 2017 |
8,795 |
9,220 |
1.05 |
111 |
17,117 |
FY 2018 |
8,235 |
8,632 |
1.05 |
112 |
16,054 |
Overall average |
8,676 |
9,095 |
1 |
104 |
16,261 |
*Initiations, quality assurance contacts, and further testing |
|||||
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Fiscal Year |
Respondents |
Total # of Responses* |
Average responses per year |
Average minutes per response |
Total hours |
FY 2015 |
57 |
60 |
1.05 |
106 |
106 |
FY 2016 |
1,126 |
1,180 |
1.05 |
106 |
2,094 |
FY 2017 |
1,550 |
1,625 |
1.05 |
110 |
2,978 |
FY 2018 |
1,451 |
1,521 |
1.05 |
110 |
2,785 |
Overall average |
1,395 |
1,462 |
1.05 |
108 |
2,654 |
*Initiations, quality assurance contacts, and collection testing |
|||||
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
368 |
1 |
368 |
120 |
736 |
Activity (1b) Initiation of ORS NCS overlap establishments |
221 |
1 |
221 |
66 |
243 |
Activity (2) Re-interview for quality assurance activities |
30 |
1 |
30 |
15 |
8 |
Activity (3) Testing of occupational requirements collection. |
0 |
1 |
0 |
60 |
0 |
FY 2015 TOTALS |
619 |
|
619 |
|
987 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
6,881 |
1 |
6,881 |
120 |
13,762 |
Activity (1b) Initiation of ORS NCS overlap establishments |
2,360 |
1 |
2,360 |
66 |
2,597 |
Activity (2) Re-interview for quality assurance activities |
462 |
1 |
462 |
15 |
115 |
Activity (3) Testing of occupational requirements collection. |
350 |
1 |
350 |
60 |
350 |
FY 2016 TOTALS |
10,053 |
|
10,053 |
|
16,824 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
9,584 |
1 |
9,584 |
120 |
19,168 |
Activity (1b) Initiation of ORS NCS overlap establishments |
411 |
1 |
411 |
66 |
452 |
Activity (2) Re-interview for quality assurance activities |
500 |
1 |
500 |
15 |
125 |
Activity (3) Testing of occupational requirements collection. |
350 |
1 |
350 |
60 |
350 |
FY 2017 TOTALS |
10,845 |
|
10,845 |
|
20,095 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
Collection Activity |
Number of Respondents Per Activity (Net) |
Responses per Respondent |
Total Annual Responses by Activity |
Minutes per Response |
Total Hours |
Activity (1a) Initiation of ORS |
9,002 |
1 |
9,002 |
120 |
18,004 |
Activity (1b) Initiation of ORS NCS overlap establishments |
334 |
1 |
334 |
66 |
368 |
Activity (2) Re-interview for quality assurance activities |
467 |
1 |
467 |
15 |
117 |
Activity (3) Testing of occupational requirements collection. |
350 |
1 |
350 |
60 |
350 |
FY 2018 TOTALS |
10,153 |
|
10,153 |
|
18,839 |
Note: The sum of individual items may not equal totals due to rounding. |
|||||
The table below summarizes the data, including figures on the actual number of respondents to be contacted each year.
Fiscal Year |
Respondents |
Total # of Responses* |
Average responses per year |
Average minutes per response |
Total hours |
FY 2015 |
589 |
618 |
1 |
96 |
987 |
FY 2016 |
9,591 |
10,053 |
1 |
100 |
16,824 |
FY 2017 |
10,345 |
10,845 |
1 |
111 |
20,095 |
FY 2018 |
9,686 |
10,153 |
1 |
111 |
18,839 |
*Initiations, quality assurance contacts,
and collection testing |
|||||
Overview of ORS collection forms
These forms are primarily used as note-taking devices by the field economists (BLS staff). The field economists ask probing questions that will vary depending on the knowledge level of the respondent. The forms provide the field economist with a list of the information required for the survey, not a list of all questions asked. For quality assurance re-interviews, the field economists will ask for specific items of data in a prescribed manner from data stored in the electronic database. ORS considers the establishment data in the electronic databases the official copy of the establishment data for survey purposes.
Form |
Function |
Activities used |
Time |
Establishment, work level, and schedule
collection form |
Government general establishment information, work level of occupation, and work schedule; records check of these data |
(1a) ORS initiation |
54 minutes |
(2) Quality assurance checks job requirements initiation |
5 minutes |
||
Establishment, work level, and schedule
collection form |
Private industry general establishment information, work level of occupation, and work schedule; records check of these data |
(1a) ORS initiation |
54 minutes |
(2) Quality assurance checks job requirements initiation |
5 minutes |
||
Occupation requirements (ORS Form 4 PPD-4G) |
Government initiation collection of vocational preparation, cognitive elements, physical demands, environmental conditions, and job tasks; records check of this collection |
(1a) ORS initiation |
66 minutes |
(1b) ORS initiation with NCS |
66 minutes |
||
(2) Quality assurance checks job requirements initiation |
10 minutes |
||
Occupation requirements (ORS Form 4 PPD-4P) |
Private industry initiation collection of vocational preparation, cognitive elements, physical demands, environmental conditions, and job tasks; records check of this collection |
(1a) ORS initiation |
66 minutes |
(1b) ORS initiation with NCS |
66 minutes |
||
(2) Quality assurance checks job requirements initiation |
10 minutes |
Form |
Total Respondents Per Form |
Frequency |
Total Annual Responses* |
Minutes for the Predominant Form Use |
Total Hours* |
Establishment, work level, and schedule
collection form |
1,231 |
1 |
1,231 |
54 |
1,108 |
Establishment, work level, and schedule
collection form |
7,381 |
1 |
7,381 |
54 |
6,643 |
Occupation requirements |
1,345 |
1 |
1,345 |
66 |
1,479 |
Occupation requirements |
8,376 |
1 |
8,376 |
66 |
9,213 |
Collection not tied to a specific form (Quality Assurance, Testing) |
836 |
1 |
836 |
- |
472 |
TOTALS |
19,168 |
|
|
18,915 |
|
*The sum of individual items may not equal totals due to rounding. |
|||||
Individual respondent cost per year (for all responses) is expected to be an average of $67.41 for FY 2015, $70.22 for FY 2016, $77.94 for FY 2017, and $77.94 for FY 2018. This amount is based on an average cost of $42.13 per hour per respondent. The estimate, based on previous NCS collection and ORS testing, is that 70 percent of reporting time comes from professional and related workers, and the remaining 30 percent comes from office and administrative support workers. Professional and related specialty earned an average of $49.58 per hour in total compensation; office and administrative support workers earned an average of $24.76 per hour in total compensation. (Hourly costs of pay and benefits measured by the Employer Cost for Employee Compensation data series for Civilian workers in September 2014.) The figure of $42.13 is a weighted hourly average.
Estimated annualized cost to all respondents for all activities is $41,582.31 in FY 2015, $708,795.12 in FY 2016, $846,602.35 in FY 2017, and $793,687.07 in FY 2018. These totals are based on an average hourly cost of $42.13 to the respondent.
There are no capital and start-up costs or operation and maintenance and purchase of service costs resulting from the collection of this information.
The ORS survey is part of the Interagency Agreements between BLS and SSA. The cost of the FY 2015 Agreement is around $19 million.
The ORS survey is a new collection.
The ORS data collected are to be published annually, as an ongoing annual survey. This information will be made available on the www.bls.gov website.
Approval to not display the expiration date for OMB approval is not being sought.
There are no exceptions to the certification statement.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Supporting Statement |
| Subject | 2015 ORS version |
| Author | Paul Carney |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |