Note to Reviewer - Job Observation Test

September 19, 2016

MEMORANDUM FOR: REVIEWER of 1220-0189

FROM: Jennifer Coleman

Division Chief, National Compensation Survey

Office of Compensation and Working Conditions

Bureau of Labor Statistics

SUBJECT: Non-substantive Change Request for the

Occupational Requirements Survey (ORS) –

Job Observation Test

Clearance is being sought by the Bureau of Labor Statistics’ (BLS) Occupational Requirements Survey (ORS) program for further testing of occupational requirements collection by job observations. The goal of this job observation test is to compare data obtained from observation of a selected occupation at an establishment with data obtained previously for the same occupation by interviewing a representative of that establishment.

An additional 1,771 respondent burden hours are being requested to do this job observation test of 1,250 occupations. For private industry 1,063 occupations are to be observed and 187 occupations in State and local governments. 

These respondent burden testing hours are beyond those currently approved for ORS testing.  Currently approved ORS testing hours for FY 2017 and FY 2018 are 300 private industry test hours and 50 State and local government test hours annually in the ORS OMB Clearance (1220-0189) approved on 8/26/2015.

Survey collection will span a 12 month period which will start no earlier than January 1, 2017 and end no later than: March 31, 2018.

The 1,771 additional respondent burden hours requested for this test break out into 1,328 hours in FY 2017 and 443 hours in FY 2018.

• FY 2017 additional testing hours requested for private industry are 1,129.

• FY 2017 additional testing hours requested for State and local government are 199.

• FY 2018 additional testing hours requested for private industry are 377.

• FY 2018 additional testing hours requested for State and local government are 66.

The ORS job observation test respondent burden is expected to average 85 minutes per establishment, the major time break outs are 25 minutes to explain and prepare for the observation and 60 minutes of observation time. One field economist will go to each establishment, observe the same occupation previously collected by interview, and independently write up their observations.

Table 1. Respondent burden hours by fiscal years, type of respondents and totals

Group	Respondents	Total # of Responses	Average Minutes per Response	Burden hours
Private 2017 2018	619 444	619 444	85 85	877 629
Total Private	1063	1063	85	1506
State and Local Governments 2017 2018	109 78	109 78	85 85	154 111
Total State and Local Government	187	187	85	265
Test Survey totals	1250	1250	85	1771

This test builds off the strengths of the limited ORS job observation test conducted mid-2015. Data collected will support BLS efforts to further measure the validity of current ORS collection methods and assess the relative practicality of different data collection methods for ORS data.

This ORS Job Observation Test of cognitive, physical and environmental elements is to measure the coding agreement/differences between an observed job and an unobserved job for the tested elements. This test will produce information on the rates of agreement between data collection methods, and will give insight into the availability of data via observation that was not able to be collected via interview. This research can help ORS evaluate concerns expressed by stakeholders about the potential measurement error associated with the collection mode of establishment respondent interviews and assess the value of observational data for imputation.

From our earlier limited ORS job observation test we did find some limitations to job observations. The pace of data collection via observation is slow compared to an interview and imposes a burden on the establishment to accommodate the collector, which may cause a conflict with operational activities. Cooperation can be hard to obtain for observation in a number of industries. Patient privacy concerns in hospital and other medical establishments limit observations. Schools and day care centers also express privacy issues with observations. Safety and insurance concerns limit observation on many occupations.

Another limitation to job observation can be that the observer may not see the occupation doing “typically performed” functions during the observation. The presence of the observer may impact how the workers perform their jobs and not all tasks may be required during the observation. The test will allow BLS to assess the impact of these limitations on the direct observation methodology and to explore means of overcoming them.

For sampling, the frame of occupational observations is stratified by SOC code. The job observation test sample will be taken from 22 groups consisting of 25 8-digit SOC codes (see Attachment D below for list of specific SOC codes). In each of these 22 SOC-defined strata, no more than 60 observations will be completed from each, with a total test size not to exceed 1,250 occupational observations. The occupational observations will be from at most 1,250 establishments, but it is possible that some establishments will have more than one selected occupational observation.

Two collection forms and two collection introduction scripts were developed for this test. Copies of the collection forms, introduction scripts, and list of SOC’s used for this test are included in the attachments.

Form	Function	Activities	Time
ORS 2017 Job Observation Test Government Recording form (ORS Form 4 OJT-1G )	Government general occupation information and ORS elements observed; records check of these data	Observation of selected occupation	85 minutes
ORS 2017 Job Observation Test Private Industry Recording form (ORS Form 4 OJT-1P )	Private industry general occupation information and ORS elements observed; records check of these data	Observation of selected occupation	85 minutes

If you have any questions about this non-substantive change request, please contact Paul Carney at 202-691-5180 or e-mail at [email protected] or Jennifer Coleman at 202-691-7128 or e-mail at [email protected].

Attachment A – Recruitment Script

Attachment B – ORS Observation Test Form – Private Industry

Attachment C – ORS Observation Test Form – State and Local Government

Attachment D – Specific SOC codes tested listing.

4