February 28, 2018
NOTE TO THE REVIEWER OF: |
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
|
FROM: |
Erica Yu Research Psychologist Office of Survey Methods Research
|
SUBJECT: |
Submission of Materials for the Pretesting of proposed revisions to field procedures for the Occupational Requirements Survey |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The total estimated respondent burden hours for this study is 95 hours.
If there are any questions regarding this project, please contact Erica Yu at
202-691-7924.
Introduction and Purpose
The Occupational Requirements Survey (ORS), which gathers job-related information regarding physical demands, environmental conditions, mental and cognitive demands, and vocational preparation requirements, is undergoing revisions to its sampling procedures. These proposed revisions are considered by the Office of Compensation and Working Conditions (OCWC) to be necessary to meet the goals of the survey and the needs of stakeholders.
The revised sampling procedure includes a change to the way that field economists will collect data from respondents. Currently, field economists select which jobs at an establishment to include in the survey by conducting a probability sampling of occupations at the establishment, based on a listing of occupations and the number of employees in each occupation. Under the proposed revisions, field economists will instead be provided with up to 16 target standard occupational classification codes (SOCs) for each establishment. These targeted SOCs are SOCs that have been identified by the ORS statistical methods group as high priority for data collection. For example, SOC number 13-2011 Accountants and Auditors may be one of the targeted SOCs for a particular establishment. A field economist will ask whether the establishment has any jobs matching the target SOCs until she reaches the goal number for successfully matched jobs (4-8 of those targeted SOCs, depending on establishment size) or until she asks about all provided target SOCs (up to 16). The field economist would begin the selection process based on 13-2011 (and the other targeted SOCs) rather than the previous method of conducting a probability sample of occupations. This revision is expected to increase the number of occupations for which a sufficient number of quotes (i.e., enough quotes to pass publication criteria) are collected.
While this revised sampling approach is deemed necessary, field collection issues may arise from the change to data collection procedures to focus on SOC targeting that cannot be addressed via statistical adjustments. These issues will require a clearly defined set of procedures to give field economists guidance on how to handle these issues. A team of field staff, economists, statisticians, and research psychologists has considered these problems and written procedures for anticipated circumstances. However, these procedures should be tested before implementation in production. The purpose of this study is to test these procedures with a range of establishments to refine and improve data collection procedures and field economist training.
Methodology
BLS staff (interviewers) will call establishments that previously participated in ORS. The interviewers will explain that the purpose of the study is to pre-test the proposed data collection procedures for identifying job title/worker characteristic combinations (e.g., an accountant who works full-time and is not part of a union) to match a supplied list of target SOCs. Using the example of SOC number 13-2011 Accountants and Auditors, the establishment may have no positions that match that position, one position that matches, or multiple positions with different job title/worker characteristic combinations. If there are no positions, the match is unsuccessful. If there is one position, that position is selected for ORS collection (limited to most important tasks for the purpose of the test). If there are multiple positions, follow up questions are needed to select a single job title/worker characteristic combination.
In the study, interviewers will conduct an abridged form of the ORS interview, focusing only on job selection. After each job is identified, the interviewer will collect a limited task list to be used to verify the accuracy of the job selection process for matching to targeted SOCs. At the end of the interview, participants will be asked to provide feedback about the data collection process.
In order to secure respondent cooperation, the interviewer will use contact scripts based on the scripts previously used by field economists in prior tests (Attachment A). One script exists for when the individual contacted has previously participated in ORS and a second script exists for when the individual contacted has not previously participated (another person at the establishment completed ORS previously). The primary difference between these scripts is an emphasis on background and introductory information on ORS. Participants who agree to participate will schedule an appointment for an interview or continue on to the interview immediately. Participants who decline to participate will be thanked and not contacted again.
If cooperation is secured, the interviewer will move to the semi-structured interview protocol (Attachment B). Using this protocol, interviewers will aim to identify a single job title/worker characteristic combination to match a targeted SOC. Each company will have 4-16 target SOCs with a collection goal of 4-8 jobs, based on establishment size. The interviewer will repeat the job selection protocol as needed until the goal is reached, up to 16 times. Once the goal is reached, the interviewer will move on to the debriefing.
3. Participants
The frame for the study will be all private industry establishments from ORS production samples 702 and 703 (8,500 establishments in each sample). From this frame, we will exclude establishments that are selected for the ORS quality assurance activities because they will have already been re-contacted by ORS staff. Establishments from the quality assurance activities make up about 5% of the total ORS sample. Additionally the frame will exclude any of the establishments already re-contacted for recent tests such as the Job Observation test and the Cognitive test.
The sample will attempt to target a mix of SOCs, a mix of industries, and a mix of establishment size. The goal is to interview 60 participants and no more than 80 establishments will be contacted. Participants may be communicated with through phone, email, or both.
4. Burden Hours
The total burden hours for this study will be 95.
We may contact up to 80 establishments, with recruitment expected to take up to 15 minutes (80 x 0.25 = 20 hours). We expect to complete up to 60 interviews, with each lasting up to 60 minutes including debriefing (60 x 1.00 = 60 hours). It is expected that participants may spend up to an additional 15 minutes preparing for the interview (60 x 0.25 = 15 hours). The total expected burden for this study is 95 hours.
5. Payment to Participants
No payments or gifts will be provided to any participants.
6. Data Confidentiality
Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national survey. The BLS confidentiality pledge will be provided to participants:
This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
The statement will be provided by email upon request and participants interviewed over the phone will be informed of the confidentiality provisions verbally and asked for their verbal consent.
Attachments
Appendix A: Initial Contact Scripts for Occupational Selection Test Respondents
Appendix B: Interview Scripts for Occupational Selection
Securing Appointment Script: Previous ORS Respondent (sample group 703 or 702)
Hello, my name is [name], and I’m calling from the Bureau of Labor Statistics.
You may recall you were contacted previously by [previous FE] on [approximate date of contact – e.g., last March, earlier this year, etc.] to discuss vocational, physical, environmental, and cognitive demands of a few of your organization’s jobs at XXXX location. We collect this data to help the Social Security Administration make better decisions about workers who are applying for disability. We all may know someone who suffers from a disability and we ourselves may one day need it. Therefore, we want to ensure we have the most up-to-date data on occupations within our national economy.
First, I want to thank [you/your company] for helping us with this important research. Do you mind if a note taker joins us for the call? [Note taker name] will joining as part of [his/her] skill development and will assist me in capturing the information you provide.
I’m contacting you today because we’re testing what we hope is a better way of selecting jobs for the survey. I won’t be asking a lot of questions. I have a list of types of jobs, and I’d like to ask you if any of these exist at your company. If any do, I’d like to briefly talk about the primary responsibilities of the jobs. We’re aiming to identify [4/6/8] distinct jobs at your organization. I would really appreciate your feedback throughout the process, so we can learn how well this approach works and if it’s easy for you to understand.
This call will take no longer than [45/60] minutes, but it might be easier if you have a list of all the jobs at [Sampled Location] to look through while we speak. The list would ideally include job titles as well as any distinctions that your company makes for your workers such as full-versus part-time status. If you’re willing to share the list with me, I might be able to move more quickly through the selection process by looking at it before our discussion and identifying some jobs we may want to talk about. We can stop whenever you need to. Do you have time to help us out now? When would be a better time for me to call you back?
This study is being conducted for internal research only. The BLS will use the information you provide for statistical purposes only and takes rigorous security measures to protect confidential information to the fullest extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The OMB clearance number for this voluntary collection is 1220-0141. BLS could not conduct this survey without a currently valid OMB control number.
Key Points that must be shared with the participant:
Prior participation
Study background, purpose and use (internal research and statistical purposes only)
Establishment-specific details- Location
Appointment duration
Confidentiality Pledge
Voluntary Nature of Survey
Note-taker, if present
OMB Approval for the Study
Securing Appointment Script: New Respondent (who did not participate in ORS702/703 interview)
Hello, my name is [name], and I’m calling from the Bureau of Labor Statistics. I’m calling because your organization was chosen to participate in a study of occupational requirements that we’re conducting on behalf of the Social Security Administration. We’re working to collect data for SSA to use in its disability determination process. Your colleague, [Name of ORS702/703 respondent] assisted us last year with information about several jobs at XXXX location.
I’m contacting you today to ask for your continued assistance and to thank you for your company’s previous commitment to this important study. Do you mind if a note taker joins us for the call? [Note taker name] will joining as part of [his/her] skill development and will assist me in capturing the information you provide.
In our previous contact with [company name], we talked with [ORS702/703 respondent] to select a few jobs at XXXX location and talked about the primary responsibilities of those jobs. We are now testing a new way of selecting jobs for the survey and we need feedback from our respondents, like yourself, to learn whether this new approach is working and is easy to use. In our new approach, I have a list of types of jobs, and I’d like to ask you if any of these exist at your company. If any do, I’d like to briefly talk about the primary responsibilities of the jobs. We’re aiming to identify [4/6/8] distinct jobs at your organization. I would also appreciate your feedback throughout the process and will have a couple of quick questions for you at the end about how we can improve the process for you.
The call will take no longer than [45/60] minutes, but it might be easier if you have a list of all the jobs at [Sampled Location] to look through while we speak. The list would ideally include job titles as well as any distinctions that your company makes for your workers such as full-versus part-time status. If you’re willing to share the list with me, I might be able to move more quickly through the selection process by looking at it before our discussion and identifying some jobs we may want to talk about. We can stop whenever you need to. Do you have time to help us out now? When would be a better time for me to call you back?
The study is being conducted for internal research only. The BLS will use the information you provide for statistical purposes only and takes rigorous security measures to protect confidential information to the fullest extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The OMB clearance number for this voluntary collection is 1220-0141. BLS could not conduct this survey without a currently valid OMB control number.
Key Points that must be shared with the participant:
Overview of ORS
Prior participation
Study background, purpose and use (internal research and statistical purposes only)
Establishment-specific details -Location
Appointment duration
Confidentiality Pledge
Voluntary Nature of Survey
Note-taker, if present
Throughout the interview, the interviewer will ask probing questions when the information received is vague, incomplete or if the respondent’s comments warrant additional probing.
For establishments with OES-reported job titles:
(Optional). Can you provide a list of all jobs at your company? The list would ideally include job titles as well as any distinctions that your company makes for your workers such as full- versus part-time status. If so, that may help speed up our selection process. If not, we can proceed without it.
If job list was already received prior to interview, interviewer would remind participant that the interviewer will be looking at the list and referring to it.
If asked for in initial contact, the interviewer would ask the participant if they were able to get a list and to remind them it may be useful to have in front of them.
Repeat the following series of questions for each supplied target SOC. Once the goal number of targets is reached (4/6/8 based on establishment size), move to debrief questions:
Your company previously reported to the Bureau of Labor Statistics that you have [OES-reported job title] at this company. Do you currently have any employees in that position?
No – move to question 3
Yes – skip to question 6
Do you have anyone in the company who [SOC manual definition of target]?
(For example: 13-2011 Accountants and Auditors manual definition in SOC 2010 “Examine, analyze, and interpret accounting records to prepare financial statements, give advice, or audit and evaluate statements prepared by others.”)
No –repeat question 2 using the next target. If the answer is no for question 2 and question 3 for all targets, go to the job list provided in question 1 to randomly select the goal number (4/6/8 based on establishment size).
Yes - move to question 4
Maybe – Ask probing questions to determine whether company’s job fits within the target.
If no, repeat question 2 using the next target.
If yes, move to question 4.
Do you have more than one job that fits that description?
Yes - move to question 5
No – skip to question 6
Can you list all the job titles that you feel fit into this category?
Does this job title(s) have full time and part time employees?
Yes
No
Are there any other differences in work schedules?
Yes
No
Are there any other distinctions that separate the employees into different categories?
Ask probing questions to obtain enough information to be able to select one job title/worker characteristic combination that is narrowly defined and appropriate for ORS collection The ORS coded characteristics are full-/part-time, union/nonunion, time/incentive, and specific vocational preparation/leveling (which will vary if junior/senior, team lead/non-lead distinctions exist).
Do you separate employees into:
union and nonunion workers Yes No
time based versus incentive based Yes No
junior and senior distinctions Yes No
team lead versus non-lead workers Yes No
How many employees do you have in this position, [selected job title/worker characteristic combination]?
What are the most important tasks this position, [selected job title/worker characteristic combination], does for your organization?
Obtain critical task list, then go back to question 2 using the next target.
For all other establishments:
(Optional). Can you provide a list of all jobs at your company? If so, that may help speed up our selection process. If not, we can proceed without it.
If job list was already received prior to interview, interview would remind participant that they are looking at the list.
If asked for in initial contact, the interviewer would ask the participant if they were able to get a list and to remind them it may be useful to have in front of them.
Repeat the following series of questions for each supplied target SOC. Once the goal number of targets is reached (4/6/8 based on establishment size), move to debrief questions:
Do you have anyone in the company who [SOC manual definition of target]?
(For example: 13-2011 Accountants and Auditors manual definition in SOC 2010 “Examine, analyze, and interpret accounting records to prepare financial statements, give advice, or audit and evaluate statements prepared by others.”)
No –repeat question 2 using the next target. If the answer is no for all targets, go to the job list provided in question 1 to randomly select the goal number (4/6/8 based on establishment size).
Yes - move to question 3
Maybe – Ask probing questions to determine whether company’s job fits within the target.
If no, repeat question 2 using the next target.
If yes, move to question 3.
Do you have more than one job that fits that description?
No – skip to question 5
Yes - move to question 4
Can you list all the job titles that you feel fit into this category?
Do these job titles have full time and part time employees?
Yes
No
Are there any other differences in work schedules?
Yes
No
Are there any other distinctions that separate the employees into different categories?
Ask probing questions to obtain enough information to be able to select one job title/worker characteristic combination that is narrowly defined and appropriate for ORS collection. The ORS coded characteristics are full-/part-time, union/nonunion, time/incentive, and specific vocational preparation/leveling (which will vary if junior/senior, team lead/non-lead distinctions exist).
Do you separate employees into:
union and nonunion workers Yes No
time based versus incentive based Yes No
junior and senior distinctions Yes No
team lead versus non-lead workers Yes No
How many employees do you have in this position, [selected job title/worker characteristic combination]?
What are the most important tasks this position, [selected job title/worker characteristic combination], does for your organization?
Obtain critical task list, then go back to question 2 using the next target.
Debriefing Questions for all respondents:
Would you say the process of identifying jobs for the survey was easy or difficult?
Easy
Difficult - If difficult then ask: Tell me more about what you found difficult
Now that you’ve been through the selection process with me, is there anything I could have done or said beforehand that would make the process easier or quicker for you?
If used a job list: Do you think providing me the job list was useful in this process?
Yes – If yes: How so?
No
If didn’t ask the participant for a job list: If I had asked you to get a job list for our call, do you think it would have been helpful?
Yes – If yes: How so?
No
How comfortable do you feel that the jobs we selected together fall within our stated target of the kinds of jobs we were looking for?
Do you remember the job selection process we used last year,?
Yes – If yes: How does this job selection process compare?
No
Do you have any other comments or thoughts?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Erica Yu |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |