Two Generic Clearance Requests - Focus Groups

Interviewee Name: _______________________ Interviewer Name: ______________________

Interviewee Title: _________________________ Interviewee Organization: ________________

Date: _________________ Time: __________________

Introduction: The Social Security Administration (SSA) is undertaking a project to develop a new occupational information system (OIS) tailored specifically for the SSA‘s disability programs and adjudication process. To collect the job data for its OIS, SSA must train individuals as job analysts to analyze work using a variety of techniques and processes that will allow them to obtain detailed descriptions, specifications, measurements and requirements for jobs throughout the labor market of the United States.

Our team at ICF International is currently working with SSA on two initiatives related to this project: (1) Developing the job analysis methodology that job analysts will use to collect occupational data and (2) Designing a business strategy for the training, certifying, and recruiting of job analysts across the U.S. As a part of this effort, we are determining a training and certification strategy for training the training and certifying individuals to serve as job analysts for the SSA. These individuals will be recruited and trained throughout the U.S.

In our interview today, we will be asking for your perspective on a number of topics related to training. Please note that this meeting is being recorded to ensure accuracy of data collected. We also have an individual on the phone (state recorder’s name) who will be taking notes during the call. Do you have any concerns with this? Do you have any questions before we begin?

Introduction

1. In 30 seconds, please briefly introduce yourself and describe what you do in your job that pertains to training?

Key Elements of Training Structure and Design

2. What do you consider the most critical factor in determining how a training program should be designed?

Probe:

• For example, to what extend should the ISD ADDIE model or other principles of training design be considered?

3. What are some examples of effective ways to structure training? (e.g., mentor-based; criterion-based; lecture only)

4. In training, what is the best way to teach a specific process, especially for processes in which strict guidelines need to be followed? What elements are critical to include in this type of training? (i.e., To train candidates in how to analyze jobs.)

Probes:

1. What needs to be considered in the training curriculum?

2. What design features or types of training should be included in the training? (e.g., Frame-of-Reference is a training type)

3. What type of activities should be included to help participants learn the processes?

4. How should the method of training delivery be varied to help participants learn a specific process?

5. How long do you estimate the training might need to be?

Training for Data Collection

5. What are some programs or concepts of training that could be used to help teach participants how to perform large-scale data collection?

Probes:

1. What needs to be considered in designing the training?

2. What design features should be included?

3. What type of activities should be included?

4. How long do you estimate the training might need to be?

6. (Optional: If SME has job analysis experience) What are some examples of training programs that teach participants how to analyze jobs and/or collect job related information from job incumbents?

7. What types of techniques can be trained to help analysts learn how to extract potentially-sensitive information from individuals (e.g., interviewees), uphold the principles of confidentiality and keep information secure?

Probes:

1. How should training help participants understand the types of information that should be kept confidential?

2. In terms of interviewing, what types of techniques should be trained to help analysts learn how to de-identify or collect data in such as way that is non-attributional?

8. What important elements need to be covered in the training of analysts who will be working and collecting data from job incumbents through interviews, onsite observation, surveys and other data collection tools?

9. For SSA, analysts will need to learn about the physical and mental demands of each job through observation, use of measuring devices (e.g., light meters, rulers), and interviews with people doing the job. What factors need to be integrated in the training to teach analysts how to collect this type of data?

10. How should the training be structured if participants may have varying skill levels and experience?

Training Assessment

11. What are some of the key standards for certificate programs that must be considered in designing the training?

Probes:

1. What components are necessary to ensure the certificate program is directly linked to the training and assesses skills taught in the training?

2. What types of assessments should be presented? At what intervals in the training should those be presented? What content should be included in the assessments?

12. What are some effective ways to assess candidate retention of skills taught in training?

Probes:

• What are specific types of written tests that could be conducted?

• What are examples of interactive or mock trial assessments that could be conducted?

13. What are other effective ways to evaluate training effectiveness?

Training Implementation

14. What is the most critical factor in determining how training should be delivered?

15. What method of training delivery would be most suitable for teaching a process and/or for teaching large-scale data collection?

Probes:

1. What are the pros and cons of that method?

2. (Ask only if the participant has job analysis knowledge) What method of training delivery do you think would be most suitable for teaching job analysis?

16. For a large-scale, national training, what are some effective methods for training delivery?

17. What are the benefits and disadvantages to consider with each of the following training delivery methods for a large-scale training initiative?

    • In-person instructor led

    • Self-paced online training

    • Distance learning (instructor led)

    • Webcasts

    • Multi-media training (e.g., CD-ROM; audio/video)

    • Blended/Mixed method delivery

    • Reference documentation (ongoing training source)

    • Group Discussion Forums

    • Others?

18. What is the best method for ensuring training is delivered in a consistent manner across trainers?

Probes:

1. What are some important considerations for trainers?

2. What are some effective train-the-trainer models?

3. What resources should be provided to trainers and participants to help ensure consistency in what is taught and applied?

19. How much time do you estimate it will take to develop the type of training with the features we have been discussing?

    • (For example, features of the training that may consist of: candidates with varying skill sets;; collecting data via face-to-face interviews and surveys; potentially administering at different points in time across geographic locations; data analysts collecting data independently via interviews, observations and focus groups; the data collected to inform an occupational database used to make disability determinations; could be up to 1000 participants.)

20. How can cost of the training be accurately estimated?

Probes:

1. What factors help determine cost?

2. What do you suspect it will cost to develop and administer training for job analysts? Per participant when administered?

21. What type of resources would be needed to implement the type of training for job analysts that we have been discussing?

22. If the training is to be implemented on a national scale and potentially at various times during the year, how might various aspects of training be impacted?

Probes:

• Training structure?

• Training design?

• Training delivery?

• Transfer of training/Application of training by participants?

• Consistency in training?

• Assessment of training?

• Certificate program tied to the training?

Participant Engagement

23. What should be included in the training to ensure that participants can successful transfer or apply what they learn in training to real-life job setting?

24. How should the training be developed to maximize participant engagement during the training?

25. Given that participants may not be SSA employees but may be contractors or temporary hires, what can be included in the training or how can the training be structured to increases participant motivation and commitment?

We will use the Paperwork Reduction Act statement below with each of the focus groups:

Paperwork Reduction Act Statement - This focus group, 0960-0526, meets the requirements of 44 U.S.C. § 3507, as amended by section 2 of the Paperwork Reduction Act of 1995.  We estimate that it will take about 60 minutes to conduct these focus groups.  Send only comments relating to our time estimate to:  SSA, 6401 Security Blvd, Baltimore, MD  21235-6401.

ICF International 4