Formative Study of Trade Adjustment Assistance (TAA) Navigators

Appendix D

Navigator observation rubric

 

navigator Observation RUBRIC

Introduction and consent

[Note to observer – At the beginning of the activity, introduce yourself, explain the purpose of the observation, and obtain verbal consent:]

My name is [NAME] and I am from a research company called [Mathematica/SPR]. We were hired by the U.S. Department of Labor to conduct a study of navigators in the workforce system. With your verbal consent, we would like to observe this activity to create research findings that can help improve services for future participants. We will not collect your name during the observation. Do you have any questions?

[Respond to any questions.]

Do you consent to have me observe this activity?

[If anyone says no:] Okay, thank you for your consideration. [Site visitor should leave the activity space.]

[If yes:] Great, thank you!

NOTE: If the activity is an information session open to the public, the observer will determine with the Navigator whether to introduce the observation and collect consent from the group.

Public reporting burden for this direct observation is estimated to average 0 hours per response. The burden estimate includes the time for reviewing instructions, searching existing data sources, gathering, and maintaining the data needed, and completing and submitting the observation. This collection of information is voluntary. You are not required to respond to this collection of information unless it displays a valid OMB control number. Please send comments regarding the burden estimate or any other aspect of this collection of information to the Chief Evaluation Office, U.S. Department of Labor, 200 Constitution Ave NW, Room S-4307, Washington, DC 20210 and reference OMB control number 1290 – 0043.

Date: _______________ Observer name _________________________________________

State/program name: ____________________ Location: ____________________________

Start time: _________ End time: ____________ Full activity observed: Y / N

Type of activity:
 Outreach and education. Specify: ________________________________________________

 Direct assistance to a group. Specify: ____________________________________________

 Direct assistance to an individual. Specify: ________________________________________

 Cross-program coordination. Specify: ____________________________________________

 Other. Specify: ______________________________________________________________

Setting for the activity:

 In person  Video session (e.g., Zoom)

Describe the meeting space (e.g., private office, cubicle, conference room, open space)

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Is the setting appropriate for the nature of the activity (e.g., sufficient privacy, enough space, etc.)? Y / N

Why/why not? _________________________________________________________________

Were there distractions during the activity (e.g., office noises, children present)? Y / N

Describe: _____________________________________________________________________

If virtual setting is used, was a camera used by the staff and participants? Y / N

Were there technical issues with the connection (e.g., ability to see or hear each other)? Y / N

__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Notes during the activity

1. How many people (recipients of Navigation services) are present? __________________________________________________________________________

2. Describe the people (recipients of Navigation services) present (e.g., approximate ages, ethnicities, and gender identities) [Note: Do not include people’s names.] ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

3. How many Navigators are present? _______If there is more than one Navigator present, describe the rationale for the service being provided by more than one?

____________________________________________________________________________________________________________________________________________________

4. Describe whether there are staff, other than the Navigator present, and, if so, how many and what type of staff they are (e.g., titles/roles)? ____________________________________________________________________________________________________________________________________________________

5. [Ask only if this is an observation of participant services] Do the staff appear diverse and/or representative of the participant group? Y / N Describe:_______________________________________________________________________________________________________________________________________________________________________________________________________________________

6. Describe the focus and content of the activity:

• What is the goal of the activity?

• What are participants and staff doing?

• What topics are discussed?

• What information is provided about the program?

• What language is the information presented in (if not English)? Are there interpretation services available or used?

• What forms, tools, or equipment (e.g., computers, tablets) are used? How?

• Are materials available in different languages? Which ones?

__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

7. Describe the structure of the activity.

• Are Navigators or other staff presenting materials?

• Are Navigators attempting to collect information from participants/employers?

• Are Navigators or other staff and participants/employers interacting?

• Was there sufficient equipment or materials for the activity?

• If conducted virtually, did all of the technology work as needed?

• [For observations of participant activity only] Was attendance taken or information gathered about participants?

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

8. Describe the interactions between participants and staff.

• How engaged are participants/employers with the staff? For group activities, are all participants/employers engaged and interacting with Navigators? Do Navigators make an effort to engage all participants/employers?

• Do Navigators appear competent, caring, and respectful in their interactions with participants/employers?

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

9. How engaged do participants/employers appear with the activity (e.g., are they paying attention, are they asking questions)? _________________________________________________________________________________________________________________________________________________________________________________________________________________________________

______________________________________________________________________________________________________________________________________________________

10. Summarize the strengths and weaknesses of the activity you observed.

• Is the activity well-structured and defined?

• Do participants/employers appear to understand the purpose of the activity?

• How satisfied with the activity do participants/employers appear?

• Is time allotted for questions and answers? Were Navigators able to answer all questions or, if not, did they describe a process and a timeframe for providing an answer to participants/employers?

• Do Navigators make it clear what the next steps are for participants/employers (if applicable)?

__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Additional notes

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Mathematica^® Inc. 1