Supervisor questionnaire - question-by-question justification

SHSO LEL Supervisor Survey – Question-by-Question Justification

This attachment provides an overview of the questionnaire items to be used in this study and their relationship to the survey’s study goals. The majority of the questions on this survey contain overlapping content with LEL survey questions, as one of the goals of this study is to examine differences in perceptions of key aspects of LEL programs and LEL duties between LELs and the State Highway Safety Office personnel who supervise the LELs. Currently there are 49 State Law Enforcement Liaison (LEL) programs and the programs vary with regard to the number of LELs working under a given program (approximately 240 LELs in total across all programs). The various LEL programs are organized in many different ways. Given the variety of LEL network models and the large amount of annual funding allocated to support LEL programs, NHTSA is seeking an assessment of LEL programs across the nation. Broadly speaking, the data elements below were selected with the intent of characterizing the various LEL models across the States to assist in the selection of LEL programs for in-depth case study and where possible, will be included in analyses to determine which characteristics lead to maximum LEL effectiveness and best practices (leading to high levels of Law Enforcement Agency [LEA] participation, increases in citations during grant-funded activities, reductions in traffic fatalities and injuries, and increases in observed seat belt use). One of the outcomes of the survey will be a descriptive matrix of programs by organizational type, program features and pros, cons and challenges for various LEL approaches in carrying out the LEL mission.

Question Number	Target Information	Relevance	Application
Eligibility Questions & Targeting Appropriate Respondents
A	Are you the person to whom this survey was addressed?	Intended respondent is required to ensure representative sample.	Confirm respondent is person actually selected in the survey sample
B	Do you manage or supervise the LELs in your State (or those who serve in a position that performs similar duties)?	Survey is intended for LEL managers and supervisors.	Ensure respondent is appropriate person to complete the survey
C	Are you willing to complete the survey?	Consent of participant is required	Respondent indicates their consent to participate in survey
Section A: General Overview of LEL Positions Under Your Sponsorship Responses to the questions in this section will assist our understanding of LEL positions, work environments, and LEL coverage areas.
1	Select the title of the people in your State who are the intermediaries between the State Highway Safety Office and Law Enforcement Agencies, who promote and support traffic safety and increase participation in NHTSA-funded traffic safety programs	While most of these positions are called Law Enforcement Liaisons (LELs), in some States they are State employees called Traffic Safety Specialists, Program Managers, Grants Managers, or Program Coordinators. They may also have other job titles.	To compare differences in terminology for the LEL position across the 50 states, and to define this position as a Law Enforcement Liaison (LEL) for respondents as they answer the remaining survey questions.
2	What organization or agency sponsors the LELs in your State?	State LELs work for a number of different employers. Although most State LELs work directly for their State Highway Safety Office (SHSO), some LELs work for both their SHSO and a law enforcement agency. In a few States, LELs are employed by non-profit associations, universities, and other government departments (e.g., Department of Health).	To compare and contrast similarities and differences in LEL sponsorship across states.
3	How many LEL-type positions are there in your State?	For States that have LELs, the number of LELs can range from 1 to 17	Needed to characterize the LEL programs across all states
4	Indicate how many of your State’s LEL positions are full time and how many are part time.	Research is needed to document the variations in LEL employment across states	To enable analyses describing the relationship between LEL effort and effectiveness in LEA recruitment.
5	Indicate approximately how many hours the full-time and part-time LELs perform LEL duties each week.	Research is needed to document the variations in LEL employment across states	To enable analyses describing the relationship between LEL effort and effectiveness in LEA recruitment.
6	Is there a written description for the LEL positions in your State?	Model job descriptions for LELs currently exist. Individuals with written job descriptions know what is expected of them and may perform better than those who do not have written job descriptions (documentation of personal accountability).	To enable analyses describing LEL program effectiveness as a function of the presence of a written job description
6a	If YES to Question 6: Was the LEL Model Position Description used as a guide for creating your State’s LEL position description?	Research is needed to document use of the LEL Model Position Description across states. States that use Model to draft their position descriptions may hire LELs who vary on many attributes from those who do not	Compare and contrast differences in use between states, and differences in LEL backgrounds across States (LEL Survey Questions 11-14, 16)
7	Use the rating scale below to rate the importance of each characteristic for enabling LELs to meet their LEL job responsibilities	These criteria are included in the model LEL position description	To assist with future revisions of the LEL position description and to determine how important LEL supervisors believe each characteristic is for meeting LEL duties vs. what the LELs believe.
8	Do the LELs perform non-LEL-related tasks for your sponsoring agency in addition to their LEL-related responsibilities?	There may be pros and cons of hiring employees of the sponsoring agency to perform LEL work. This question is needed to document the variations in LEL employment across states.	To enable analyses describing the relationship between LEL work environment and LEL program effectiveness. Also, to determine differences in perspectives between LELs and their supervisors.
8a	If YES to Question 8: Describe these non-LEL-related tasks	Non-LEL work may complement LEL-related duties adding to expertise in areas of traffic safety (pro) or may take priority over performing LEL work (con).	Will be used as descriptive data, enhancing understanding of analyses conducted for Question 8.
9	How are your LELs’ responsibilities delineated?	This question is needed to document the present variations in LEL organization of coverage area across states, which often varies.	Q 9 and 10 will enable analyses describing the relationship between LEL program organization/coverage area and LEL program effectiveness.
10	How many LELs cover each of the following highway safety program areas?	Research is needed to document the present variations in LEL coverage area across states.	Q 9 and 10 will enable analyses describing the relationship between LEL program organization/coverage area and LEL program effectiveness.
11	Are the LELs in your State responsible for providing services to all LEAs in their area of responsibility, or only to highway safety program grantees within their area of responsibility?	Research is needed to document similarities and differences in how the various LEL programs across the states operate.	Used in analyses to determine best practices in maximizing LEL program effectiveness. Also, to determine differences in perceptions between LELs and their supervisors.
12	What reporting information do you require from your LELs to document performance of their LEL responsibilities?	Those who are required to track and report performance indices may be better performers compared to those who are not required to track and report these indices.	To determine whether tracking and reporting job performance measures is associated with more effective LEL performance (more LEA recruits and better highway safety outcomes among the LEAs). Also, to determine differences in perceptions between LELs and their supervisors.
SECTION B: HOW YOUR AGENCY ASSISTS LELS IN CARRYING OUT THEIR DUTIES AND RESPONSIBILITIES Responses to the questions in this section will help us understand how LEL supervisors and the sponsoring agency assist the LELs in performance of their work.
13	How many staff in your agency, including yourself, work directly with or supervise the LELs in your State?	Research is needed to determine staffing levels	To determine differences between States in the supervisor-to-LEL ratio. Also, to determine whether multiple respondents’ surveys must be considered when characterizing a particular State’s SHSO role in managing LELs.
14	In what ways does your agency assist the LELs in carrying out their responsibilities?	To determine the depth and breadth of assistance provided to LELs in carrying out their activities and differences across states	To determine relationship between SHSO assistance and effective LEL programs.
SECTION C: HOW YOU COMMUNICATE WITH YOUR LELs Responses to the questions in this section will help us understand how often LEL supervisors communicate with the LELs in their State, the methods they use for their communications, and their beliefs regarding the most effective means of communication.
15	How frequently (on average) do you communicate with your LELs?	More frequent communications translates to greater supervision and higher collaboration, and may positively influence LEL program effectiveness.	To determine similarities and differences across States in the level of supervisor communication, and to determine associations between frequent supervisor communication and LEL program effectiveness and traffic safety performance.
16	Use the rating scale below to rate the effectiveness of the following modes of communication with your LELs	There are many ways that supervisors can communicate with LELs, and this question will assist in the understanding of which ways are most effective	Will assist in determining best practices for LEL management.
SECTION D: HOW YOUR AGENCY EVALUATES THE JOB PERFORMANCE OF YOUR LELs Responses to the questions in this section will provide information about how the LEL sponsor evaluates LEL job performance. These questions focus on their individual work performance, not the highway safety outcome performance goals and measures.
17	Does someone in your agency evaluate the work performance of your LELs?	LELs whose work performance is evaluated may be better performers than those whose work is not evaluated.	Question 17 and its sub questions will be used to describe similarities and differences in supervision of LEL programs across States. Also to be used in analyses examining the relationships between performance evaluations and measures such as those listed in Q 15 (frequency of communication) as well as better performance by their state on the core traffic safety performance measures.
17a	If YES to Question 17: Are you the individual within your agency who performs evaluations of LEL work performance?	If there are multiple LEL supervisors within a State, responses to Q 17b, 17c, and 17d may differ depending on whether the respondent performs evaluations.	If there are multiple respondents from a State, the responses to Q 17b-d will be most valid from those who evaluate LELs (respond “Yes” to Q 17)
17b	If YES to Question 17: Does your agency provide information to the LELs about how they will be evaluated at the start of the evaluation period? (e.g., is there a plan, established goals, activity measurements, etc.?)	LELs who know how they will be evaluated may be better performers	Question 17 and its sub questions will be used to describe similarities and differences in LEL performance evaluation across States. Also to be used in analyses examining the relationships between performance evaluations and measures such as those listed in Q 15(frequency of communication) as well as better performance by their state on the core traffic safety performance measures.
17c	If YES to Question 17: How often does your agency provide feedback to your LELs regarding their job performance?	Frequent feedback may be associated with better performance, and may be more like to occur for LELs who work in offices with their SHSOs. May be useful for identifying best practices	Question 17 and its sub questions will be used to describe similarities and differences in LEL performance evaluation across States. Also to be used in analyses examining the relationships between performance evaluations and measures such as those listed in Q 15(frequency of communication) as well as better performance by their state on the core traffic safety performance measures.
17d	If YES to Question 17: What metrics does your agency use to evaluate the work of your LELs?	Research is needed to characterize the performance metrics used across the States and to describe best practices	Question 17 and its sub questions will be used to describe similarities and differences in LEL performance evaluation across States. Also to be used in analyses examining the relationships between performance evaluations and measures such as those listed in Q 15(frequency of communication) as well as better performance by their state on the core traffic safety performance measures.
SECTION E: LAW ENFORCEMENT AGENCY (LEA) RECRUITMENT AND FUNDING FOR NHTSA HIGHWAY SAFETY GRANTS Responses to the questions in this section will assist in our understanding of how LEAs are recruited and funded and their participation rate in NHTSA’s highway safety programs in the past 12 months.
18	Do the participating LEAs in your State receive highway safety grant funding?	LEAs may be more likely to participate in NHTSA highway safety programs, if there is a monetary incentive (grants)	Descriptive statistics will be used to characterize similarities and differences across states
19	Who is responsible for evaluating the grant applications submitted by the LEAs?	Selection of the best qualified candidates is likely to result in higher performing grantees. Research is needed to characterize grantee selection process across states	To be used in analyses of associations between grantee selection and grantee performance. Useful in developing best practices in grantee selection
20	Do you participate in the evaluation of grant applications or provide input to the grantee selection process?	Research is needed to characterize grantee selection process across states	Questions 20 and 20a will be used in descriptive analyses describing grantee selection process
20a	If YES to Question 20: Describe the criteria and process used to select LEA grantees.	Research is needed to characterize grantee selection process across states	Questions 20 and 20a will be used in descriptive analyses describing grantee selection process
21	How many LEAs received grants in 2016 in each of the following program areas?	Research is needed to compare and contrast participation by focus area across states	Useful for understanding the focus of highway safety programs by state
22	Does your agency use crash data to help recruit LEAs to participate in NHTSA’s highway safety programs?	Crash data may persuade agencies to participate, if they see where the problem areas in their jurisdiction lie	Descriptive statistics will be used to characterize similarities and differences across states and in analyses of relationship between use of crash data and LEA participation rate
23	Have your LELs developed any unique approaches to encourage more LEAs to participate in grants and other highway safety activities?	Research is needed to characterize LEA recruitment processes across states	Questions 23 and 23a will be used in descriptive analyses describing LEA recruitment methods. May lead to best practices.
23a	If YES to Question 23: Describe these unique approaches	Research is needed to characterize LEA recruitment processes across states	Questions 23 and 23a will be used in descriptive analyses describing LEA recruitment methods. May lead to best practices.
24	Does your State have a standardized process to identify LEAs to be funded versus those who will voluntarily participate in your highway safety programs?	Questions 24 and 24a are needed to compare and contrast similarities and differences in States’ LEL program activities	Useful in identifying best practices
24a	If YES to Question 24: Describe this standardized process	Questions 24 and 24a are needed to compare and contrast similarities and differences in States’ LEL program activities	Useful in identifying best practices
25	What percentage of the LEAs in your State are eligible for highway safety grant funding, but do not elect to participate in these funded programs?	Research is needed to document magnitude of non-participation problem	Questions 25 and 26 will be used in descriptive statistics to help NHTSA understand how large the population of non-participators is and the reasons for non-participation, so that new methods can be developed to attract participation.
26	What are the reasons that eligible LEAs do not participate in these funded programs?	LELs have many challenges in motivating LEAs to participate in NHTSA mobilizations and training. This question addresses this issue.	Questions 25 and 26 will be used in descriptive statistics to help NHTSA understand how large the population of non-participators is and the reasons for non-participation, so that new methods can be developed to attract participation.
26a	Are there strategies for overcoming any of the barriers checked in Question 26?	Research is needed to gather strategies that work to remove barriers to LEA participation.	Useful in identifying best practices
26b	IF YES to Question 26a: Describe these strategies	Research is needed to gather strategies that work to remove barriers to LEA participation.	Useful in identifying best practices
SECTION F: HOW YOUR AGENCY USES PERFORMANCE MEASURES TO DETERMINE THE EFFECTIVENESS OF NHTSA-SPONSORED HIGHWAY SAFETY GRANT ACTIVITIES The purpose of the following questions is to learn if and how each sponsoring agency uses traffic safety outcome and activity performance measures to determine the effectiveness of NHTSA-sponsored highway safety program grants.
27	Do your highway safety programs use traffic safety performance measures to determine the effectiveness of program activities?	Traffic safety performance measures are required for each state’s traffic safety activities, but it is unknown to what extent these goals are used at the local level.	Question 27 and its sub-questions will be used to determine the proportion of sponsoring agencies that track their grantees’ performance, and may be used in analyses to determine the relationship between tracking program activities and high grantee performance. Also may lead to identification of best practices for maximum LEL program performance. Will also be used to compare sponsor and LEL perceptions.
27a	If YES to Question 27, What are these traffic safety performance measures and goals	Traffic safety performance measures are required for each state’s traffic safety activities, but it is unknown to what extent these goals are used at the local level.	Question 27 and its sub-questions will be used to determine the proportion of sponsors who track their grantees’ performance, and may be used in analyses to determine the relationship between tracking program activities and high grantee performance. Also may lead to identification of best practices for maximum LEL program performance. Will also be used to compare sponsor and LEL perceptions.
27b	If YES to Question 27, Who defines the goals for the outcome measures?	Traffic safety performance measures are required for each state’s traffic safety activities, but it is unknown to what extent these goals are used at the local level.	Question 27 and its sub-questions will be used to determine the proportion of LELs who track their grantees’ performance, and may be used in analyses to determine the relationship between tracking program activities and high grantee performance. Also may lead to identification of best practices for maximum LEL program performance. Will also be used to compare sponsor and LEL perceptions.
28	Does your State use benchmarks to evaluate the level of enforcement activity of each LEA grantee? (e.g., performance measures for counts of crackdowns and mobilizations, hours worked, citations or warnings written during each, contacts per hour)	Questions 28 and 28a are needed to compare and contrast similarities and differences in States’ LEL program activities. States with benchmarks may show better LEA grantee performance than States without.	Useful in identifying best practices
28a	If YES to Question 28a, please elaborate on this standardized process (or upload a document describing the benchmarks and process, if you have one available electronically. See last survey question in Section I for upload link).	Questions 28 and 28a are needed to compare and contrast similarities and differences in States’ LEL program activities. States with benchmarks may show better LEA grantee performance than States without.	Useful in identifying best practices
SECTION G: INFORMATION ABOUT YOUR STATE LEL BUDGET This question that will help us better understand what is included in each State’s LEL budget.
29	What does the LEL program budget in your State include?	Research is needed to compare and contrast items included in LEL budget by state	Will be used to describe similarities and differences in States’ budgets for LEL activities, and may be used in analyses of LEL program effectiveness
SECTION H: ADDITIONAL INFORMATION – OPEN FEEDBACK Questions in this sections are used to gather information from those interested and willing to provide more detailed information about how they manage and support the LELs in their duties, how their State’s LEL program is organized and operates, and ways they think the LEL position or program could be improved.
30	Please provide any additional information that would be helpful to our understanding of how your LEL program operates.	Research is needed to fill in any gaps left unanswered by existing survey questions	Useful in identifying best practices
31	If you have additional information about how your agency manages and supports the LELs, please provide it here.	Research is needed to fill in any gaps left unanswered by existing survey questions	Useful in identifying best practices
32	If you have any recommendations about how the LEL position or LEL program could be improved, please provide them here.	Research is needed to fill in any gaps left unanswered by existing survey questions	Useful in identifying best practices
33	Please provide a profile of the LEL business model used by the State.	States may have different business models or none at all; research is needed to identify various business models	Compare and contrast different states’ business model, for identifying best practices
34	Would you be willing to be interviewed by our research team in the near future to discuss topics such as LEL program costs, including salaries, travel expenditures, and even the sponsoring agency management time?	Project has an optional task to conduct case studies with goal of identifying LEL program organizational models that reflect promising practices for LEL programs	Will assist in identifying states interested in participating in more in-depth study.
SECTION I: UPLOAD DOCUMENTS This section provides instructions and upload buttons for ease of respondents’ uploading of requested documentation.
35	If there is a written description for your LEL positions, please upload a copy, or email, fax, or mail it to the address provided at the end of this survey.	States may have different LEL position descriptions	Comparison and contrast with LEL position descriptions in other states and with the model program description
36	If there are written procedures with benchmarks for evaluating level of enforcement activity for LEA grantees, please upload them, or email, fax, or mail to the address provided at the end of the survey	States may have different benchmarks for evaluating level of enforcement activity for LEA grantees	Useful in identifying best practices

5