Appendix D External Peer Review Comments

Appendix D: External Peer Review Comments

Name of Project Officer : [Hope Tiesman]

Title of Proposed Project: [The Epidemiology and Impact of Workplace Violence in Pennsylvania Teachers and Paraprofessionals]

Name of Reviewer: [Dan Mercer]

Telephone Number of Reviewer:

Fax Number of Reviewer:

E-mail address of Reviewer: [Mercer, Dan ([email protected])]

******************************************************************************

B. CRITIQUE

1. Significance:

Does this study address an important problem in occupational safety? If the aims of the project are achieved, how will scientific knowledge be advanced? What will be the effect or impact of this study on the DSR mission to reduce worker injuries?

This study addresses a very important problem in occupational safety-- violence in the workplace directed toward primary and secondary school employees. Scientific knowledge advances incrementally, of course. The increment this study will supply is to quantify workplace violence in school settings in Pennsylvania.

This issue has heretofore received remarkably little inquiry, as noted in the “Current State of Knowledge” statement (A.) in the “Introduction” (II.). Student aggressiveness and violence toward school employees (among students with severe learning disabilities, for example) is a known and accepted workplace hazard. Little is known about the general, baseline level of workplace violence among school employees. The general conception, probably secured by news media focus on episodes of extreme violence in U.S. schools, is that violence is on the rise. A scientific approach to the problem, supported by a study that measures it in different situations, is needed.

A useful feature of the study, as noted in the “Protocol Summary” (B.) of the “Project Overview” (I.), is that it will explore workplace violence in a variety of school settings and among a variety of school employees, not only in the classroom among teachers. Non-teachers were not included in the University of Minnesota study (II.A.) that in part inspires this particular project.

I am not qualified to comment on the impact of this study on the DSR mission.

2. Approach:

Are the scientific framework, design (including the composition of the study population), methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the project officer acknowledge potential problem areas including feasibility, and consider alternative tactics?

The scientific framework, design, methods, and planned analyses have all been carefully developed, and are integrated into and appropriate to the aims of the study. In particular, the scientific framework, and, especially, the study design and methods have been very carefully thought through. The study could likely have been conducted earlier, but the delays have strengthened it and enhanced the prospect of success.

The project officer does acknowledge potential problem areas, and, to the extent possible, has taken great pains to avoid these problems. A good example of the project officer’s diligence lies in the determination of the sampling frame and data collection, described in detail in III.A.4. and III.C.2. The officer has not proceeded with the project earlier--delaying it for about one year--because she has not been comfortable with that frame. The officer has been diligent in pursuing the best possible way to collect this information. The methodological approach has been supported by careful statistical consideration of the sampling methodology and sample size.

There are, of course, limits, some them inherent, to a cross-sectional survey study design approach. Non-response bias is a primary concern in a survey study. Substantial follow-up will likely be required, even with the endorsement and participation of the Pennsylvania State Education Association (PSEA) and the American Federation of Teachers Pennsylvania (AFT). As stated in the “Limitations” (section 8) of “Data Handling and Analysis” (C.), of the “Research Methods” (III.), the investigators are aware of these limits, and will take the precautions and steps necessary to minimize their impact. PSEA has pledged to do all it can to encourage and facilitate participation. Response rates will be bolstered by the fact that this issue is of increasing concern and visibility to school employees. Cross-sectional analysis can be misleading simply because observations in cross-section may occur when workplace violence is at an atypically low or high level.

Overall, the approach has been careful and top-notch. I have been and continue to be impressed by the thoroughness of it and attention to it. Hopefully, this study or one very similar to it can be repeated at several points in the future so that the rates and time trends of workplace violence in school settings can be reliably established.

3. Innovation:

Where needed, does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies?

While I think it is difficult to argue that the exact aims are original and innovative in a universal sense, the project does advance research on the topic in an important way: by investigating the issues among a different group of workers at a different point in time. As the “Current State of Knowledge” section (II.A.) indicates, the issues have been explored in Minnesota. They have not been explored in Pennsylvania. And they have not been explored anywhere among school “support” personnel such as teachers’ aides and assistants, food service workers, and transportation workers. Because the settings are less structured, more transitory, and, frankly, marked by a much higher student to employee ratio, there is reason to think that rates and types of violence differ from rates and types found in classroom settings.

As described in broad terms in the “Protocol Summary” (I.B.), a survey approach, where specific details and circumstances can be probed (at least in a standardized way as questions on the survey form), is superior to past approaches that have been used. Past approaches, while somewhat informative and “inexpensive”, are not as useful owing to the bias that may have introduced in the violence that was reported.

4. Project Officer (Investigator):

Is the project officer appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the project officer and other researchers (if any)? Please do not include descriptive biographical information unless important to the evaluation of merit. For new or less experienced NIOSH staff, note if the level of supervision appears adequate.

The project officer has training in and experience with studies and analyses of this type. Dr. Tiesman’s academic training and work in the occupational injury field provide her with an ideal background in the field of epidemiological research. Dr. Tiesman has nearly ten years of experience with issues and research in the field of occupational injuries. The fact that she has managed several large survey research projects is important. Among the various components of this research project, I expect that administering the survey instrument will be challenging. My conversations and communications with Dr. Tiesman give me confidence that she is prepared to handle the concepts and technical aspects involved.

Dr. Tiesman is supported by an experienced and well-trained team, particular Dr. Amandus (with whom I have communicated at length about this project) and Mr. Hendricks (with whom I have had some contact). I am unfamiliar with Dr. Grubb’s work, and hence am unable to comment on it or about her potential contribution to team.

I believe Dr. Tiesman at first underestimated the complexity of setting the sample frame and administering the data collection instrument. However, to her credit, she recognized her mis-estimation and moved quickly to address the issues before attempting to move ahead. (This is primarily what I refer to above when I indicate that delays have improved the probability of the project’s success.)

5. Environment:

Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Please do not include a description of available facilities or equipment unless important to the evaluation of merit.

I anticipate a high probability of success because the approach has been careful and scientific, and I am confident that the principles of good science and high standards will continue to guide the project and be practiced by the principal investigators.

The proposed study design utilizes PSEA’s reach into classroom and school support personnel settings. PSEA has interest in the level and outcomes of workplace violence in Pennsylvania schools. PSEA has been assured and is confident that its members will be protected. This assurance and confidence is rooted in the careful approach DSR-AFEB is taking to the data collection process described in “Research Methods” (part III), specifically the “Data Collection” portion (2.) of “Data Handling and Analysis” (section C.). Because of this assurance and confidence, PSEA can and will recommend that members participate in the survey. That recommendation will greatly increase the survey response rate. That is, in part, what makes this such a useful collaboration between DSR-AFEB and PSEA.

6. Overall Evaluation:

In one paragraph, briefly summarize the most important points of the Critique, addressing the strengths and weaknesses of the application in terms of the five review criteria. Recommend a score reflecting the overall impact of the project on the field of occupational safety and health, weighting the review criteria as you feel appropriate for each application. An application does not need to be strong in all categories to be judged likely to have a major impact and, thus, deserve a high merit rating. For example, an investigator may propose to carry out important work that by its nature is not innovative, but is essential to move a field forward.

I have been involved in discussions and strategy about this research with Drs. Tiesman and Amandus over an extended period (more than a year). As indicated above, I have been impressed, as they have encountered hurdles, with their willingness and ability to regroup and find other ways to complete the research. They might have been expected to concede the project, but to their credit they have not. The results of this research are of great interest and use to PSEA, and, once they are aware of it, will be of great interest and use to other groups with a stake and interest in primary and secondary education in Pennsylvania. That it is such methodologically rigorous research will ultimately contribute greatly to its value.

C. OTHER CONSIDERATIONS

7. Gender, Minority, and Children Inclusion (As Relevant)

No children (defined as persons under 18 years of age) will be included in this study. Female participation, in proportion to share of school workforce, is essential. (About three-fourths of PSEA members are female.) Exposure to and outcomes of workplace violence may well differ by gender. While exposure and outcomes may differ by racial group, as well, PSEA minority membership is quite limited. AFT may have a larger share of its members who are racial minorities, but I’m not in a position to comment intelligently on this issue.

8. Human Subjects Note that NIOSH projects involving human subjects must obtain review and approval from the NIOSH Human Subjects Review Board.

It is my understanding that this project has sought and received the necessary human subjects research approvals.

9. Researcher Hazards

Potential hazards to researchers will be minimal, given data collection methods.

10. Other

(Please type any other comments here)

A. IDENTIFICATION

Name of Project Officer : Hope Tiesman

Title of Proposed Project: The Epidemiology and Impact of Workplace Violence in Pennsylvania Teachers and Paraprofessionals]

Name of Reviewer: Nancy Nachreiner

Telephone Number of Reviewer: 612-625-2487

Fax Number of Reviewer: 612-626-4837

E-mail address of Reviewer: [email protected]

******************************************************************************

B. CRITIQUE

1. Significance:

Does this study address an important problem in occupational safety? If the aims of the project are achieved, how will scientific knowledge be advanced? What will be the effect or impact of this study on the DSR mission to reduce worker injuries?

This study addresses an important problem: work-related violence in the school setting. Scientific knowledge will be advanced through this study by more thoroughly understanding the magnitude of the violence problem, for both educators and other employees in the school setting. The circumstances of violence (physical and non-physical) will be described, as well as consequences. This enhanced understanding will be critical in creating realistic and appropriate violence prevention interventions in this setting. Creation of intervention strategies without first thoroughly understanding the nature of the problem would not be an efficient use of time or resources: this is a necessary first step.

2. Approach:

Are the scientific framework, design (including the composition of the study population), methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the project officer acknowledge potential problem areas including feasibility, and consider alternative tactics?

This study will measure prevalence, circumstances, and outcomes of work-related violence in the school setting. A previous study of violence against teachers accessed the population through licensing data; however, this study is broader and includes licensed educators and paraprofessionals. Because no comprehensive employee list exists, it was an interesting approach to access “denominator” data through union membership. Investigators indicate that 90% of teachers and 65% of paraprofessionals in Pennsylvania are members of one of two unions, and both have agreed to participate in this study. In order to be more representative of the entire union membership of teachers and paraprofessionals in Pennsylvania, investigators also plan to assess differences between responders and non-responders, based on the demographic data available from the unions. No additional data are available to compare union to non-union members, as acknowledged by the investigators.

A random sample of union members will be selected from: 1) Philadelphia, 2) Pittsburgh, and 3) the remaining areas of the state. Participants will be contacted a maximum of three times: 1) initial mailing; 2) reminder postcard; 3) second survey mailing. A great deal of planning has already been incorporated into providing confidentiality of identifying information.

Investigators may wish to consider additional contact approaches if response is low. For the Minnesota Educators’ Study, an initial mailing and up to four follow-up contacts were utilized. In addition, this study included an incentive, which also may have increased study participation. Will the unions be encouraging participation in any way, which may also bolster response?

Also, additional details on inclusion/exclusion criteria will be helpful. Must participants have worked for a certain amount of time in the past year to be eligible? Is it possible that a teacher would be a member of both unions, and if so, will you be assessing for duplicates between your lists?

3. Innovation:

Where needed, does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies?

This study is unique in its approach to assessing the problem of work-related violence through the use of union membership rosters, in order to include both licensed educators and paraprofessionals. More comprehensive denominator data will be available through this approach, rather than limiting participation to employees at one specific school or district. This study utilizes an established survey instrument; however, additional questions have been added, and the instrument is being used for a broader population.

4. Project Officer (Investigator):

Is the project officer appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the project officer and other researchers (if any)? Please do not include descriptive biographical information unless important to the evaluation of merit. For new or less experienced NIOSH staff, note if the level of supervision appears adequate.

Dr. Tiesman has nine years of experience in this field, and has previously managed several large studies. Dr. Amandus is well-known for his work in this field, and Mr. Hendricks also has much experience with these kinds of data. Consultant Dr. Grubb will provide expertise in the area of occupational psychology: a unique perspective to this issue.

5. Environment:

Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Please do not include a description of available facilities or equipment unless important to the evaluation of merit.

This study benefits from the unique cooperation of unions in this work environment in order to assess the problem of work-related violence in Pennsylvania schools. Support of these unions may be developed even further in encouraging participation.

6. Overall Evaluation:

In one paragraph, briefly summarize the most important points of the Critique, addressing the strengths and weaknesses of the application in terms of the five review criteria. Recommend a score reflecting the overall impact of the project on the field of occupational safety and health, weighting the review criteria as you feel appropriate for each application. An application does not need to be strong in all categories to be judged likely to have a major impact and, thus, deserve a high merit rating. For example, an investigator may propose to carry out important work that by its nature is not innovative, but is essential to move a field forward.

This is an important area of study. This proposal is unique in that the aim is to describe the magnitude of the problem in the population of Pennsylvania educators and paraprofessionals, along with circumstances and consequences of the problem. The research team and environment are strong. The approach may be even stronger if the unions can actively encourage members to participate. The investigators are using a previously developed and tested tool, though innovative new questions have been added regarding electronic aggression. Pilot testing of these new questions will be important. Some specific comments are listed below in “Question 10: Other.” In addition, potential methods to increase response may be considered, including additional mailings, in order to optimize the response.

C. OTHER CONSIDERATIONS

7. Gender, Minority, and Children Inclusion (As Relevant)

71% of all education workers in Pennsylvania are female; 94% are white. Age distribution was not provided; though it is expected that most workers would be age 18 or older.

8. Human Subjects Note that NIOSH projects involving human subjects must obtain review and approval from the NIOSH Human Subjects Review Board.

The risks to participants are minimal. Perhaps participants will be recalling uncomfortable details of a violent event. In this case, investigators may wish to develop a protocol regarding handling of these concerns, if contacted by a participant. In addition, instructions may be provided to skip questions if the participant wishes. It may be ideal to have the participant “X” the question, to indicate that the question has been actively refused, rather than “missing.”

9. Researcher Hazards

Not applicable

10. Other

This is an interesting study which will contribute to the knowledge of prevalence, circumstances and consequences of work-related violence against Pennsylvania educators and paraprofessionals.

Specific comments are included below.

*Are there any state-wide data available for additional comparisons between union and non-union members?

*A key point, acknowledged by the researchers, is that because of the study design, causation cannot be assessed; however correlated factors can be described, as well as circumstances and consequences of violence.

*Electronic Violence (aggression) is a new section added to the previously developed instrument. It will be important to test this section, and thoroughly describe how it differs from the non-physical violence already assessed in an earlier section.

*Recall bias was assumed to “unlikely be differential in nature;” however, those who have experienced violence may be more likely to recall specific details near the time of the event, while those who have not experienced violence may be describing things in general. For example, Q58 regarding job satisfaction: no time frame is given, so perhaps those who experienced violence are describing job satisfaction following a violent event; while those without a history of violence are describing things in a more general sense.

*Will you highlight where individuals can view the aggregate results or report? Perhaps a website?

*You may wish to consider highlighting in your letter and other materials that you are interested in obtaining feedback from both those who have and those who HAVE NOT experienced violence. In its current form, the letter and the survey appear to indicate that the primary audience is those who have experienced violence. If you are estimating the prevalence of violence, I think you would like to hear from those who have and those who have not experienced violence.

*Will anyone have less than a GED/High school diploma? If so, you may want to consider an additional response for Q 6.

*Q7: you may want to consider moving “Special Education Teacher” (the current response option 12) to immediately follow response option 1 (classroom teacher – not special ed).

*I would recommend assessing the amount of time worked (hours worked/weeks worked/months worked) in more detail to assist in violence rate calculations. There is currently a question about job classification (full-time, part-time, or substitute), but it may be very helpful to have an indication of how much time the employee worked (exposure time).

*Educators in our study sometimes described “ongoing” physical violence, where they were not able to provide details for a specific event. You may also wish to consider how you would like respondents to describe these types of events on your questionnaire.

*Q33: you may want to consider another option which may occur in some of the special education settings where violence may occur, but not necessarily related to discipline, a fight, or drug sales.

6