Section B

National Occupational Research Agenda (NORA) 2016 Decade Review

Request for Office of Management and Budget Review and

Approval for Federally Sponsored Data Collection

Supporting Statement Part B

Project officer:

Sidney C. Soderholm, PhD

National Occupational Research Agenda (NORA) Coordinator

National Institute for Occupational Safety and Health

395 E Street, SW, Suite 9100, MS P-06

Washington, DC 20201

202-245-0665 (voice)

202-245-0628 (fax)

Email: [email protected]

October 1, 2014

Table of Contents

1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with Nonresponse

4. Test of Procedures or Methods to be Undertaken

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Statistical Methods (title used for Collections of Information Employing Statistical Methods)

We will use a non-probability purposive (targeted) sample derived from the list of all current and past council members. Members for whom we have accurate contact information will be selected, without replacement. Statistical sampling will not be performed and statistical inferences will not be drawn. Statistical descriptions may be developed for some of the data, e.g., medians, means and standard deviations. No generalizations to a larger population will be made.

1. Respondent Universe and Sampling Methods

The sample universe is all 554 current and former members and leaders of a NORA sector council who joined the council before October 1, 2013. For the purpose of this request, only the 352 individuals who are not Federal employees comprise the burden. No sampling will be performed. Each sector council dealt with the unique issues in its sector and had its own way of setting and accomplishing goals. As a result, NIOSH management needs to collect data from enough respondents to assess each council individually as well as the overall effort. Sampling only a portion of the population would risk receiving too few responses to adequately assess some councils.

The survey instrument will be sent to everyone for whom we have an email address. NIOSH sector council co-leaders had a valid email address for every council member during their tenure on the Council. Email addresses are now being centralized in a single database for administration of the questionnaire. Based on the responses of the NIOSH leaders of four of the councils, so far, we anticipate that we will not have a current email address for approximately 13% of non-federal individuals in the population. In most of those cases, the individual left the council and the organization they represented on the council. Survey requests that are returned due to non-active email accounts will be removed from the sample and there will be no replacement.

NIOSH has not conducted a formal survey of this population and cannot estimate a response rate based on previous results. A meta-analysis of response rates for internet-based surveys found a range of 11% to 82% (Manfreda, Bosnjak, Berzelak, Haas, & Vehovar, 2008), with higher response rates associated with multiple contacts, a pre-notice and personalization of the invitation as described in the Dillman method (Dillman, Smyth & Christian, 2009). We will use the Dillman method in this survey to maximize the potential for a good response rate. Vehovar and Manfreda (2008) describe high response rates often result from membership surveys. This information collection may benefit from that effect. Surveying the entire sample population of sector council members will assure that representatives from each of the 10 NORA sector councils will be included in the survey and hopefully result in valuable individual success stories and suggestions for future improvements.

2. Procedures for the Collection of Information

The survey instrument has been designed using the web based Survey Monkey program (surveymonkey.com). Screen shots of the survey are in Appendix E. We will collect all responses through Survey Monkey.

Each respondent will receive a personalized pre-survey email from the Director of NIOSH. The email will inform individuals of the upcoming survey and encourage participation. The pre-survey email is attached as Appendix F.

The NORA Evaluation Team will send a personalized email one week later. This email will include the link to the survey instrument. This email is attached as Appendix G.

One week after the personalized email containing the survey link is sent, the NORA Evaluation Team will send a follow up email to non-respondents. The list of emails of non-respondents is maintained by Survey Monkey. This correspondence will encourage those who have not completed the questionnaire to do so. See Appendix H for this email.

A final email reminder will be sent by the NORA Evaluation Team to non-respondents one week later, encouraging participation in the survey. This email is attached as Appendix I.

A week later, the survey will close and no more responses will be accepted by Survey Monkey.

There will be no follow-up with respondents who submit a partially-completed questionnaire.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Dillman et al. (2009) provide guidance on administering questionnaires, including internet surveys, based on his Tailored Design Method. Research of the Dillman method asserts that multiple contacts are proven more effective than any other technique for increasing survey response rates for all surveys, including web-based. Increased response rates are attributed to sending a pre-notice and personalization of the request. All those approaches will be used in this survey.

Since NORA sector councils have had turnover during the second decade, some former council members became inactive six or more years ago. The number of individuals without a valid email address in our records will be reported as not being reached and removed from the denominator data of respondents. The number of individuals who receive an email but do not respond will be reported as non-respondents.

4. Test of Procedures or Methods to be Undertaken

We conducted a pilot test of the draft of the questionnaire by asking 8 federal employees of NIOSH who hold NORA sector council leadership positions to complete the questionnaire and provide feedback during a teleconference discussion. Based on their response, some questions were modified substantially for clarity and to better elicit responses that would meet the goals of the overall evaluation. Respondents to the pilot test estimated completion of the questionnaire would require approximately 15 minutes.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The statistical and analytical aspects of this survey were designed by NIOSH staff and CDC Evaluation Fellows with doctoral training in survey design and statistics. Input was received from NIOSH leadership participating in a workgroup that has been convened to help direct and manage this evaluation effort. No one outside of NIOSH was consulted on any part of the statistical design or analysis plan.

The persons primarily involved in designing the overall program evaluation and survey instrument that is the subject of this ICR are:

Sarah A. Felknor, MS, DrPH

Associate Director for Research Integration and Extramural Performance

404/498-2530

[email protected]

Donjanea Fletcher Williams, Ed.D., LPC

Program Evaluation Fellow

404/498-2564

[email protected]

Sidney C. Soderholm, PhD

NORA Coordinator

202/245-0665

[email protected]

They comprise the NORA Evaluation Team.

The person who will receive the data and analyze it is:

Donjanea Fletcher Williams, Ed.D., LPC

Program Evaluation Fellow

404/498-2564

[email protected]

Dr. Williams will be supported by other members of the NORA Evaluation Team as needed in the analysis and reporting of the survey data.