Summary of changes and adjustments

A summary of Changes and Adjustments in the NPIN Data Collection Instruments

There are two data collection instruments in the NPIN Outcome Evaluation Effort:

1. The NPIN Web Site User Survey

2. The NPIN Products and Services User Survey

A. Changes and Adjustments in the NPIN Website User Survey

For the NPIN Web Site User Survey, the number of questions has remained at 24. The first section has been revised, so “Your Impressions of the Web site” has become “Your Experiences with the NPIN Web site.” Questions have been added to reflect current Web practice to identify user’s top tasks and ease of completion. Questions include: the frequency of them visiting the NPIN Web site, how they came to NPIN site, the tasks users came to the NPIN site to accomplish, whether they were able to accomplish their tasks, and how they are planning to use the information obtained. Questions to rate the NPIN site and ease of navigation were retained.

The reason we have replaced “Your Impressions of the Web site” section with “Your Experiences with the NPIN Web site” is due to the fact that most of our target population have been using the NPIN Web site for at least three years, so it is relevant to ask about their experience with the NPIN Web site, while three years ago, many of the respondents had only been using the NPIN Web site for a short period of time and would not have much to offer in terms of experience.

In the “Relevance to Your Work” section, a question was added asking users what disease topics they look for on the NPIN Web site. This change is also due to the fact that after our users have had more experience, they would be able to provide input in a more detailed fashion. In order to get more people to subscribe to NPIN’s electronic mailing list, we have also replaced one general question asking whether they find the information on the NPIN Web site current with a question asking their experience of subscribing to NPIN’s electronic mailing list. These changes have improved the quality of the questions and the information that will be collected with these new questions without increasing any burden on the respondents.

In terms of the use of the NPIN Web site, a new question was included asking users to identify one thing they would most likely to change or fix, as they now would be able to offer suggestions for improvement in regard to the use of the NPIN Web site and the future use of the NPIN Web site.

In the “How to Better Meet Your Needs” section and the “Background Information” section the questions have remained the same with only slight wording improvement and new subcategories like “Twitter”, “Facebook” and “LinkedIn” added.

B. Changes and Adjustments in the NPIN Products and Services

User Survey

For the NPIN Products and Services User Survey, the number of questions has been reduced from 27 to 18, which will substantially reduce the burden on the respondents.

The questions in “Your Expectations and Satisfaction” section, have remained unchanged.

In the “NPIN Customer Service” section, the three questions regarding ordering materials have been removed as materials are now available from NPIN online, which is more convenient, cost-effective and definitely more efficient.

In the “Use of NPIN Products/Services” section, one question has been added to identify whether users prefer Live-Chat or the 800 number, which is now an important service provided by NPIN.

In the “Relevance to Your Work” section, some new categories (such a Twitter) have been added to the NPIN products/services as they have become relevant. One question regarding whether the information provided by NPIN was current has been removed.

In the “Future Use of NPIN Products and Services” section, one question regarding ordering materials has been removed, as it is no longer applicable. The other two questions have remained the same.

In “How to Better Meet Your Needs” section, one Likert scale question involving ordering materials has been removed, as “ordering materials” is no longer valid when all materials are now available online.

In conclusions, we have substantially reduced the burden on the respondents as we have reduced the number of questions and replaced some of the old questions, some of which are either no longer relevant or somehow should be improved. Now we have two better data collection instruments (and one is substantially shorter). As a result we will collect better data for our evaluation, which will enable us to provide better services to our prevention partners and the general public.

3