CMS-10257.Supporting Statement Part B

Supporting Statement: Part B
Request for Clearance for National Medicare Training Program
Training Needs Assessment Survey

1.
For this survey, the sample will be drawn from lists of individuals who utilize CMS
NMTP training products and resources. These lists consist of representatives of the State Health
Insurance Assistance Program (SHIP), and individuals who partner with CMS Regional Offices
to conduct Medicare beneficiary outreach and education activities. To ensure individuals receive
only one initial survey invitation, duplicate information will be removed once the lists have been
merged. The final, merged and cleaned sample list will include approximately 5,000 individuals
from which we aim to achieve an 80% response rate, resulting in 4,000 completed surveys. This
number of completed surveys will allow statistically significant (p<05) comparisons between
various partner segments. A large sample is necessary since not all partners will have used all of
the training materials offered. Descriptive analysis techniques, as well as t-tests, chi-square tests,
and analysis of variance (ANOVA) will be used to compare findings from traditional and nontraditional partner segments. Pacific Consulting Group’s leverage analysis technique will also be
used to identify issues that have high priority for respondents.
2.
The sample to be provided by CMS will include partner email addresses with individual
contact and organization names, as well as phone numbers. We expect that as stakeholders with
an interest in improving the resources they rely on, the partner audience will be willing to take
the time to provide their input. Initial survey invitations will emphasize the importance of their
feedback to CMS. These survey invitations will be sent via email and will include a link to the
web survey. Those not responding to the initial email invitation will be sent up to two reminders
via email. After the two reminder notices have been issued (within three weeks of the start of the
field period), the data collection subcontractor will begin conducting follow-up phone calls with
non-respondents. The fielding period will be six weeks. In our experience, few additional
surveys are completed beyond that point to merit further postponement of the analysis effort.
3. Using the following methods, we will aim to achieve an 80% response rate:
• Survey invitation emails will be sent from a CMS address and will include a link
to the survey.
• The survey follow-up emails will denote that the survey is being conducted under
the auspices of CMS.
• The survey invitation will have the signature of a CMS executive to add
credibility to the request.
• Up to two email reminders will be sent to participants and each will include a link
to the survey.
• Mid-way through the fielding period non-responders will be contacted by phone
to obtain their participation. They will be encouraged to complete the survey over
the phone with the interviewer at that time or at another time that is more
convenient for them. If they would prefer to complete the survey online they will
still have that option. Respondents who indicate that they would like to complete
the survey online and fail to do so within a couple days will receive another
reminder call.
Pacific Consulting Group
4/17/2008

1

4. A pre-test of the survey will be conducted with nine partners representing different
organizations. These partners will be randomly selected from the sample list. This pre-test will
serve as a means to improve the questionnaire and minimize burden on respondents. Web
surveys have been shown to be an effective means of obtaining input from organizations who
work with consumers while placing minimal burden on the respondents.
5. For questions regarding the study, questionnaire design, or statistical methodology, contact:
Andrea Ptaszek, Ph.D.
Project Manager
Pacific Consulting Group
399 Sherman Avenue, Suite 1
Palo Alto, CA 94304
650.327.8108

Pacific Consulting Group
4/17/2008

2