B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
The process evaluation of the NNTCQ Initiative does not employ statistical sampling methods. The evaluation design, the analysis plan, and data collection procedures have determined the desired process data must be obtained from an array of documentary, existing file data, and key informant/stakeholder knowledge and judgment. The key informants and stakeholders are administrative and programmatic managers of the entities involved in the implementation of the Initiative and/or affected by its implementation. These entities include the Federal government, all the States, national partners such as NACQ, State cessation quitline service providers, and entities that partner with the quitlines.
The respondent universe is therefore those managers who can speak knowledgably about the Initiative implementation in relation to these entities. Consideration was initially given to a stratified sampling approach to selecting states to represent the universe of States. Expert input and Federal managers concurred, however, that the variety of State situations and attributes desirable for a stratified sample that would ensure full Sate representativeness along the lines required for the planned analysis and also reflecting the reality of the varied State circumstances would not be practical.
The attributes and circumstances include such constructs as whether the State had an existing quitline prior to the Initiative, the types of services offered by the quitline, the priority populations targeted by the quitline, State tobacco control budget, quitline budget, smoking prevalence, tobacco-related mortality and health care costs, and health disparity issues such as smoking prevalence by race, ethnicity, income, and age and private and public insurance coverage for tobacco cessation. Moreover, two of the Initiative’s primary process objectives were to establish quitlines in every state and to create a national toll-free telephone number providing access to each State’s quitline. Each state received funding to assist in establishing or strengthening its quitline. These and other factors led to the decision to seek respondents from all 51 States and D.C. Further input from experts and Federal managers identified three complementary, uniform classes of respondents for the States. One is the tobacco control manager, who would be knowledgeable about state tobacco control policy and funding, the overall tobacco landscape, and the relationship of the State tobacco program to Federal activities and funding, including but not limited to the Initiative. The second is the State tobacco quitline administrator, the State official who is directly responsible for overseeing the budget and operations of the quitline, securing and monitoring the provider of quitlines cessation services, monitoring the quitlines utilization and outcomes, and otherwise having direct programmatic knowledge of the quitline as it was involved and affected by the implementation of the Initiative. The third is the operations manager of the quitlines cessation services provider, which works with the state to determine appropriate cessation treatment protocols and levels of service, staffs the quitline, provides telephone and IT infrastructure, answers the quitline calls, provides counseling services, trains the counselors, provides referral and prescription services as needed, and monitors and reports on specific quality measures. There are nineteen such service providers serving the 51 State quitlines, including four states serves by NCI Cancer Information Service’s own quitline staff. This sampling plan is to do a census survey of each of these three types of respondent.
Another objective of the Initiative implementation is to enable and foster partnerships in the States between the quitline and entities that share a common or mutually beneficial interest. These potentially include, for example, public and private health care organizations, anti-tobacco advocacy groups, insurers, employers, media, other State tobacco-related programs, and affinity or advocacy groups for populations disproportionately affected by tobacco use or lacking equal access to tobacco cessation services. Each State’s partnering profile will be unique and the affect of the Initiative on partnering activities and the partners themselves will be highly varied. Thus, in addition to seeking the State respondents’ input about the affect of the Initiative on partnering, the data collection will also obtain the partners’ perspective. To identify partners, each State quitline administrator will be asked to nominate two partners for the Partner interview, based on their assessment of such criteria as the nature and scope of the partner’s involvement with the quitline, whether it is a new partnership fostered by the Initiative or an existing one influenced by it, and any unique or special aspects to the partner’s role. This purposely subjective approach is expected to yield the texture and diversity of partnering activities and relationships that will enable the analysis to characterize the influence of the Initiative on this key aspect of the overarching goal to promote and provide access to appropriate tobacco cessation services to the nation as a whole and those most in need.
Along with NCI and CDC, the North American Quitline Consortium has been an integral constituent of the planning and implementation process. Moreover, the Consortium’s members are the State quitlines. NAQC provides a forum for the interchange of ideas, is furthering the process of guidelines and standards for quality of services, and houses member workgroups focusing on operations, clinical standards, research, and promotion. Thus, the administrative managers and workgroup leaders are an essential source of information about the Initiative from the perspective of both the top-level planner/implementers and the constituency of quitlines that are the immediate object of the implementation process. The Consortium has a very small permanent staff; data will be collected from two administrators. In addition, data will also be collected from three workgroup member representatives, who will be selected so as not overlap with other respondent groups.
Research analysts will conduct semi-structured telephone interviews using instruments designed specifically for each of the five different respondent types to be surveyed (see Attachments 2A-2E). Staff will receive training in general interviewing techniques (e.g., contact procedures) and in the more specific skills used to collect both quantitative and qualitative data, including effective probing techniques and the recording of responses to open-ended questions. Each of the five unique instruments will be reviewed in detail with the interviewers. Audio recording of interviews is planned and will allow for more detailed analysis of responses provided if necessary.
Each instrument will target a distinct population: NAQC staff and members, State Tobacco Control Managers, State Quitline Administrators, State Quitline Service Providers and State Quitline Partners. With the exception of the Service Providers and Partners, respondents will be known members of the tobacco and quitline initiative community, and identified as interview respondents in advance. As described in Section B.2, State Quitline Administrators will be asked to nominate two partners, who will be contacted by telephone for the Partner interview. The Partners are widely varying types of organizations with a wide range of relationships to the quitlines, formal and informal. While it is not possible to predict all the partners for a given state or the two that each state may nominate, the following list describes the general types of organizations that may partner with the quitlines:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Similarly, the partner organizational structure may include public agencies, not-for-profit organizations, private companies, and coalitions/associations. The roles and relationships that exist between the partners and the quitline are even more varied than the types of partners – each role and relationship may be nearly unique. However, the following list suggests the range of the general types of roles and relationships that a partner may have with a quitline:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Several steps will be taken to reduce non-response and maximize response rates. NCI and CDC are holding regional meetings of the NNTCQ in May 2006; attendees will include all state tobacco control managers, many state quitline administrators, and most quitline service providers. The attendees will be informed of the planned research agenda and encouraged to participate. We will work with the North American Quitline Consortium to have the consortium formally endorse participation in the study. We will also ask state quitline administrators to contact their partners on our behalf prior to our attempts to interview them. Prior to contact attempts, a letter of information and encouragement will be sent to the respondents (see Attachment 3).
Interviewers will take proactive measures to ensure respondent’s schedules are accommodated and time used wisely. When contacting respondents, interviewers will be flexible enough to complete the interview during this contact or willing to schedule the interview for a time that works best for the respondent. A toll-free number will be provided to the respondents in case there is a need for a return call. We expect very high response rates: the respondents are members of the tobacco cessation community who are highly active and thoroughly engaged in this field; many are personally involved in the Initiative, through activities, meetings, memberships, and funding mechanisms. We will strive for a 100% response rate, and have reason to expect 90% or better.
The data collection method will employ executive telephone interviewing techniques that are standard for surveys of organizational managers and administrators. In contrast to many organizational surveys in which the most substantial challenge is identifying and locating a knowledgeable respondent, the Initiative process evaluation will have current individual names and contact information in nearly every instance. Accordingly, no test of data collection procedures is anticipated.
The evaluation Expert Panel includes a state tobacco control manager and a senior manager of a major quitline services provider. The instrument testing plan is to submit the appropriate instruments to them for review and comment, and to solicit input from a state partner identified through their auspices. The NAQC is a national partner of the NNTCQ and their Executive Director will review the NAQC instrument. . This approach combines the benefits of expert review with the perspective of the proposed respondent groups. Since the respondent group sizes are small and respondent types are highly specialized, this approach will be efficient and meaningful. This process is expected to help refine the data collection instruments in three ways. First, the experts will be in a position to advise about any question which the state informants could not provide a response because it does not reflect the way states operate and track their quitlines and tobacco control programs. While the questionnaires are the product of inputs from national NNTCQ partners, CDC guidance documents, NAQC survey instruments, the Evaluation Expert Panel, and extant literature, there is still a slight possibility that a small number of items will not be practical to be asked and will be removed. Second, a number of questions in all the instruments involve fairly comprehensive response category lists, intended to cover, to a practical extent, the wide diversity of situations and activities across the 51 states. The experts will advise on any significant omissions of relevant categories or of superfluous or irrelevant categories. If needed, the response categories will be adjusted accordingly. Finally, the experts will advise on the question and response wording in terms of accuracy, clarity, and standard terminology. If needed, the wording will be refined accordingly. It is also important bear in minds that experts themselves practitioners of the various roles and capacities of the respondent set for these instruments. In that sense, they effectively constitute a small pretest sample as well as expert reviewers.
The contractor collecting and analyzing the data is Westat. Westat’s design and analytical advisory subcontractors are the University of Illinois at Chicago, Institute for Health Research and Policy, and the University of Wisconsin Medical School, Center for Tobacco Research and Intervention.
The following individuals were critical in developing the research plan, the conceptual framework, telephone questionnaire items, and data collection strategies underlying the evaluation of the Initiative. Many of the same individuals will be involved with analysis of data once those data are collected.
National Cancer Institute
Mary Anne Bright, RN, MN
Director
Cancer Information Service
Acting Deputy Director, Office of Communications
National Cancer Institute, NIH, DHHS
Phone: 301.594.9048
Candace Deaton, MPA
Project Officer
Cancer Information Service
National
Cancer Institute, NIH, DHHS
Phone:
301.594.9072
Linda
Squiers, PhD
Project
Officer for Research
Cancer
Information
Service
National
Cancer Institute, NIH, DHHS
Phone:
301.594.9075
Centers for Disease Control and Prevention
Martha C. Engstrom, MS
Health
Scientist
Epidemiology Branch
Office on Smoking and Health
Centers
for Disease Control and Prevention
National Center for Chronic
Disease Prevention
and Health Promotion
Phone:
770.488.5749
Brick Lancaster, MA, CHES
Chief, Program Services Branch
Office on Smoking and Health
Centers
for Disease Control and Prevention
National Center for Chronic
Disease Prevention
and Health Promotion
Phone:
770.488.1091
Westat
Charles Carusi, PhD
Program Manager
Phone: 301.294.2859
Craig Love, PhD
Senior Evaluator
Phone: 240.314.2443
Lance Potter, MA
Senior Evaluator
Phone: 240.314.2363
Kristie Taylor, PhD
Senior Research Analyst
Phone: 301.517.4105
Simani Price, PhD
Senior Research Analyst
Phone: 301.610.5536
Susan Swain, MA
Senior Survey Methodologist
Phone: 301.294.3890
Meredith Grady, MPH
Qualitative Research Specialist
Phone: 240.453.2748
Cathy Cox, MPH
Research Analyst and Survey Specialist
Phone: 301.294.4443
University of Illinois at Chicago, Institute for Health Research and Policy
Susan J. Curry, PhD
Director
Institute for Health Research and Policy
University of Illinois at Chicago
Phone: 312.355.4438
University of Wisconsin Medical School, Center for Tobacco Research and Intervention
Michael C. Fiore, MD, MPH
Professor of Medicine and
Director, Center for Tobacco Research and Intervention
University of Wisconsin School of Medicine and Public Health
Phone: 608.262.8673
Paula Keller, MPH
Senior Policy Advisor
Center for Tobacco Research and Intervention
University of Wisconsin School of Medicine and Public Health
Phone: 608.262.4094
| File Type | application/msword |
| Author | curriem |
| Last Modified By | curriem |
| File Modified | 2006-10-16 |
| File Created | 2006-10-16 |