OMB Clearance Application:
Healthy People User Study
DRAFT: January 29, 2008
Office of the Assistant Secretary for Planning and Evaluation
TABLE OF CONTENTS
BackgrounD 1
1. Need and Legal Basis 1
2. Information Users 3
3. Use of Improved Information Technology 5
4. Duplication of Similar Information 6
5. Small Businesses 7
6. Less Frequent Collection 8
7. Special Circumstances 8
8. Federal Register Notice/ Outside Consultation 8
9. Payments/Gifts to Respondents 9
10. Confidentiality 9
11. Sensitive Questions 10
12. Burden Estimate (Total Hours & Wages) 10
13. Capital Costs (Maintenance of Capital Costs) 13
14. Cost to Federal Government 13
15. Program or Burden Changes 13
16. Publication and Tabulation Dates 13
17. Exemption for Display of Expiration Date 24
18. Exceptions to the Certification Statement 24
B. Collections of information employing statistical methods 24
1. Respondent Universe and Sampling Methods 24
2. Procedures for the Collection of Information 29
3. Methods to Maximize Response Rates and Deal with Nonresponse 30
4. Tests of Procedures or Methods Undertaken 32
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 32
The Office of Disease Prevention and Health Promotion (ODPHP) and the Office of the Assistant Secretary for Planning and Evaluation (ASPE), Department of Health and Human Services (HHS) are requesting Office of Management and Budget (OMB) approval to survey state, local, and tribal organizations about their use of the federal Healthy People initiative. ODPHP and ASPE solicited this data collection in task order request 07EASPE000044 and have contracted (HHSP233200700001T) with NORC at the University of Chicago to conduct this study. This has several purposes: 1) It will provide HHS with information regarding the utility of the Healthy People initiative and strategies for improving the usefulness of the initiative to state, local, and tribal organizations; 2) It will help to inform the assessment, development, and implementation of Healthy People 2020; and 3) The study will provide data to assist ODPHP in monitoring progress on ODPHP’s PART measure.
A. Justification
Healthy People 2010 is an important Federal initiative that establishes national health promotion and disease prevention goals. HP2010 represents the third of a series of publications by HHS that specifies ten-year health objectives for the nation. Its overarching goals are to increase the quality and years of healthy life and eliminate health disparities. HP2010 consists of 28 primary focus areas and 467 measurable health objectives designed to identify the most significant preventable threats to health and to establish public health priorities. The central theme of HP2010 focuses on the role of communities and community partnerships in promoting healthy living in the US. HP2010 is a powerful force in the effort to promote health and prevent disease in the U.S. The agenda reflects extensive consultation with over 350 national organizations, 250 state agencies, health experts, and the public.
In light of the tremendous collective energy that goes into developing the initiative, it is important to assess how the key target audiences are using Healthy People, and identify barriers to the initiative’s success. The goal of this assessment is to create a comprehensive picture of how HP2010 contributes to state, local and tribal disease prevention and health promotion planning. HHS is eager to document the utilization of HP2010, and to seek input from key users on how the next iteration of the initiative, Healthy People 2020, could be improved to encourage greater involvement. This study will identify examples of effective strategies and approaches to using HP2010, and, where possible, the short-term results of those efforts. Finally, the study will identify barriers to implementation and use at a point in time when HHS could take action to facilitate or support use in the forthcoming Healthy People 2020. The main research questions include:
What are the organizational characteristics of users and non-users of HP 2010?
Are organizations aware of HP 2010, and if so, how are the organizations using the initiative?
What are the reasons that organizations are not using HP 2010?
What components of HP 2010 are most useful to users?
How will users make a final assessment of progress towards the goals of Healthy People 2010?
What key components should be considered in framing the next iteration of health promotion and disease prevention objectives for the nation?
ASPE/ODPHP are seeking OMB approval to conduct a short survey using a self-administered questionnaire of state, local, and tribal health organizations. The survey will be administered through mail and respondents will have the option to complete the survey as a web-based electronic survey. The sample size for all respondent groups is 502.
Once the data are collected, ASPE/ODPHP expect to conduct a limited number of key informant interviews with no more than nine respondents to learn in greater detail how the Healthy People initiative can best position future activities to support disease prevention and health promotion efforts at the state, local and tribal levels.
This collection of data is authorized by Section 301 of the U.S. Public Health Service Act (42 U.S.C.241). A copy of this legislation can be found in Appendix 1.
Though information about HP 2010 has been disseminated through various mechanisms including websites, published materials, and other efforts, a 2005 Assessment of the Uses and Users of Healthy People 2010 and HealthierUS conducted by NORC is the most substantive source of information about the uses and users of HP2010 available. There is little additional information available about how organizations are using HP2010 concurrently with existing programs to improve health. This study seeks to investigate characteristics of organizations that use and do not use HP2010 to generate information about improving the utility of the initiative. HHS will use the information gleaned from this study to make crucial planning decisions in light of its work on the next decade’s health objectives, as well as highlight effective strategies that can assist the community in working towards the nation’s disease prevention and health promotion goals.
HHS has used the results of the 2005 Assessment in the initial planning for the next iteration of the Healthy People initiative, Healthy People 2020 (HP2020). HHS has also used the results of the 2005 Assessment to strengthen current outreach and assistance work with state, local, and tribal entities on HP2010. One example was the finding in the 2005 Assessment that Healthy People State Coordinators desired greater interaction with their HHS Regional Health Administrators (RHA). As a result of this finding, HHS has implemented strategies to convene Healthy People State Coordinators with their RHAs, such as at the annual Healthy People State Coordinators’ meeting. Results from the 2005 Assessment also informed the ODPHP OMB PART performance measure “percentage of states that use national objectives in their health planning process.”
Having data on the use of HP2010 from a broad sample of governmental health entities allows HHS to take a more strategic approach to the design, dissemination, and implementation of HP2020. The current study is of greater significance to HHS than the 2005 User Assessment because the initiative has been in the field for a substantial portion of the decade, and the Healthy People initiative changed as a result of the Healthy People midcourse review. The current study will provide HHS with perspective on if and how the use of Healthy People has changed since the midcourse revisions. The data collected during this study will also be useful to OMB and HHS in continuing to monitor ODPHP’s progress toward reaching its PART measure target. Finally, the results of the current study will be useful to policymakers as they will clarify which aspects of the Healthy People initiative are useful to constituent groups and perhaps identify areas for augmentation or policy development.
The survey will be sent to respondents as a hardcopy self-administered questionnaire (SAQ) with the option of completing the survey online via the internet. It is anticipated that respondents will choose the option of least personal burden, thereby reducing the overall burden of the study. A postcard reminder will be sent to any non-respondents two weeks after the initial mailing, highlighting the convenience of the online completion option. Any outstanding non-respondents at four weeks after the initial mailing will be contacted using computer-assisted telephone interviewing (CATI) to confirm that the hardcopy SAQ was received, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the hardcopy SAQ and indicates a preference for hardcopy completion, NORC will mail or fax the respondent a new hardcopy SAQ. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey. It is estimated that 60 percent of respondents will require a follow up telephone call, and 15 percent of respondents will opt to complete the survey via the web. Once a mailed or faxed copy of the survey has been received at NORC, the data will be directly entered into the electronic survey and the data will be maintained electronically.
NORC conducted a literature review, and the search did not identify any systematic evaluation of types of users and uses of HP2010 other than the 2005 Assessment of the Uses and Users of Healthy People 2010 and HealthierUS conducted by NORC. Other literature indicates a commitment to the goals of HP2010 from a diverse set of organizations, but does not provide more than scattered descriptions of organizational efforts toward a HP2010 objective.
NORC’s 2005 Assessment (OMB No. 0990-0276) found that overall, 83 percent of the respondent organizations were aware of HP2010. All of the responding states, 84 percent of the local health organizations, and 60 percent of the tribal health organizations reported awareness, with tribes statistically less likely to be aware of the initiative than both local and state health organizations. Seventy-one percent of the 189 organizations aware of HP2010 reported using it in their organization. One-hundred percent of the responding states reported using the initiative compared to 65 percent of local organizations, and 48 percent of tribes. The 2005 User Assessment also devoted a significant portion of its questionnaire to assessing the uses and users of HealthierUS.
The 2005 Assessment established the groundwork for the current study. While the 2005 Assessment provided HHS with valuable information about users and uses of both Healthy People and HealthierUS, the results from that study have greatly influenced the development of the current study to ensure that HHS obtains important new information. For example, although the tribal response rate was good for a survey of its type, the 2005 tribal sample was not large enough to draw statistically significant conclusions on several important findings. In the current study, the tribal sample is doubled to ensure that findings will be of statistical significance, increasing the precision and reliability with which findings on the tribal organizations’ use can be reported. The 2005 Assessment also found that 100 percent of the responding states reported using the initiative. This may suggest that the individuals responding on behalf of the state organizations were almost entirely State Healthy People Coordinators, who would be expected to respond affirmatively to being aware of and using Healthy People as it is a primary function of their jobs. In the current study, two distinct groups of state-level respondents (State Healthy People Coordinators and Directors of Chronic Disease Programs) will be explicitly targeted as samples as they are expected to be familiar and not be familiar with the initiative, respectively. This approach will determine whether knowledge of the initiative is role-based or organization-based at the state level. Several years have elapsed between the data collection for the 2005 Assessment and the data collection that will occur in the current study. It is important that HHS ascertain whether the midcourse review process affected the use of Healthy People, as well as to monitor if and how use has changed during this time. The current study will also allow continued monitoring of ODPHP’s OMB PART measure related to the percentage of states using national health objectives in their health planning processes. The current study will also be less burdensome to respondents as the questionnaire will focus on only Healthy People and will not include corresponding questions regarding HealthierUS as in the 2005 Assessment.
No small businesses will be involved in this study.
The design of this study requires only one data collection activity per respondent. Without collecting this data, HHS will not have access to a comprehensive assessment of the level and types of involvement from the target audiences of HP2010. The federal government will find enormous benefit in having information available that will answer the questions about how, where, and for whom their public health initiative is being used. Additionally, without this data collection, HHS will not have an enumeration of the activities planned by these key target audiences to assess progress towards HP2010 goals and objectives at the end of the decade and input from these groups on activities related to Healthy People 2020. Finally, this data collection will allow the continuation of ODPHP’s OMB PART measure based on the percentage of states currently using HP2010.
There are no legal obstacles to reduce the burden of collection.
This request complies with the information collection guidelines of 5 CFR 1320.5(d)(2).
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on $( ).$ For Federal Register information, see the Office of the Secretary Certification Form. In addition, we have consulted with the Indian Health Service related to data collection from tribal entities.
NORC at the University of Chicago staff consulted include (full contact details for these individuals can be found in Section B.5 of this document):
Daniel S. Gaylin, MPA
Caitlin Oppenheimer, MPH
Stephen Pedlow, MS
Angela Debello, MA
The NORC Institutional Review Board
Indian Health Service representatives consulted include:
Doug Black
Philip Smith
Hankie Ortiz
There will be no payments or gifts to respondents.
Data will be treated in a confidential manner, unless otherwise compelled by law. Personal identification information (i.e., respondent names) will not be collected in the survey instrument and the unit of sampling is the organization, not the individual. Although the individual will be asked to report his/her position and organization name, this information will be used solely by NORC to categorize and summarize types of respondents for comparison purposes during the analysis phase of the project. Specific information linking organization name and the respondent’s job title to particular survey responses will not be included in any information viewed by ODPHP, ASPE, or any other HHS officials. Further, the study’s briefs and report will not identify any specific organizations. Respondents will be informed in the survey’s cover letter that members of the federal government will not view information on job title. All potentially identifying information will be destroyed at the study’s conclusion.
The surveys will not include any questions of a sensitive or personal nature. Respondents will be asked to answer from the perspective of their organization about particular aspects of the government programs, as well as the respondents’ opinions of different aspects of HP2010. The questions are designed to solicit information solely regarding uses of the initiative in a professional/worksite setting.
Estimated Annualized Burden Hours
In Exhibit 1, we provide estimates of the collection burden on participants from each of the four samples for this effort. Study participants from each sample will participate in data collection one time only, responding via a self-administered mailed questionnaire, completing the questionnaire via an online option or over the telephone with the assistance of a computer-assisted telephone interviewer (CATI). The data collection instrument is the same form for respondents from all four samples. Hour burden estimates were derived using the 2005 Assessment questionnaire as a baseline, and will be verified during the pilot/pretesting of the survey instrument to be conducted during the OMB review period.
Exhibit 1. Estimated Burden Hours
Type of Respondent |
# of Respondents |
No. Responses per Respondent |
Average Burden Per Response (Hours) |
Total Burden Hours |
State Healthy People
Coordinators |
51 |
1 |
15/60 |
13* |
State Chronic Disease
Program Directors |
51 |
1 |
15/60 |
13* |
Local Health Organizations (Frame B) |
300 |
1 |
15/60 |
75 |
Tribal Health Organizations (Frame C) |
100 |
1 |
15/60 |
25 |
TOTAL |
502 |
|
|
126* |
*Numbers have been rounded.
Annualized Cost to Respondents
Exhibit 2. Estimated Burden Cost
Type of Respondent |
Total Burden Hours |
Average Hourly Wage Rate1 |
Total Hour Cost |
State Healthy People
Coordinators |
13* |
$31.54 |
$410.02 |
State Chronic Disease
Program Directors |
13* |
$31.54 |
$410.02 |
Local Health Organizations (Frame B) |
75 |
$31.54 |
$2,365.50 |
Tribal Health Organizations (Frame C) |
25 |
$31.54 |
$788.50 |
TOTAL |
126 |
$31.54 |
$3,974.04 |
1 Based on hourly wage for Administrators and officials, public administration, “National Compensation Survey: Occupational Wages in the United States, 2005,” U.S. Department of Labor, Bureau of Labor Statistics. Extracted November 20, 2007 from www.bls.gov.
*Numbers have been rounded.
Data collection for this study will not result in any additional capital, start-up, maintenance, or purchase costs to respondents or record keepers. Therefore, there is no burden to respondents other than that discussed in the previous section (A.12).
All costs for conducting the Healthy People User’s Study are included in the contract between the Department of Health and Human Services and NORC under contract number HHSP233200700001T. The total estimated cost is $390,382.00 over a sixteen month period to conduct the surveys, analyze and present findings, and write a final report. This is an annualized cost of $292,786.50.
This is a new collection of data.
The data collected in this survey will be analyzed and interpreted to produce preliminary and final briefings as well as a final study report to the Department of Health and Human Services. NORC will deliver the final report to ASPE in hardcopy and a print-ready electronic format. Publication of findings on the internet is at ASPE’s discretion. The remainder of this section discusses data sources and the analytic techniques that will be employed. Information will be collected over a three- to four-month period following OMB approval. Exhibit 3 provides a schedule of data collection, analysis, and reporting following OMB approval.
Exhibit 3. Timetable for Data Collection, Analysis, and Publication
Activity |
Expected Date of Completion |
Survey sent to respondents and data collected |
1-4 months following OMB approval |
Data analysis |
4-5 months following OMB approval |
Preliminary briefing and preparation of draft report |
6-7 months following OMB approval |
Final report |
7-8 months following OMB approval |
Final briefing |
9 months following OMB approval |
This assessment includes one mailed self-administered questionnaire (SAQ), which will be sent to members of state, local, and tribal health organizations. Each individual will be asked to complete the one-time survey, expected to take approximately 15 minutes. Respondents will have the option of completing the survey online. Results will be summarized within and across organization type. To facilitate return of the surveys and ensure high response rates, we are providing respondents with pre-addressed and stamped envelopes. A postcard reminder will be sent to any non-respondents two weeks after the initial mailing, highlighting the convenience of the online completion option. Any outstanding non-respondents at four weeks after the initial mailing will be contacted using computer-assisted telephone interviewing (CATI) to confirm that the hardcopy SAQ was received, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the hardcopy SAQ and indicates a preference for hardcopy completion, NORC will mail the respondent a new hardcopy SAQ. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.
The surveys are designed to ascertain how state, local, and tribal health organizations use HP2010. The surveys also seek to understand how state, local, and tribal health organizations perceive the utility of HP2010. The questionnaire consists of three sections, which are outlined below.
Background. Captures data about organizational characteristics such as type, size, and health priorities of organization, as well as the job title of the respondent.
Uses of HP2010. Captures data about whether the organization uses HP 2010, how they use the initiative, factors that enable or hinder its use within the organization. This section collects information about end of the decade activities and plans for the future.
HP2010: Non-users. Captures data from respondents that report their organization does not use HP 2010 on why they do not use the initiative, barriers to use, and ascertains general perceptions about the initiative.
The survey is included as Attachment 1.
This section details the tabulations and statistical analyses that will be conducted for this study. This study will use both univariate and, where possible, multivariate techniques to analyze the data.
Data analysis will focus on identifying results of the established key research questions. In addition to answering this core set of questions, the analysis will also compare the groups and determine the extent to which certain characteristics of the organization seem to be related to the extent of awareness, the extent of use, the nature of use, and the kinds of barriers experienced. Exhibit 4 lists the key research questions and sub-questions.
Exhibit 4: Key Research Questions
|
|
|
|
|
|
|
|
|
|
6. What key components should be considered in framing the next iteration of health promotion and disease prevention objectives for the nation? |
|
Both descriptive and inferential statistics, such as the standard t-test, chi-square test, and multiple comparison procedures will be utilized in the analysis. Standard errors will also be provided for these estimates. Non-parametric statistical techniques may also be used to analyze the data, including the chi-square test for cross tabulations, the Wilcoxon rank-sum (Mann-Whitney) two-sample test, and the Komolgorov-Smirnov test for equality of distributions. Nonsampling errors arising from unit and item nonresponse will be dealt with through weighting and imputation where appropriate.
The remainder of this section presents specific analyses that will be conducted to answer the research questions for each initiative.
1) What characteristics are associated with users and nonusers?
To determine the characteristics associated with users and non-users of HP2010, chi-square tests of association between organizational characteristics, opinions, and use of the initiatives will be conducted. The analyses will focus on identifying aspects of the program which HHS could change that would have the greatest impact on organizations most likely to utilize the program.
2) What is the extent of the awareness of HP2010 and how is HP2010 being used? Has the use of HP2010 changed since the 2005 User Assessment?
Ascertaining the awareness level of the initiatives is a main goal of the assessment. The main statistical technique used in analyses will examine the proportion of respondents that indicate awareness of the initiatives. Simple univariate statistics will examine the data overall, and chi-square tests of association or student’s t-tests will be used to compare data among and between respondent groups.
By comparing responses between different kinds of organizations (using data obtained from the background section of the survey), it will be possible to identify characteristics of organizations that require additional outreach. Univariate statistics will be used to assess awareness of HP2010 in each of the four respondent groups.
There are many ways that organizations could use HP2010, and HHS has anecdotal evidence from many organizations as well as the results of the 2005 User Assessment. This survey will provide an opportunity to further document utilization of HP2010 in a uniform manner. Several questions on the survey relate to gaining information about how organizations utilize the program. Initial questions will establish whether the organization uses HP2010, and subsequent questions seek to catalog how it is being used. Wherever possible, answer options have been narrowed as possible responses in order to minimize the burden on the respondent. Additional questions that will be assessed relate to how users interact with the program (e.g., through the publications or website), the frequency with which HP2010 is used as a resource, and how organizations use the initiative to measure health outcomes.
Descriptive statistics will be used to identify how the program is being used, and chi-square tests will be used to determine if HP2010 is used similarly across respondent categories and organizational characteristics (i.e., comparisons across the four respondent groups may be made as well as different organizational sizes within a respondent category). We will also use logistic regression, where appropriate, to determine if organizational characteristics are associated with the likelihood of using the initiative in specific ways.
We will also make comparisons between the results of the 2005 Assessment and this 2007 User Study. Most of these analyses will exclude the new group of state Directors of Chronic Disease Programs. Descriptive statistics will be used to compare how the program usage has changed, and chi-square tests and t-tests will be used to determine if these differences are statistically significant. If appropriate, logistic regression and other advanced statistical tools will be used to better understand the changes between the two data sets.
3) What components of HP2010 are most useful to users?
HP2010 has several components, including: the books, which summarize the focus areas and identify the nation’s health objectives; the data templates, which identify current health measures by race, ethnicity and sex subpopulations and sources for data tracking; and the companion documents which assemble health objectives specific to a particular target audience. HHS is interested in learning which aspects of the initiative are considered useful by key target users. An assessment of the value each component brings to the overall program will help direct resources and effort.
Simple descriptive statistics will be used to identify the elements of HP2010 that are considered most useful, and chi-square tests of association will be used to compare these opinions and organizational characteristics and the level of organizational use of the programs. Analyses will also assess the perceptions of HP2010 by key target users, in terms of the program’s relevance to the organization’s own work, and its use in achieving the organization’s health objectives. These questions ask respondents to rate, on a scale of 1-5, how relevant and effective the initiative is for their organization. Mean scores will be computed and compared among different organizational characteristics using the student’s t-test, which assumes normally distributed data. An alternative non-parametric test (with no accompanying normality assumption) that will be used is the Wilcoxon signed rank test. Analyses will be conducted to examine possible correlation between overall opinion of the program and utilization of the program.
In addition to the closed-ended questions on the surveys related to this issue, a limited number of open-ended questions have been included to allow respondents maximum flexibility in making suggestions to improve the program without biasing the responses. Responses to these questions will be reviewed and common responses will be grouped and categorized for assessment.
4) How will users make a final assessment of progress towards the goals of Healthy People 2010?
As the 2010 target year approaches, HHS must consider how to make a final assessment of the HP2010 program. By gathering information on the types and methods of assessment occurring at the state, local, and tribal levels on use of HP2010, HHS may be able to devise appropriate strategies for an overall measurement of the program’s impact. Any final assessment of Healthy People 2010 will likely involve a combination of the same statistical analyses mentioned above: descriptive analyses, chi-square tests, t-tests, and possibly logistic regression and other advanced statistical methods.
5) Among nonusers, why is HP2010 not being used and what changes would help organizations to use it more? Among organizations that do use HP2010, what changes can be made to encourage further utilization?
Gaining insight into how HHS can reduce barriers to utilization of HP2010 and encourage greater participation and action towards their goals is a key objective for this project. Each of the four respondent groups is a key target user of the initiative. For organizations that indicate they do not use HP2010, this project provides the opportunity to understand why organizations do not utilize the program in anticipated ways. Descriptive statistics will be used to explore possible program and organizational causes for non-use of the initiative. Among users of HP2010, descriptive statistics will assess reasons that prevent them from expanding their use of the initiative.
The limited number of open-ended questions that seek suggestions for improving the program will be examined and categorized where possible.
6) What key components should be considered in framing the next iteration of health promotion and disease prevention objectives for the nation?
HHS is particularly interested in gaining information from on-the-ground users as to how the Healthy People initiative can be improved in light of the current development work on the forthcoming Healthy People 2020 initiative. By increasing the usefulness and utility of the next Healthy People to the state, local, and tribal entities, HHS can increase the usage of the program to improve the health of the nation.
ASPE does not seek this exemption. All data collection materials will display the OMB expiration details.
There are no exceptions to the certification statement.
Surveys are being administered to collect information about how four groups of target-users, state (two groups), local, and tribal health departments, use and perceive Healthy People. The surveys will also collect information from non-users to determine the factors that prevent target groups from using the initiative in their organizations.
The results will be generalizable to the respondent universe, which consists of government entities that interact with HHS and their constituents to improve the health of the populations they serve.
The sample will include 502 organizations from the 50 states, the District of Columbia, and Native American tribes. The unit of analysis for the sample will be the organization, meaning that no more than one survey will be sent to each organization, although this sample treats state Healthy People State Coordinators as separate organizations from state Directors of Chronic Disease Programs. The project will census state health departments (Healthy People State Coordinators and Directors of Chronic Disease Programs separately), and sample local and tribal health organizations. This sample of public health officials will be able to provide the type of data necessary to evaluate the Healthy People initiative.
The sample frame will be constructed from multiple sources and will result in four separate lists for Healthy People State Coordinators, state Directors of Chronic Disease Programs, local health departments, and tribal nations. A list of the 51 State Healthy People Coordinators will serve as the primary contacts for the states (sample frame A). The list of state chronic disease directors will serve as the second set of state-level contacts (sample frame A*). The list of approximately 3,000 members of the National Association of County and City Health Officials (NACCHO) will serve as the sample frame for the local officials (sample frame B), and the tribal officials will be selected from a list of approximately 400 tribal health leaders provided by the Indian Health Service (sample frame C).
These frames will be used to draw samples that satisfy the study’s goals. The proposed sample design satisfies two key requirements. First, all 51 organizations from frames A and A-1 will be included with certainty. Second, the design will draw samples that produce nationally representative estimates for urban and rural organizations in group B, and nationally representative estimates by tribal size and region in group C.
Our total sample of 502 organizations will consist of all 102 organizations from frames A (State Healthy People Coordinators) and A* (State Chronic Disease Program Directors), plus 400 more sampled from frames B (Local Health Organizations) and C (Tribal Health Organizations). We will include in the sample 300 organizations from frame B and 100 organizations from frame C.
Exhibit 5 shows the sizes of the frames provided, as well as the sample sizes and expected response rates and respondent sizes.
Exhibit 5: Sample Statistics
|
Population |
Sample |
Expected Response Rate |
Total Expected Respondent (n) |
State
Healthy People Coordinators |
51 |
51 |
90% |
46 |
State
Chronic Disease Program Directors |
51 |
51 |
80% |
41 |
Local Health Organizations (Frame B) |
~3,000 |
300 |
80% |
240 |
Tribal Health Organizations (Frame C) |
~400 |
100 |
75% |
75 |
TOTAL |
~3,500 |
502 |
80% |
402 |
Selection Methods
For sample frame B and C we will use systematic samples with equal probability of selection (within frame) and implicit stratification. The only difference is on which variables will be used for implicit stratification. Implicit stratification involves sorting the frame on certain variables so that the sample drawn is representative on that variable. For example, assume that 44% of local health organizations are in a rural setting and 56% are in urban settings. By sorting on urban-rural status and then drawing a systematic sample, the resulting sample will be very close to including 44% of organization in a rural setting.
We will sort on multiple variables so that samples will be representative on more than one dimension. The variability in sample size percentages will increase for variables that appear later in the sorting. Serpentine sorting will be used when sorting on multiple variables to maximize the effect of the stratification. Serpentine sorting involves sorting by an order that is alternately increasing or decreasing. For example, serpentine sorting on urban/rural status and region could result in this sort order: Rural Northeast, Rural Midwest, Rural South, Rural West, Urban West, Urban South, Urban Midwest, and Urban Northeast. This sort order successfully keeps the two West strata together.
It should be noted that the level of precision for subgroup estimates may not be sufficient to make meaningful comparisons between frames. To account for this imprecision, we will employ strategic collapsing of strata in estimation to create estimates with a higher level of precision. For example, the urban groups and rural groups may be collapsed to form nationally representative estimates of urban and rural areas.
Local Health Organizations
The NACCHO list frame consists of approximately 3,000 records. However, we will remove any “inappropriate” records (e.g., “tribal” records) so that our sampling frame contains only local health organizations. Inappropriate records to be deleted include duplicate records, records without title or agency name, as well as other inappropriate records such as public health consultants, foundations, special interest groups (for hand gun violence, for example), students, professors, etc.
Since it is desired to have a representative sample with respect to urban and rural organizations, we will sort the file first on urban/rural status. Using the zip code from the file, we will map each organization to the state and county in which it resides. We will then determine if this county is inside a Census defined Metropolitan Statistical Area (MSA) or not. The Census Bureau defines MSAs as the counties that involve economic activity related to a central city. If the county is in an MSA, we will count this organization as “urban.” Otherwise, we will classify the organization as “rural.” Suburban organizations will be classified as “urban.”
Tribal Organizations
The target respondent is the lead tribal health representative, meaning the person within the tribe who has the authority and responsibility for disease prevention and health promotion activities. The tribal list frame consists of approximately 450 records. This file will also contain a code for the approximate size of Indian population excepting urban Indian health agencies and a few other organizations. The tribes are divided into small (< 2,500 Indian population), medium (2,500 – 10,000), and large (> 10,000). The tribal health agencies with unknown population size will be placed into a fourth category.
To ensure a representative mix of small, medium, and large tribes, we will sort the file on this size code. To the extent that tribal organizations are geographically diverse, we will draw the sample to be as representative as possible by sorting on Census Region.
The sample will include 502 organizations from state, local, and tribal organizations. The unit of analysis for the survey will be the organization, so that no organization will be asked to complete more than one survey, although this sample treats state Healthy People State Coordinators as separate organizations from state Directors of Chronic Disease Programs. Fielding of the survey will entail mailing surveys, along with a cover letter, to the key staff member at each organization. A self-addressed stamped envelope will be included with each survey so survey respondents can return the survey directly to the researchers. Respondents will also be offered the option to complete the survey online via the internet. A postcard mailing will be sent to respondents two weeks after the initial mailing, and a phone call will be made to those who have not responded after four weeks. The phone call will also provide an opportunity for the researchers to remail questionnaires that have been lost or misplaced, or to access the respondent’s case online and enter his or her responses over the telephone.
Project investigators will use an electronic receipt control system using case ID numbers to track the initial questionnaire mailing, address updates, remailing of questionnaires, mailing of postcard reminders, and complete and incomplete questionnaire returns. Reports from the system will identify the sample members which require prompting for completion of the survey.
All data from the completed questionnaires will be keyed (data entered) to create the analytic data file. Ten percent of the questionnaires will be randomly selected for keying a second time (double entry). The accuracy of the data entry process will be verified by comparing the data from the first entry with the data from the second entry. The double keying verification process will allow researchers to report the rate of accuracy to the Project Officer. The questionnaires will be processed in two batches. Data entry of the second and final batch will be completed within two weeks of the close of data collection.
The investigators will use a number of proven methods to maximize participation in the study. First, the instrument itself is designed to maximize response rates. The style of the survey is inviting and user friendly, with a maximum of 38 questions. The instructions for the survey are straightforward, and there are a limited number of skip patterns. The questionnaire will be pilot tested with six respondents from the sampling frame, and questions will be amended to reflect suggested improvements from these respondents. In addition to the questionnaire, each respondent will receive a cover letter encouraging participation in the survey. The cover letters (Attachments 2 and 3) will convey the importance of the survey to the ODPHP and the ASPE. The cover letters will also indicate that the respondent will not be identified to any government agency. The survey will be sent to respondents as a hardcopy SAQ with the option of completing the survey online via the internet. Completed hardcopy surveys may be returned via the included self-addressed, stamped envelope or by fax transmittal. It is anticipated that respondents will choose the option of least personal burden, thereby reducing the overall burden of the study. A postcard reminder will be sent to any non-respondents two weeks after the initial mailing, highlighting the convenience of the online completion option. Any outstanding non-respondents at four weeks after the initial mailing will be contacted using computer-assisted telephone interviewing to confirm that the hardcopy SAQ was received, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the hardcopy SAQ and indicates a preference for hardcopy completion, NORC will fax and/or mail the respondent a new hardcopy SAQ. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.
In the 2005 Assessment of the Users Healthy People 2010 and HealthierUS and, these same procedures were used with the same respondent population with good success. The response rate for the state Healthy People Coordinators group was 86%, the local sample has a response of 76%, and the tribal sample had a response of 73%. Overall the response to the survey was 78%. Given a greater familiarity with web-based surveys and the increased attention on the Healthy People program due to planning activities for 2020, we believe high response rates will be achieved. NORC is also conducting advance outreach work with IHS to facilitate higher response rates from tribal organization and will work with HHS to publicize the survey at regional stakeholder meetings.
A pilot test of six individuals will be conducted during the initial OMB review period.
The following individuals contributed to the questionnaire and study design and will be involved in the interpretation and analysis of findings:
Daniel Gaylin, MPA
Executive Vice President
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9417
Caitlin Oppenheimer, MPH
Principal Research Scientist
Department of Health Policy and Evaluation
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9322
Angela DeBello, MA
Associate
Director
Department of Public Health and Epidemiology
NORC at the University of Chicago
55
East Monroe Street
Chicago, IL 60603
312-759-4069
Steven Pedlow, MS
Senior Survey Statistician
Department of Statistics and Methodology
NORC at the University of Chicago
55
East Monroe Street
Chicago, IL 60603
312-759-4084
The government project officer for this study is:
Wilma Tilson, MPH
Health Policy Analyst
US Department of Health and Human Services
Office of Assistant Secretary of Planning and Evaluation
Office of Health Policy
200 Independence Ave, SW
Room 447 D
Washington, DC 20201
Attachment 1
Survey Questionnaire
Attachment 2
Cover letter to state health officials
ASPE Letterhead
[Date]
[Name and address of state health director]
[Dear . . .]
I am writing to request your participation in a study on Healthy People, the Federal initiative to improve the health of Americans through the promotion of disease control and prevention activities. The offices of the Assistant Secretary for Planning and Evaluation (ASPE) and of Disease Prevention and Health Promotion (ODPHP) at the U.S. Department of Health and Human Services (HHS) are conducting an evaluation of the Healthy People program.
Your organization’s participation in this study is critical to the success of this evaluation, and will provide HHS with important information on ways to improve the initiative and promote specific strategies to prevent disease and improve health at the state, tribal, and local levels. The information you provide on this survey will be held strictly confidential. Your organization’s identity will be separated from the responses to the survey. The information gathered will be used solely by ASPE and ODPHP, or its representatives for research, and will not be disclosed or released to other persons for any purpose except as required by law.
If you have any questions, please feel free to call NORC at [toll-free number], or Caitlin Oppenheimer, MPH, the NORC Project Director, at (301) 634-9322.
Thank you in advance for your participation.
Sincerely,
Ben Sasse
Assistant Secretary for Planning and Evaluation
Attachment 3
Letter to local and tribal heatlh officials
ASPE Letterhead
[Date]
[Name and address of local/tribal health organization contact]
[Dear . . .]
I am writing to request your participation in a study on Healthy People, the Federal initiative to improve the health of Americans through the promotion of disease control and prevention activities. The offices of the Assistant Secretary for Planning and Evaluation (ASPE) and of Disease Prevention and Health Promotion (ODPHP) at the U.S. Department of Health and Human Services (HHS) are conducting an evaluation of the Healthy People program.
Your organization’s participation in this study is critical to the success of this evaluation, and will provide HHS with important information on ways to improve the initiative and promote specific strategies to prevent disease and improve health at the state, tribal, and local levels. The information you provide on this survey will be held strictly confidential. Your organization’s identity will be separated from the responses to the survey. The information gathered will be used solely by ASPE and ODPHP, or its representatives for research, and will not be disclosed or released to other persons for any purpose except as required by law.
If you have any questions, please feel free to call NORC at [toll-free number], or Caitlin Oppenheimer, MPH, the NORC Project Director, at (301) 634-9322.
Thank you in advance for your participation.
Sincerely,
Ben Sasse
Assistant Secretary for Planning and Evaluation
Appendix 1
Section 301 of the Public Health Service Act (42 U.S.C.241)
TITLE 42 > CHAPTER 6A > SUBCHAPTER II > Part A > § 241
§ 241. Research and investigations generally
(a) Authority of Secretary
The Secretary shall conduct in the Service, and encourage, cooperate with, and render assistance to other appropriate public authorities, scientific institutions, and scientists in the conduct of, and promote the coordination of, research, investigations, experiments, demonstrations, and studies relating to the causes, diagnosis, treatment, control, and prevention of physical and mental diseases and impairments of man, including water purification, sewage treatment, and pollution of lakes and streams. In carrying out the foregoing the Secretary is authorized to—
(1) collect and make available through publications and other appropriate means, information as to, and the practical application of, such research and other activities;
(2) make available research facilities of the Service to appropriate public authorities, and to health officials and scientists engaged in special study;
(3) make grants-in-aid to universities, hospitals, laboratories, and other public or private institutions, and to individuals for such research projects as are recommended by the advisory council to the entity of the Department supporting such projects and make, upon recommendation of the advisory council to the appropriate entity of the Department, grants-in-aid to public or nonprofit universities, hospitals, laboratories, and other institutions for the general support of their research;
(4) secure from time to time and for such periods as he deems advisable, the assistance and advice of experts, scholars, and consultants from the United States or abroad;
(5) for purposes of study, admit and treat at institutions, hospitals, and stations of the Service, persons not otherwise eligible for such treatment;
(6) make available, to health officials, scientists, and appropriate public and other nonprofit institutions and organizations, technical advice and assistance on the application of statistical methods to experiments, studies, and surveys in health and medical fields;
(7) enter into contracts, including contracts for research in accordance with and subject to the provisions of law applicable to contracts entered into by the military departments under sections 2353 and 2354 of title 10, except that determination, approval, and certification required thereby shall be by the Secretary of Health and Human Services; and
(8) adopt, upon recommendations of the advisory councils to the appropriate entities of the Department or, with respect to mental health, the National Advisory Mental Health Council, such additional means as the Secretary considers necessary or appropriate to carry out the purposes of this section.
The Secretary may make available to individuals and entities, for biomedical and behavioral research, substances and living organisms. Such substances and organisms shall be made available under such terms and conditions (including payment for them) as the Secretary determines appropriate.
(b) Testing for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects; consultation
(1) The Secretary shall conduct and may support through grants and contracts studies and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects. In carrying out this paragraph, the Secretary shall consult with entities of the Federal Government, outside of the Department of Health and Human Services, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct for such entity studies and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects.
(2)
(A) The Secretary shall establish a comprehensive program of research into the biological effects of low-level ionizing radiation under which program the Secretary shall conduct such research and may support such research by others through grants and contracts.
(B) The Secretary shall conduct a comprehensive review of Federal programs of research on the biological effects of ionizing radiation.
(3) The Secretary shall conduct and may support through grants and contracts research and studies on human nutrition, with particular emphasis on the role of nutrition in the prevention and treatment of disease and on the maintenance and promotion of health, and programs for the dissemination of information respecting human nutrition to health professionals and the public. In carrying out activities under this paragraph, the Secretary shall provide for the coordination of such of these activities as are performed by the different divisions within the Department of Health and Human Services and shall consult with entities of the Federal Government, outside of the Department of Health and Human Services, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct and support such activities for such entity.
(4) The Secretary shall publish a biennial report which contains—
(A) a list of all substances
(i) which either are known to be carcinogens or may reasonably be anticipated to be carcinogens and
(ii) to which a significant number of persons residing in the United States are exposed;
(B) information concerning the nature of such exposure and the estimated number of persons exposed to such substances;
(C) a statement identifying
(i) each substance contained in the list under subparagraph (A) for which no effluent, ambient, or exposure standard has been established by a Federal agency, and
(ii) for each effluent, ambient, or exposure standard established by a Federal agency with respect to a substance contained in the list under subparagraph (A), the extent to which, on the basis of available medical, scientific, or other data, such standard, and the implementation of such standard by the agency, decreases the risk to public health from exposure to the substance; and
(D) a description of (i) each request received during the year involved—
(I) from a Federal agency outside the Department of Health and Human Services for the Secretary, or
(II) from an entity within the Department of Health and Human Services to any other entity within the Department,
to conduct research into, or testing for, the carcinogenicity of substances or to provide information described in clause (ii) of subparagraph (C), and (ii) how the Secretary and each such other entity, respectively, have responded to each such request.
(5) The authority of the Secretary to enter into any contract for the conduct of any study, testing, program, research, or review, or assessment under this subsection shall be effective for any fiscal year only to such extent or in such amounts as are provided in advance in appropriation Acts.
(c) Diseases not significantly occurring in United States The Secretary may conduct biomedical research, directly or through grants or contracts, for the identification, control, treatment, and prevention of diseases (including tropical diseases) which do not occur to a significant extent in the United States.
(d) Protection of privacy of individuals who are research subjects The Secretary may authorize persons engaged in biomedical, behavioral, clinical, or other research (including research on mental health, including research on the use and effect of alcohol and other psychoactive drugs) to protect the privacy of individuals who are the subject of such research by withholding from all persons not connected with the conduct of such research the names or other identifying characteristics of such individuals. Persons so authorized to protect the privacy of such individuals may not be compelled in any Federal, State, or local civil, criminal, administrative, legislative, or other proceedings to identify such individuals.
| File Type | application/msword |
| File Title | OMB CLEARANCE APPLICATION: |
| File Modified | 2008-02-12 |
| File Created | 2008-02-12 |