Ssa Chsi-eval

Goal of the study: To gather data on the awareness, use, and benefits of the Community Health Status Indicators (CHSI) 2015 web-based tool among local health departments.

Intended use of the resulting data: To make decisions regarding the need for modifications and continued maintenance of the website in light of the awareness, use, and benefits of this tool among local health departments

Methods to be used to collect: Two focus groups will be held with local health department data analysts and community health planning officials.

The subpopulation to be studied: Local health department health directors, data analysts, and community health planning staff who focus on community health assessment and planning in their work and have some familiarity with CHSI

How data will be analyzed: Qualitative analysis of narrative focus group data to extract themes among participants’ responses; the data will be reviewed and coded manually.

Shape1

Section A – Justification

Circumstances Making the Collection of Information Necessary

Background

This information collection is being conducted using the Generic Information Collection mechanism of the OSTLTS OMB Clearance Center (O2C2) – OMB No. 0920-0879. The respondent universe for this information collection aligns with that of the O2C2. Data will be collected from 78 health department officials (i.e., health directors, data analysts and community health planning staff) who work in the areas of community health assessment and planning. These health officials will be selected from local health departments that have indicated use of the Community Health Indicators (CHSI) 2015 website tool as reported in the 2016 NACCHO (National Association of City and County Health Officials) Profile assessment. The targeted local health departments are listed in Attachment A (Attachment A—Targeted Local Health Departments).

This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241). This information collection falls under the essential public health service(s) of

1. Monitoring health status to identify community health problems

2. Diagnosing and investigating health problems and health hazards in the community

3. Informing, educating, and empowering people about health issues

4. Mobilizing community partnerships to identify and solve health problems

5. Development of policies and plans that support individual and community health efforts

6. Enforcement of laws and regulations that protect health and ensure safety

7. Linking people to needed personal health services and assure the provision of health care

when otherwise unavailable

8. Assuring a competent public health and personal health care workforce

9. Evaluating effectiveness, accessibility, and quality of personal and population-based health

services

10. Research for new insights and innovative solutions to health problems

Community Health Status Indicators (CHSI) 2015 is an interactive web application that produces health profiles for all 3,143 counties in the United States.¹ Each county profile contains indicators of health outcomes (i.e., mortality and morbidity) and health status (e.g., health care access and quality, health behaviors, social factors, physical environment) that were identified based on a review of commonly utilized population health metrics for community health assessment.² CHSI compares the health indicator values reported for each county with respective indicator values for the U.S. median national average; the Healthy People 2020 national target objectives for health promotion and disease prevention;³ and the values for a set of “peer counties” throughout the U.S. matched with each other on characteristics such as population size, demographics, housing characteristics, and economic factors. Where data are available, indicator values are stratified by subpopulations (e.g., age, race, ethnicity, socioeconomic status) to highlight vulnerable populations within each county.

CHSI was developed for use by public health professionals and community planners for the purpose of setting priorities and targeting resources for community health improvement.⁴^,⁵ Community health improvement planning is a critical public health activity which encapsulates the essential public health services of monitoring health status and educating communities about health issues. These services are provided by local health departments (LHDs) as part of their basic functions.⁶ Also, the Public Health Accreditation Board requires that LHDs have a current community health assessment (CHA) and community health improvement plan (CHIP) available to be eligible for accreditation.⁷ Because CHSI provides community health status data that can be utilized for community health improvement planning, CHSI can be a useful resource for LHDs. Hence, LHDs are an important stakeholder group for CHSI.

In 2009, CHSI underwent revisions that included updated health indicators and enhanced features intended to improve its utility among stakeholders. The updated health indicators reflect the latest data available at the time of the update and include 42 core health indicators that align with a population health framework; the population health framework emphasizes multiple determinants of health—including diseases, health behaviors, health care, and factors in the social and physical environments.⁸ Enhanced features include peer county comparisons and graphics such as color-coded bar charts that highlight quartile percentages for selected indicators. The latest revision of the tool was released online to the public in March 2015 and is referred to as “CHSI 2015.”⁹

The goals of CHSI 2015 are to improve the ability of its stakeholders to:

Assess community health status and health disparities
Promote a shared understanding of the wide range of factors that promote health
Mobilize multi-sector partnerships to work collaboratively to improve population health

CHSI 2015 stakeholders include organizations engaged in community health assessment and improvement; community members; policy makers; and public and private sectors such as education, transportation, social services, and the business sector that share responsibility for creating healthy communities.⁹

LHDs are an important stakeholder group for CHSI 2015. However, CDC has no information regarding the awareness of CHSI 2015 among LHDs, how the application is being utilized by LHDs, or the benefits of this tool for their needs. The assessment is particularly interested in understanding if the updated components to CHSI 2015 provided any enhanced benefits for LHDs. With the CHSI 2015 update, it was anticipated that enhancements such as the peer county comparisons feature would positively benefit LHDs by, for example, providing a tool that LHDs can use to effectively educate their constituents or communicate with decision-makers to leverage needed or ongoing funding for vital public health programs. It is also possible that CHSI could better enable LHDs to identify vulnerable populations by race/ethnicity, gender, or neighborhood to more effectively target resources to improve population health. The enhancements to CHSI or its prioritization of commonly utilized health indicators for public health monitoring may enable LHDs to have more efficient CHA or CHIP processes. These are all potential benefits of CHSI 2015 for LHDs. However, there has been no prior assessment of CHSI 2015 with LHDs to assess if the revisions to CHSI had their intended effect, and consequently, CDC has no data to validate use/utility or benefits of CHSI 2015.

Therefore, the purpose of the assessment is to gather data on the awareness, use, and benefits of CHSI 2015 among local health departments. CHSI was developed to aid in community health assessment and improvement planning for important stakeholder groups such as LHDs. Given this, and given the investment of CDC resources toward the CHSI 2015 update, the assessment will elicit baseline information to determine the awareness, use, and benefits of CHSI among LHDs. The information from this data collection will be used to make decisions regarding the need for modifications and continued maintenance of the website in light of the assessed awareness, use, and benefits of this tool among local health departments.

The National Association of City and County Health Officials (NACCHO) is the cooperative partner with the CDC for this data collection. NACCHO is a nationally-recognized organization whose membership consists of 2,800 Local Health Departments (LHDs) across the United States.¹⁰ In this data collection, NACCHO has the responsibility for recruiting participants and conducting the focus groups, and they will also analyze the data and prepare a report of the results for the CDC. NACCHO was asked to partner with the CDC for this data collection because of NACCHO’s extensive knowledge of and longstanding relationships with LHDs. Accordingly, they have credibility with LHDs which is useful when recruiting participants for this data collection. Furthermore, NACCHO conducts an annual assessment of LHDs and, therefore, they have knowledge of LHDs who use CHSI and could be included in this data collection. NACCHO regularly provides focus group facilitation as part of its consultation services, and hence they are experienced in this area.

Overview of the Information Collection System

Ultimately, the primary source of data collection will be the conduct of focus groups to gather information on the awareness, use, and benefits of CHSI 2015 (outlined in #3 below). However, identifying the participants for the focus groups will necessitate 2 smaller data collection efforts (outlined in #1-2 below).

Health Director Referral Instrument: 26 LHD health directors will be asked to identify potential respondents for the data collection.

Focus Group Participant Recruitment Instrument: this form will be completed by 52 data analysts and community health planning staff who were identified by the health directors at the 26 LHDs

Focus groups: 16 data analysts and community health planning staff who work in the area of community health assessment and planning at the LHDs will participate in focus groups.

NACCHO is the cooperative agreement partner for this data collection and will recruit participants and conduct the focus groups. All three instruments were pilot tested. The health director referral instrument was pilot-tested with 1 public health professional, the recruitment instrument with 2 public health professionals, and the focus group interview guide instrument with 6 public health professionals. Feedback from the health director referral instrument and focus group participant recruitment instrument was used to estimate the time required for respondents to read and complete the questions. Feedback from the focus group interview guide instrument pilot was used to refine the order of the proposed questions, avoid repetition of questions that appeared similar, and estimate the time required for participants to answer the questions in a facilitated group format.

Items of Information to be Collected

The first data collection for this assessment is a referral instrument (Attachment B—Health Director Referral Instrument) that will be sent by NACCHO in the form of an email to health directors at 26 targeted local health departments for the purpose of referring participants for the focus groups. The health director referral instrument will describe the purpose of the data collection and ask the health directors to provide a name, email, and phone number for up to 2 staff persons. Recommended staff persons will consist of data analysts and community health planners who are primary users of CHSI and/or are regularly involved in the work of community health assessment and planning. The health directors will send the referred names and contact information to NACCHO by return email, and NACCHO will use this information to follow-up with the noted contacts as necessary to inquire about their receipt of a recruitment instrument forwarded by their health directors (described next) and arrange their participation in a focus group if desired. The second data collection for this assessment is a recruitment instrument (Attachment C—Focus Group Participant Recruitment Instrument) which will be attached to the health director referral instrument email. In the referral instrument email, health directors will be instructed to forward the attached focus group participant recruitment instrument to the 2 staff persons that they identified on the referral instrument as candidates who can provide feedback about CHSI. The purpose of the focus group participant recruitment instrument is for the referred staff persons to report their level of awareness and familiarity with CHSI. Questions on the focus group participant recruitment instrument also ask respondents to identify their health department, job function, familiarity with other community assessment tools in addition to CHSI, and their interest in participating in a focus group to provide further feedback about CHSI. Respondents will be asked about other community assessment tools (in addition to CHSI) because investigators are interested in identifying which aspects of CHSI may be unique or enhanced compared to other resources. The information provided on the focus group participant recruitment instrument will be used by NACCHO to select up to 16 individuals for the focus groups.

The third and primary data collection instrument for this assessment is the focus group interview guide instrument (Attachment D—Focus Group Interview Guide Instrument). Two focus groups will be conducted with no more than 8 participants each (i.e., 16 participants in total). The focus group interview guide instrument will be used to gather information on the awareness, use, and benefits of CHSI. The focus group was deemed the most appropriate method to collect in-depth feedback for this assessment. The focus group process involves presentation and discussion of open-ended questions in a dynamic, conversational format utilizing a skilled facilitator and permits deeper exploration of participant responses than if data were collected via an online, written instrument. The focus group also elicits interactive feedback between the participants that can encourage perspectives and insights that individual data collection alone may not provide.¹¹

The focus group interview guide instrument consists of 12 open-response questions that will collect information on the following areas:

Awareness of CHSI 2015, for example,

How and where did participants hear about CHSI 2015
Suggestions on how to publicize CHSI to improve awareness

Use of CHSI by local health departments, for example:

How is the tool being used for CHA or CHIP?
Use of CHSI for purposes other than CHA or CHIP
Reasons some LHDs are not utilizing CHSI
How special features of CHSI such as peer county comparisons and vulnerable populations maps are being utilized
Recommended website improvements based on user experiences

Benefits of CHSI for local health departments, for example:

Whether and how CHSI assists local health departments to develop better quality community health assessments
The extent to which CHSI fosters planning to address community health priorities
Unique benefits of CHSI relative to similar web-based community assessment resources
What could have made CHSI more beneficial for LHD needs

Purpose and Use of the Information Collection

The purpose of the assessment is to gather data on the awareness, use, and benefits of CHSI 2015 among local health departments. An updated and revised version of the CHSI application was released to the public in March 2015. However, CDC has no information regarding how the application is being utilized by LHDs or the benefits of this tool for their needs. Given that CHSI was developed to aid in community health assessment and improvement planning for important stakeholder groups such as LHDs, and given the investment of CDC resources toward the CHSI 2015 update to enhance its usage and benefits for LHDs who routinely conduct CHAs and CHIPs, the assessment will elicit baseline information to determine the awareness, use, and benefits of CHSI among LHDs.

This assessment will provide information that will be used to plan what modifications may be needed to the CHSI tool to best meet the needs of LHDs. The information from this data collection will be used to make decisions regarding the need for modifications and ongoing maintenance of the CHSI website tool. There has been no prior assessment of CHSI 2015 to assess if CHSI 2015 revisions had their intended effect with LHDs, therefore CDC has no data to validate use/utility or benefits of CHSI 2015 or how to best apply resources toward CHSI in the future.

Use of Improved Information Technology and Burden Reduction

The health director referral instrument (Attachment B—Health Director Referral Instrument) will be emailed by NACCHO to the 26 health directors. The referral instrument email instructs the health directors to provide minimum information (name, email, and phone) for two contacts in their LHD who they recommend to provide feedback about CHSI. The contact information will be completed in the body of the email and returned to NACCHO by clicking “Reply” and “Send” in their email program. The health director referral instrument has an attached focus group participant recruitment instrument that will be forwarded by the health directors directly to the recommended contacts at their LHD by email. The request for minimum information and the use of email reduces burden on the health director respondents. Therefore, the referral instrument can be completed and returned with minimum effort in order to reduce the burden on the respondents.

The focus group participant recruitment instrument (Attachment C—Focus Group Participant Recruitment Instrument) is intended for completion by the staff who were forwarded the instrument by their LHD health directors (e.g., data analysts and other community health assessment and planning staff). The focus group participant recruitment instrument will be received and sent electronically as a one-page Word document. The minimum amount of questions were included on the recruitment instrument to collect information that can be used to select participants for the focus groups. The instrument has checkboxes and lines for respondents to select or write-in their answers. The instrument is to be returned to NACCHO by email to simplify processing. Overall, the simplified content and processes for completing the recruitment instrument reduce the burden on respondents. Therefore, the recruitment instrument can be completed and returned with minimum effort in order to reduce the burden on the respondents.

Finally, data will be collected via focus groups. The focus group format allows respondents to provide open feedback regarding their awareness, use, and perceived benefits of CHSI to their LHD. This method was chosen to reduce the overall burden on respondents. Feedback elicited via an open-ended written response format would be very time-consuming for respondents to complete. Moreover, using a written format, whether electronically or on paper, would possibly curb the amount of detail that respondents are willing to write down due to the time and inconvenience of writing multiple open-ended responses. Focus groups provide the opportunity for respondents to provide instantaneous feedback in a conversational setting with their LHD peers. The focus group interview guide instrument (Attachment D—Focus Group Interview Guide Instrument) was designed to collect the minimum information necessary for the purposes of this project (i.e., limited to 12 questions).

Focus groups will be conducted by telephone with everyone present and participating simultaneously (i.e., group teleconferencing). Telephone focus groups have similar benefits as face-to-face focus groups in that they solicit detailed input from participants. However, the telephone method offers a greater likelihood that participants across vast areas will be able to participate, and it saves on time and resources that would be required for people in different geographic areas to convene together.¹² The telephone option for a focus group is especially relevant considering that the respondent universe for this data collection includes health officials in local health departments across multiple states. The alternative data collection method of written assessments, whether online or on paper, is not a conducive format for many open-ended questions because it would result in a very lengthy process for respondents. Hence, the telephone focus group method reduces the overall burden on individual respondents.

Efforts to Identify Duplication and Use of Similar Information

To our knowledge, no prior assessment of CHSI 2015 has been done among local health departments or other stakeholders based upon an inquiry of current CHSI staff at CDC as well as a search of internal working documents, gray literature, and published reports. The information from the focus groups will be used to supplement other information received about CHSI such as web analytics data and basic information on awareness, use, and benefits of CHSI collected by NACCHO as part of their annual Profile Study¹³ of health officers in local health departments. None of these complementary data collection sources will provide detail on CHSI awareness, use, and benefits. Therefore, the proposed assessment of CHSI is unique and unduplicated in its data collection purpose, content, and methods.

Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

Consequences of Collecting the Information Less Frequently

This request is for a one time information collection. There are no legal obstacles to reduce the burden. If no data are collected, CDC will be unable to:

Assess how LHDs are using CHSI
Assess the perceived benefits of CHSI among LHDs
Collect baseline data from LHD users to compare with future trends
Justify the investment of CDC resources for continued updates and maintenance of CHSI for use by LHDs

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

This information collection is being conducted using the Generic Information Collection mechanism of the OSTLTS OMB Clearance Center (O2C2) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on October 31, 2013, Vol. 78, No. 211; pp. 653 25-26. No comments were received.

CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.

Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

Protection of the Privacy and Confidentiality of Information Provided by Respondents

The Privacy Act does not apply to this information collection. STLT governmental staff will be speaking from their official roles and all data will be kept secured at NAACHO headquarters in Washington D.C. and destroyed no later than one calendar year after the data collection.

NACCHO will record the focus group sessions on a recorder which captures the recordings as digital files. The digital files will be stored at the NACCHO headquarters in Washington D.C. in a password-protected documents folder that will be accessible only to two NACCHO staff who will be directly involved with recruiting participants or conducting the focus groups. The documents folder will be stored on a secured network drive maintained by NACCHO. The audio files will be listened to only by the designated NACCHO staff and only to the extent needed to verify information in the detailed notes taken by the interviewer or notetaker during the focus group. Audio files will be permanently deleted by the end of one calendar year after the focus groups have been conducted. No written transcripts will be created. Any results reported from this data collection or shared with the public will be in aggregate form only with no individually identifying information attached or affiliated with the aggregate summaries—including the names of health departments.

This information collection is not research involving human subjects.

Institutional Review Board (IRB) and Justification for Sensitive Questions

No information will be collected that are of personal or sensitive nature.

Estimates of Annualized Burden Hours and Costs

The estimate for burden hours is based on three pilot tests of the information collection instruments conducted with 9 public health professionals from local health departments. One individual was in the health director referral instrument pilot; 2 individuals completed the focus group participant recruitment instrument pilot; and 6 individuals participated in the focus group interview pilot.

In the pilot tests, the average time to complete the health director referral instrument (Attachment B—Health Director Referral Instrument) including time for reviewing instructions, gathering needed information, and completing the referral instrument was 15 minutes. Based on these results, the estimated time range for actual respondents to complete the referral instrument is 10 - 20 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 20 minutes) is used. The average time to complete the focus group participant recruitment instrument (Attachment C—Focus Group Participant Recruitment Instrument) including time for reviewing instructions, gathering needed information, and completing the instrument was 2 minutes. Based on these results, the estimated time range for actual respondents to complete the recruitment instrument is 2 - 5 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 5 minutes) is used. The average time to complete the focus group interview guide instrument (Attachment D—Focus Group Interview Guide Instrument) including time for reviewing instructions, gathering needed information, and completing the instrument was approximately 90 minutes. Based on these results, the estimated time range for actual respondents to complete the focus group interview guide instrument is 60 - 90 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 90 minutes) is used.

Estimates for the mean average hourly wage for respondents are based on the Department of Labor (DOL) Occupational Employment and Wages Estimates (May 2015) for medical and health service managers http://www.bls.gov/oes/current/oes119111.htm and for epidemiologists http://www.bls.gov/oes/current/oes191041.htm. Based on DOL data, an average hourly wage of $50.99 is estimated for the health director respondents, and an average hourly wage of $36.97 is estimated for the data analyst/community health planner respondents. The estimated average hourly wage for all respondents combined in this data collection is $41.64. Table A-12 shows estimated burden and cost information.

Table A-12: Estimated Annualized Burden Hours and Costs to Respondents

Information collection Instrument: Form Name	Type of Respondent	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Health Director Referral Instrument	Health directors	26	1	20/60	9	$50.99	$459
Focus Group Recruitment Instrument	Data analysts and community health planning staff	52	1	5/60	4	$36.97	$148
Focus Group Interview Instrument	Data analysts and community health planning staff	16 (of the 52)	1	90/60	24	$36.97	$887
	TOTALS	78			37		$1494

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to the respondents other than their time to participate in each information collection.

Annualized Cost to the Government

There are no equipment or overhead costs. The only cost to the federal government would be the salary of CDC staff and external contractors during the planning and preparation of the information collection, data collection instruments, data analysis, and reporting. The total estimated cost to the federal government is $49,800.00. Table A-14 describes how this cost estimate was calculated.

Table A-14: Estimated Annualized Cost to the Federal Government

Staff (FTE)	Average Hours per Collection	Average Hourly Rate	Average Cost
Technical Monitor – CDC GS -14	100	55.00	$5,500
ORISE Fellow – CDC GS - 12	400	32.00	$12,800
NACCHO Contract			$31,500
Estimated Total Cost of Information Collection			$49,800

Explanation for Program Changes or Adjustments

This is a new information collection.

Plans for Tabulation and Publication and Project Time Schedule

NACCHO will produce a formal written report including focus group findings regarding CHSI awareness, use, and benefits among LHDs. The report is intended primarily for CDC staff in the Division of Public Health Information Dissemination (DPHID) who are responsible for maintaining the CHSI application, and the report will be provided only to this group. NACCHO and CDC may also prepare oral presentations or written summaries to communicate findings to other NACCHO or CDC staff, LHD staff, and other stakeholder groups (e.g., academic institutions, non-profit hospitals) who use CHSI. In addition, CDC may prepare conference presentations or scientific manuscripts for peer-reviewed journals to share results with professional audiences such as the attendees of public health conferences or the readership of public health journals. CDC will use the findings to understand how LHDs are using the tool and to determine if modifications are needed to better meet the needs of LHDs and other public health stakeholders. Findings from the assessment will also help CDC to make strategic decisions regarding the investment of CDC resources for continued updates and maintenance of CHSI.

Project Time Schedule

Task	Timeline	Status
Design instruments	5 weeks	COMPLETE 
Develop protocol, instructions, and analysis plan	5 weeks	COMPLETE 
Pilot test instruments	2 weeks	COMPLETE 
Prepare OMB package	5 weeks	COMPLETE 
Submit OMB package	4 weeks	COMPLETE 
OMB approval	In process	TBD --
Conduct assessment	Within 2 weeks after OMB approval	TBD --
Code, quality control, and analyze data	Within 2 weeks after data collection	TBD --
Prepare summary reports	Within 3 weeks after data analysis	TBD --
Disseminate final results/reports	Within 2 weeks after summary reports prepared	TBD --

Reason(s) Display of OMB Expiration Date is Inappropriate

We are requesting no exemption.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

LIST OF ATTACHMENTS – Section A

Targeted Local Health Departments
Health Director Referral Instrument
Focus Group Participant Recruitment Instrument
Focus Group Interview Guide Instrument

REFERENCE LIST

1. CHSI: Information for Improving Community Health. http://wwwn.cdc.gov/CommunityHealth/home. Accessed April 7, 2016.

2. Centers for Disease Control and Prevention (CDC). Community Health Assessment for Population Health Improvement: Resource of Most Frequently Recommended Health Outcomes and Determinants, Atlanta, GA: Office of Surveillance, Epidemiology, and Laboratory Services, 2013. http://wwwn.cdc.gov/CommunityHealth/PDF/Final_CHAforPHI_508.pdf. Accessed April 7, 2016.

3. Healthy People.gov 2020 Topics and Objectives—Objectives A – Z. https://www.healthypeople.gov/2020/topics-objectives. Accessed April 7, 2016.

4 . Community Health Status Report: Data Sources, Definitions, and Notes. July 2000. http://www.gobroomecounty.com/files/hd/pdfs/CHSI-CompanionView.pdf. Accessed April 7, 2016.

5. Metzler M et al. Community Health Status Indicators Project: the development of a national approach to community health. Prev Chronic Dis 2008;5(3). http://www.cdc.gov/pcd/issues/2008/jul/07_0225.htm. Accessed April 7, 2016.

6. Operational Definition of a Functional Health Department. NAACHO, November 2005. http://archived.naccho.org/topics/infrastructure/accreditation/upload/OperationalDefinitionBrochure-2.pdf. Accessed April 7, 2016.

7. Public Health Accreditation Board, National Public Health Department Accreditation Readiness Checklists. Revised July 2015. http://www.phaboard.org/wp-content/uploads/checklist-revisedFINAL.pdf. Accessed April 7, 2016.

8. Evans RG, Stoddart GL. Producing health, consuming health care. Soc Sci Med. 31(2), 1347-1363.

9. CHSI: Information for Improving Community Health. http://wwwn.cdc.gov/CommunityHealth/info/AboutProject. Accessed April 7, 2016.

10. NACCHO. Your local health department’s biggest advocate. http://www.naccho.org/. Accessed April 7, 2016.

11. Data Collection Methods for Program Evaluators: Focus Groups. Evaluation eTA: Evaluation Briefs, No. 13 (July 2008). http://www.cdc.gov/HealthyYouth/evaluation/pdf brief13.pdf. Accessed April 7, 2016.

12. Krueger RA. Telephone Focus Groups. http://www.tc.umn.edu/~rkrueger/focus_tfg.html. Accessed April 7, 2016.

13. National Profile Study of Local Health Departments. National Association of City and County Health Officials (NACCHO). http://nacchoprofilestudy.org/reports-publications/ Accessed April 7, 2016.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	CDC User
File Modified	0000-00-00
File Created	2021-01-26