Supporting Statement Part A

This information collection is being conducted using the Generic Information Collection mechanism of the OSTLTS OMB Clearance Center (O2C2) – OMB No. 0920-0879. The respondent universe for this information collection aligns with that of the O2C2. Information will be collected from a total of 2,913 public health respondents across state, tribal, local, and territorial (STLT) health departments/jurisdictions from all 50 states (plus the District of Columbia) and 8 US territories. These STLT public health professionals acting in their official capacities include health officials, public health program managers, grant coordinators, public information officers, and STLT-agency public health professionals who have subscribed to receive updates from CDC’s Office for State, Tribal, Local and Territorial Support.

This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241). This information collection falls under the essential public health service(s) of

1. Monitoring health status to identify community health problems

2. Diagnosing and investigating health problems and health hazards in the community

3. Informing, educating, and empowering people about health issues

4. Mobilizing community partnerships to identify and solve health problems

5. Development of policies and plans that support individual and community health efforts

6. Enforcement of laws and regulations that protect health and ensure safety

7. Linking people to needed personal health services and assure the provision of health care

when otherwise unavailable

8. Assuring a competent public health and personal health care workforce

9. Evaluating effectiveness, accessibility, and quality of personal and population-based

health services

10. Research for new insights and innovative solutions to health problems¹

CDC’s Office for State, Tribal, Local and Territorial Support (OSTLTS) serves as CDC’s primary connection to health officials, leaders, and staff members of STLT public health agencies, as well as to other government leaders and national partners who support health departments. The State, Tribal, Local, and Territorial Public Health Professionals Gateway, or “STLT Gateway,” is a CDC website developed as a portal or “hub” for STLT public health leaders and professionals (see Attachment A—STLT Gateway Screenshot).² The site provides these professionals with central access to information about public health programs; guidance on public health accreditation and performance improvement; details about state and local public health data sources; analyses of state-specific laws, policies, and practices to improve health; tailored information about CDC and the US public health system; information about federal funding opportunities; best practices and success stories describing innovative STLT public health activities; CDC content that can be syndicated for free on external websites; and communication products customized for public health professional audiences. The STLT Gateway averages about 39,000 page views per month.³ The site is unique among CDC websites in that it is designed specifically for public health professionals rather than for the general public or clinicians. It is thus a valuable source of information for those professionals serving on the frontlines of the public health system who are charged with meeting their jurisdictions’ most pressing public health needs.

Recognizing the need to use the web as a key channel for disseminating public health information to the STLT community, OSTLTS has initiated an assessment and redesign of the STLT Gateway. Because of the wealth of valuable information housed on the website, it is imperative for the site’s content, design, information architecture, and user experience to align with the diverse needs of the STLT stakeholders so that OSTLTS can successfully meet one of its key roles: serving as CDC’s primary connection point for the STLT public health professional community. The purpose of the proposed information collection is to assess STLT public health professionals’ familiarity and satisfaction with the STLT Gateway; gather critical data about the topics and types of information most needed by STLT public health professionals to support their public health work; determine the degree to which the STLT Gateway meets those needs through its current content, navigation, and user experience; and identify potential areas for improvement.

Prior to this information collection, a one-time assessment of OSTLTS’s communication products for STLT health professionals was conducted in the fall of 2013 (OMB No. 0920-0879). In that assessment, a web-based questionnaire collected information about STLT health professionals’ awareness of, use of, and satisfaction with five different OSTLTS communication products and channels, one of which was the STLT Gateway. That information collection was limited to assessing STLT public health professionals’ familiarity and satisfaction with various communication products, including the website. That collection did not collect information about the topics or formats of resources and information those stakeholders most need from a CDC website, nor did it assess the labeling, organization, or information architecture of the STLT Gateway.

The assessment proposed in this package would substantially expand on the findings of the 2013 assessment by a) determining whether STLT public health professionals’ use of the website has increased over time, b) collecting valuable data about these stakeholders’ preferred web content and formats, and c) gathering actionable feedback about how this target audience would categorize and label key website topics to inform future navigational improvements.

Overall, CDC will use the results of this information collection to improve the STLT Gateway so that STLT public health professionals can readily access the CDC resources and information they need to perform their jobs effectively and efficiently. Ultimately, this initiative will enable CDC to create a high-quality website that can inform, educate, and empower these key stakeholders and consequently contribute to ensuring a competent public health workforce.

Overview of the Information Collection System

Information will be collected from a total of 2,913 STLT public health professionals via two separate information collection methods:

A web-based questionnaire: An online questionnaire about the STLT Gateway will be sent to 2,413 STLT public health professionals. The questionnaire has been created using SurveyMonkey® software and will be made available on a private, invitation-only webpage. This instrument will be used to gather information from STLT public health professionals regarding their usage of, satisfaction with, and experiences with the STLT Gateway, as well as their preferred website topics and formats (see Attachment B—Questionnaire Information Collection Instrument: MS Word Version and Attachment C— Questionnaire Information Collection Instrument: Web Version). The questionnaire was pilot tested by 5 public health professionals. Feedback from this group was used to refine questions as needed, ensure accurate programming and skip patterns, and establish the estimated time required to complete the information collection instrument.

An online card sorting exercise: An online card sorting exercise will be sent to 500 STLT public health professionals. Card sorting is a method used to help design or assess the information architecture of a website. Participants will be asked to organize a list of topics from content within the STLT Gateway into groups that make sense to them, and then label each group they create in a way that they think accurately describes the content (see Attachment D—Card Sorting Information Collection Instrument: MS Word Version and Attachment E—Card Sorting Information Collection Instrument: Web Version). The card sorting information collection instrument has been created using OptimalSort software and will be available on a private, invitation-only webpage. The instrument was pilot tested by 4 public health professionals. Feedback from this group was used to refine content labels, refine user instructions for completing the exercise, and establish the estimated time required to complete the exercise.

Items of Information to Be Collected

Questionnaire

The questionnaire information collection instrument consists of 15 main questions of various types, including dichotomous (yes/no), multiple choice (select one answer), multiple response (select all that apply), interval (rating scales), and open-ended.

To gather information from STLT public health professionals regarding their usage of, satisfaction with, and experiences with the STLT Gateway, as well as their preferred website topics and formats, the instrument will collect data on the following themes:

Familiarity with the STLT Gateway and frequency of use
Extent to which users agree that the website is useful, up-to-date, meets their needs, and is easy to understand
Information, topics, and resources most sought out or needed by STLT health professionals
Preferred formats for viewing public health information online
Subscriptions to various CDC communication products
Respondent characteristics, including jurisdiction, seniority level, and professional role

Card sorting:

The online card sorting information collection instrument consists of a list of 52 “cards” or labels that represent different topics currently covered on the STLT Gateway, such as “Training, Learning, and Continuing Education Opportunities,” “10 Essential Public Health Services,” “Community Health Assessment Resources,” “Success Stories from the Public Health”, and “Benefits and Impacts of Public Accreditation,” and so forth (for a complete list of the 52 cards that participants will sort, see page 3 of Attachment D—Card Sorting Information Collection Instrument: MS Word Version).

In this exercise, participants will be asked to examine the card topics and then sort them electronically by “dragging and dropping” the cards into groups that make logical sense to them. They will then be asked to assign a label or title to each category that accurately describes the cards in that group. For example, a user might group several cards related to training and continuing education together and then name that group “Training and Professional Development,” and then group several cards about federal grants and grant-writing guidance together and then label that group “Funding Opportunity Resources.” This information collection instrument will help inform the categorization, labeling, and information architecture of the STLT Gateway website, thus making the wealth of information on the site easier for public health professionals to locate and use.

Purpose and Use of the Information Collection

The purpose of this information collection is to assess STLT public health professionals’ familiarity and satisfaction with the STLT Gateway; gather critical data about the topics and types of information most needed by STLT public health professionals to support their public health work; determine the degree to which the site meets those needs through its current content, navigation, and user experience; and identify potential areas for improvement.

CDC will use the results of the web-based questionnaire to understand the specific informational needs of public health professionals and to optimize content in ways that meet these stakeholders’ needs. CDC will use findings of the card sorting exercise to improve the site’s information architecture and content labeling, creating a website where users can readily find and use critical content because the site’s organization resonates with them. Overall, the results of this information collection will be used to inform improvements to the website to ensure STLT health professionals can readily access the CDC resources and information they need to perform their jobs effectively and efficiently.

Use of Improved Information Technology and Burden Reduction

Information will be collected via two methods: a web-based questionnaire and an online card sorting exercise. Both methods reduce the overall burden on respondents because all activities can be completed entirely online, independently, and at their own convenience within a two-week window for the web-based questionnaire and a three-week window for the card sorting exercise.

The questionnaire information collection instrument was designed to collect the minimum information necessary for the purposes of this project (i.e., limited to 15 questions). In addition, skip patterns were included to allow respondents to answer only those questions that apply to their usage of the STLT Gateway, which will streamline responses and further reduce burden on respondents. The card sorting information collection instrument was also designed to collect the minimum data necessary; more than 850 pages of STLT Gateway content were condensed down to only 52 “cards” for participants to sort.

Including both assessments as part of this information collection will provide more enriching data for the planned website redesign. The effectiveness of digital communications depends on the successful marriage of quality content that meets the needs of web visitors and intuitive organization that facilitates visitors finding what they need. The two information collection instruments allow for the collection of meaningful data to improve both of these components of the website. Neither methodology is well-suited to provide the type of information the other will, but together they will provide a more complete picture of STLT health professionals’ web content needs and how these users would like such content to be organized and labeled.

Efforts to Identify Duplication and Use of Similar Information

Only one previous information collection effort has collected information about STLT professionals’ experience with the STLT Gateway (OMB No. 0920-0879), CDC’s sole website targeted particularly to them. In the fall of 2013, a web-based questionnaire collected data about STLT health professionals’ awareness of, use of, and satisfaction with five OSTLTS communication products and channels: Did You Know?; Have You Heard? Facts from the Field; CDC’s Vital Signs Town Hall Teleconference; the CDC STLT Connection Facebook page; and the STLT Gateway. The results of the STLT Gateway portion of the 2013 information collection were valuable to OSTLTS because they yielded a major finding: a lack of awareness about the STLT Gateway among the target audience. Since 2013, OSTLTS has responded to this finding by increasing its promotion of the STLT Gateway, incorporating information about the site into presentations and publications by OSTLTS leaders and staff, collaborating with other CDC divisions and programs to add links to the STLT Gateway.

The proposed information collection described in this package has been carefully developed to be non-duplicative of the 2013 assessment. It will address several limitations of the prior assessment, determine whether awareness and use of the STLT Gateway has increased, and collect actionable information about users’ current and future content needs. For example―

Assessing content and format needs: The 2013 assessment asked STLT public health professionals about their familiarity with and satisfaction with the STLT Gateway but did not ask about specific types of content they were looking to find on the site, nor about the information or resources they would need to see to make the site more useful. The questionnaire described in this package will ask users to identify the most useful resources currently on the STLT Gateway, as well as list the types of content that would be most useful to add to the website. The questionnaire will also ask about which formats of web-based content (e.g., research reports, short summaries, videos, infographics, etc.) STLT health professionals find most useful.
Determining whether awareness and usage of the website has increased: The questionnaire proposed in this assessment will again assess the percentage of STLT health professionals who have visited the STLT Gateway. This will enable OSTLTS to determine whether its efforts to promote awareness and use of the site have been effective, or whether additional outreach and promotional activities are needed.
Collecting information from non-users: For the 2013 information collection, respondents who indicated that they had not visited the STLT Gateway were asked no further questions about the website, their use of web resources, or their content needs. This resulted in a small sample size from which to draw meaningful conclusions. The questionnaire proposed in this package will permit respondents who have not previously visited the STLT Gateway to respond to several questions about their content interests and needs.
Addition of a card sorting exercise: The previous assessment did not include a card sorting exercise, nor did it ask any questions about the grouping or labeling of website content. The card sorting exercise described in this package will give OSTLTS valuable information about how STLT end users would logically group website topics and how they would label those groups in ways that are meaningful to them.
Assessing current content and organization: The STLT Gateway has changed substantially since 2013, with notable updates and additions to the site’s information, resources, and publications; enhancements to the site’s design and organization; and improved mechanisms for STLT professionals to interact with OSTLTS and seek technical assistance. The information collection instruments described in this package will gather user data regarding the website in its current form.

With the exception of the 2013 assessment and collection of STLT Gateway user metrics (e.g., number of page views, downloads), there are no other recent or current efforts to collect information about user experience and stakeholder needs for the STLT Gateway. Web user metrics provide limited information about end-user behavior, do not provide information about whether visitors are members of target audiences, and do not enable OSTLTS to understand the unique needs and expectations of target audiences.

Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

Consequences of Collecting the Information Less Frequently

This request is for a one-time information collection. There are no legal obstacles to reduce the burden. If no data are collected, CDC will be unable to

Assess the usefulness of the STLT Gateway in providing STLT public health officials and professionals with the information, guidance, and resources required to effectively perform their jobs
Determine whether awareness of the STLT Gateway has increased among CDC’s target audiences for the website
Identify critical unmet information and content needs of STLT public health officials and their staff and then develop plans to meet those needs by creating and disseminating needed content
Make informed decisions in the planning of the redesign efforts for the site, including how to improve the website’s content, organization, terminology, and language based on actual feedback from the target audience
Understand, and in turn resolve, challenges involving the user experience that potentially hinder end-users from accessing important CDC resources, such as information for new STLT health officials, guidance on accreditation and performance improvement, and training resources for public health practitioners

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

This information collection is being conducted using the Generic Information Collection mechanism of the OSTLTS OMB Clearance Center (O2C2) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on May 16, 2014, Vol. 79, No. 95; pp. 28513. No comments were received.

CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.

Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

Protection of the Privacy and Confidentiality of Information Provided by Respondents

The Privacy Act does not apply to this information collection. STLT governmental staff and/or delegates will be speaking from their official roles. No personally identifiable information (PII) will be collected. All responses collected will be kept on secure, password protected servers accessible only to project team members. All data from this information collection will be shared in aggregate.

Institutional Review Board (IRB) and Justification for Sensitive Questions

This information collection is not research involving human subjects.

No information will be collected that are of personal or sensitive nature.

Estimates of Annualized Burden Hours and Costs

The estimate for burden hours is based on a pilot test of the questionnaire information collection instrument by 5 public health professionals and a pilot test of the card sorting information collection instrument by 4 public health professionals.

In the pilot test of the questionnaire information collection instrument, the average time to complete the instrument, including time for reviewing instructions and completing the questionnaire, was approximately 4 minutes (range: 3–5 minutes). For the purposes of estimating burden hours, the upper limit of this range (i.e., 5 minutes) is used.
In the pilot test of the card sorting information collection instrument, the average time to complete the instrument, including time for reviewing instructions and completing the card sort, was approximately 35 minutes (range: 16–45 minutes). For the purposes of estimating burden hours, the upper limit of this range (i.e., 45 minutes) is used.

Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) Bureau of Labor Statistics for occupational employment for STLT public health professionals http://www.bls.gov/oes/current/oes_nat.htm. Based on DOL data, an average hourly wage of $32.11 is estimated across the 2,913 respondents. We calculated this average wage by averaging the mean salaries of eight different state and local health department professional roles that represent examples of the roles and seniority levels included in the respondent universe (see Attachment F— Calculating Average Hourly Wages for STLT Health Professional Respondent Burden (for Table A-12) for details on how this average was estimated). Table A-12 shows estimated burden and cost information.

Table A-12: Estimated Annualized Burden Hours and Costs to Respondents

Information collection Instrument: Form Name	Type of Respondent	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Web-Based Questionnaire	STLT public health professionals	2,413	1	5 / 60	201	$32.11	$6,454
Online Card Sorting Instrument	STLT public health professionals	500	1	45 / 60	375	$32.11	$12,041
	TOTALS	2,913	1		576		$18,495

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to the respondents other than their time to participate in each information collection.

Annualized Cost to the Government

There are no equipment or overhead costs. The only cost to the federal government would be the salary of CDC staff and contractors to develop the information collection instruments, collect data, and perform data analysis. Contractors are being used to support the following tasks: preparation of the information collection instruments, information collection, and analysis. The total estimated cost to the federal government is $47,101.65. Table A-14 describes how this cost estimate was calculated.

Table A-14: Estimated Annualized Cost to the Federal Government

Staff (FTE)	Average Hours per Collection	Average Hourly Rate	Total Average Cost
Health Scientist–GS-13, Step 7; Preparation and analysis of web-based questionnaire, preparation of OMB package, development of summary report	120	$51.76/hour	$ 6,211.20
Associate Director for Program Planning and Communication–GS-14, Step 7 Information collection strategy development and reparation of OMB package, web-based questionnaire, card sorting exercise, and communication materials	80	$61.17/hour	$4,893.60
Public Health Advisor GS-9, Step 2 Collection and validation of mailing lists, preparation and dissemination of survey and card sort invitations, monitoring of email questions from invited participants	20	$25.85/hour	$517.00
Senior Health Communication Advisor, Deloitte Consulting, LLP Preparation of web-based questionnaire, card sorting exercise, communication materials, and OMB package.	120		$8,746.80
Project Manager, DRT Strategies Preparation and analysis of card sorting exercise and summary report	60		$6,671.33
User Experience Specialist, DRT Strategies Preparation and analysis of card sorting exercise and summary report	133		$20,061.72.00
Estimated Total Cost of Information Collection			$47,101.65

Explanation for Program Changes or Adjustments

This is a new information collection.

Plans for Tabulation and Publication and Project Time Schedule

Web-based questionnaire: Once the two-week information collection period has closed, responses will be exported from SurveyMonkey® to Microsoft Excel®. Using Excel, we will perform descriptive statistical analysis including frequencies on multiple-choice items, cross-tabulations of information needs based on use of the website, and analysis of data by respondent type. Qualitative data from open-ended items will be reviewed to determine themes. Upon completion of data analysis, we will summarize the key findings using data charts, graphs and qualitative themes for incorporation into a single “Findings and Recommendations” report. All data will be reported in aggregate.

Card sorting exercise: Once the three-week information collection period has closed, we will conduct an analysis to reveal the most popular options for organization of topics on the STLT Gateway. Analysis will be conducted using various data visualization features inherent in the OptimalSort tool. Similarity matrices and dendograms will reveal clusters of similarity between card groupings created by participants, and a participant-centric analysis will identify the top three most acceptable but substantially different submissions by participants. All data will be reported in aggregate for inclusion in the “Findings and Recommendations” report.

Aggregate findings from both information collection instruments will be compiled in a findings and recommendations report for CDC/OSTLTS leaders to use to implement necessary improvements to the STLT Gateway’s content and organization, thus ensuring the website meets identified needs of STLT public health professionals across the country.

Project Time Schedule

Design instrument (COMPLETE)

Develop protocol, instructions, and analysis plan (COMPLETE)

Pilot test instruments (COMPLETE)
Prepare OMB package (COMPLETE)
Submit OMB package (COMPLETE)

OMB approval (TBD)
Conduct information collection using web-based questionnaire (Open 2 weeks)
Conduct information collection using card sorting exercise…………………...(Open 3 weeks)
Code data, conduct quality control, and analyze data (3 weeks)
Prepare summary report(s) of questionnaire and card sort findings (4 weeks)

Reason(s) Display of OMB Expiration Date is Inappropriate

We are requesting no exemption.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

LIST OF ATTACHMENTS – Section A

Attachment A—STLT Gateway Screenshot

Attachment B—Questionnaire Information Collection Instrument: MS Word Version

Attachment C—Questionnaire Information Collection Instrument: Web Version

Attachment D—Cart Sorting Information Collection Instrument: MS Word Version

Attachment E— Cart Sorting Information Collection Instrument: Web Version

Attachment F— Calculating Average Hourly Wages for STLT Health Professional Respondent Burden (for Table A-12)

REFERENCE LIST

Centers for Disease Control and Prevention. "National Public Health Performance Standards Program (NPHPSP): 10 Essential Public Health Services." Available at www.cdc.gov/nphpsp/essentialservices.html. Accessed on 8/14/14.
Centers for Disease Control and Prevention. State, Tribal, Local, and Territorial Public Health Professionals Gateway (STLT Gateway). Available at www.cdc.gov/stltpublichealth. Accessed on 8/31/17.
Centers for Disease Control and Prevention. Analysis of internal STLT Gateway web metrics reports for January–December 2016 (unpublished data); July 2017.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Gilliam, Adzua H. (CDC/OSTLTS/DPHPI)
File Modified	0000-00-00
File Created	2021-01-21