0920-05CI Supporting Statement 1-31-2007

The CDC seeks to improve the oral health of the nation by targeting efforts to improve the infrastructure of state and territorial oral health departments; strengthen and enhance program capacity related to monitoring the population’s oral health status and behaviors; develop effective programs to improve the oral health of children and adults; evaluate program accomplishments; and inform key stakeholders, including policy makers, of program results. Through a cooperative agreement program (Program Announcement 03022), CDC provides approximately $3 million per year over 5 years to 12 states and one territory to strengthen state core oral health infrastructure and capacity and reduce health disparities among high-risk groups. The CDC is authorized to do this under sections 301 (a) and 317 (k) (2) of the Public Health Service Act [42 U.S.C. section 241 (a) and 247b(k) (2)]. Copies of these Public Law sections are displayed in Attachment 1. The Catalog of Federal Domestic Assistance (CFDA) number is 93.283.

CDC requests the submission of semi-annual status reports from each funded program. The proposed information collection will be used to identify training and technical assistance needs; monitor compliance with cooperative agreement requirements; evaluate the progress made in achieving national and program-specific goals; and respond to inquiries regarding program activities and effectiveness. Previously, CDC used a variety of sources to collect state-level information including the initial cooperative agreement application, continuing applications for each budget period, periodic progress reports, and financial status reports.

This non-standardized approach to data collection resulted in reports that varied in content and detail. Historically, information has been collected and transmitted via hard-copy paper documents and maintained in large, cumbersome manual files. The manual reporting system, as outlined by CDC Cooperative agreement 3022, limits CDC’s ability to compile, summarize, and report aggregate information in an efficient and useful manner.

CDC has developed an automated management information system (MIS) to maintain individual grantee information and to normalize the information reported by these programs. The proposed data collection will employ a more formal, systematic method of collecting information that has historically been requested from individual programs and will standardize the content of this information. This will facilitate the CDC’s ability to fulfill its obligations under the cooperative agreements; to monitor, evaluate, and compare individual programs; and to assess and report aggregate information regarding the overall effectiveness of the program. The MIS will also support CDC’s mission of reducing the burden of oral diseases by enabling staff to more effectively identify the strengths and weaknesses of individual grantees and to disseminate information related to successful public health interventions implemented by these organizations.

2. Purpose and Use of Information Collection

The information regarding individual programs is currently reported manually using a variety of sources. These sources include the initial cooperative agreement application, continuing applications for each 12-month budget period within an approved project period, progress reports, and financial status reports.

The initial cooperative agreement application is submitted by funding applicants once at the beginning of an approved project period. Continuing applications are submitted annually in February. Additionally, an annual financial status report is required three months after the end of each budget period and details the audited results of funds use.

CDC uses this information for program operations, management, and reporting purposes including:

 Identifying the need for ongoing guidance, training, consultation, and technical assistance in all aspects of oral disease prevention and control

 Evaluating the progress made by programs in achieving national and program-specific goals and objectives

 Identifying successful and innovative strategies and public health interventions to reduce the burden of oral diseases

 Disseminating and sharing information among all grantees

 Monitoring the use of federal funds

 Evaluating and reporting on the overall effectiveness of the grantees

This proposed reporting methodology will improve CDC’s ability to perform these functions and responsibilities. More importantly, it will enable CDC to utilize automated technology to perform these functions in a more efficient and effective manner. The frequency with which the information will be collected will remain the same as the current requirements within the cooperative agreement, semi-annually.

3. Use of Improved Information Technology and Burden Reduction

The following information collection objectives involve the use of modern, state-of-the-art information technology to support the acquisition and reporting requirements.

 Exploit the capabilities of the Internet to provide State access to the database

 Provide a methodology for efficient and secure submission of semi-annual State and Territorial reports

The proposed methodology uses the Internet’s standard communication protocols to control both access and communications by State and Territorial program personnel. CDC can provide State and Territorial program personnel with access to program information via the web. For example, the user could browse through a series of preformatted screens that display each group of State and Territorial program data such as program activity, staffing, administrative, financial, and advisory body information. Further selected portions of State and Territorial program data (such as financial data) could be restricted to specific States and Territories and/or selected State and Territorial personnel only using additional password protection.

A major objective of this project provides special data collection procedures for efficient and secure submission of State reports that are designed to reduce the burden to the respondent. It is estimated that the MIS will decrease the reporting burden on grantees by as much as 50% once the initial data is entered since states and territories will only need to update information semi-annually.

4. Efforts to Identify Duplication and Use of Similar Information

The CDC recognized the current manual reporting of information fostered inconsistent information at many levels. This lack of standardization of the data collected impedes meaningful and efficient cross-state reporting and evaluation.

The MIS does not cause duplication and in fact, eliminates duplicative efforts under our current reporting system.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

6. Consequences of Collecting Information Less Frequently

There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances related to the MIS, all guidelines of 5 CRF 1320.5 are met, and this project fully complies.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

A sixty day Federal Register notice was published on July 7, 2006 (Volume 71, Number 130), page 38647 (See Attachment 2). There were no public comments.

Consultation with state grantees occurred to determine information needs of the state programs. Volunteers were solicited during a grantee workshop held February 2006 in Atlanta, GA. An eight member workgroup was established that represented six of the 13 grantee states (see Attachment 3).

9. Explanation of Any Payment or Gift to Respondents

Applicants or funding recipients do not receive payments or gifts for providing information.

10. Assurance of Confidentiality Provided to Respondents

The CDC Privacy Act Officer has reviewed this submission and has determined that the Privacy Act is not applicable to the data collection. Respondents are state-based health departments providing information on their organizational goals, activities, performance metrics, and resources. Although one or more contact persons are identified for each responding health department, the contact person does not provide personal information.

Respondent data will be submitted to CDC via standard Internet-based communications protocols. Data security is ensured in the event of unauthorized access is gained to the application server and/or code through the following means:

Storing user passwords in the database using one-way encryption.
Two-way encryption of user IDs and password in the database connection string.
In order to prevent unauthorized access, if any user is inactive in the application for 90 or more minutes, their session will be dropped. In order for the user to continue using the application, the user will be required to re-login.
If there are three consecutive, unsuccessful logins, the application will inactivate the user login ID for 30 minutes. After 30 minutes, the application will automatically re-activate the user login ID and allow the user to attempt to log in again.
System administrators can alter or delete a user account that is suspected to have unauthorized access.
Audit trail information can be used to determine which records were affected.
Database is backed up nightly so the database can be restored to a previous state in the event of suspected data corruption.

Data will be stored on CDC Servers which adhere to CDC standards and policies and

access to potentially sensitive data elements such as financial data can be restricted using additional password protection. CDC contracts with Northrop Grumman to provide system development and maintenance for MOLAR and the contract requires that all Northrop Grumman employees and its subcontractors sign a non-disclosure of information. Data and application management procedures within DOH have been implemented to monitor and maintain the application and data. All accounts will be approved by a system administrator inside the CDC network and reviewed on an ongoing basis. If any account becomes suspect, that account will be removed by the system administrator. The Data Steward will be responsible for periodic reviews of the data to ensure quality and accuracy as well as ensuring that information is updated in a timely manner by users.

11. Questions for Sensitive Questions

Some of the respondent's financial, performance or personnel data could be viewed as sensitive; however, this information is integral to the purposes of the MIS. The security measures described above have been put in place to guard against inadvertent or inappropriate disclosure of sensitive information.

12. Estimates of Annualized Burden Hours and Costs

A. Estimated Annualized Burden Hours

The state oral health programs are generally small and the director, program manager or coordinator will be the person responsible for developing the interim and annual reports. To determine the annualized burden hours, it is estimated from the workshop that it will take each respondent approximately nine hours to update the information with in the MIS to complete the report. This is required twice a year for all 13 recipients.

12.A Estimated Annualized Burden Hours

Type of Respondent	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
State Program Staff	13	2	9	234

Estimated Annualized Cost to Respondents

To determine the annualized burden, cost salaries of state oral health program staff were averaged for 6 of the 13 recipients to determine an average hourly wage of approximately $25. The hourly wage is a straight calculation that does not include an estimate of benefits. This hourly wage was multiplied by the burden of hours to obtain an estimate of burden cost to each program ($25 x 234 = $5,850)

Cost to respondents is estimated at $5,850.

12.B Estimated Annualized Cost to Respondents

Type of Respondents

Number of Respondents

Frequency of Response

Hours Per Response

Hourly Wage

Respondent Cost

State Program Managers

$25.00

$5,850

Total

$5,850

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

The information system was designed to use existing hardware within funded sites, and all respondents currently have access to the Internet to use the information system. No capital or maintenance costs are expected. Additionally, there are no start-up, hardware or software costs.

14. Annualized Cost to the Federal Government

Development, Implementation, and Maintenance

Major cost factors for the management information system include application design and development costs, and system modification costs based on pilot testing and feedback from system users. Ongoing costs will include system maintenance and training costs. For the purposes of calculating the estimated annualized cost to the government, the progress reporting system project has been divided into two phases: 1) development of the application; and 2) implementation and system maintenance. Table 3 provides a detailed breakdown of the estimated cost for phases 1 and 2. The total cost Table 3 is not an annualized cost. It represents the total cost for development and implementation of the system by a contractor and is a one-time expenditure. The ongoing maintenance costs and associated project support costs are assumed constant for the useful life of the system. However, because this system gathers progress reporting information associated with specific performance measures required as part of 5 year Cooperative Agreements cycles with states and territories, any change to these performance measures in the future may precipitate system modifications. The associated costs for such modifications are undetermined and are not reflected here. However, it is assumed these changes would be minimal and thus easily incorporated into the contractors overall system maintenance contract, a currently established government contract expenditure. The CDC employee cost to plan, develop, implement, train, and maintain the system is based on the sum of the hourly wages, times two (two hour meetings) of the personnel involved = $450. Multiply $450 by the number of weeks involved in planning, developing and training to the staff on the system. To determine the maintenance cost, 10% of time of the CDC staff member assigned to maintaining the system multiplied by 52 weeks or one year.

14.A Annualized cost table

Phase	Estimated Cost
	Contractor	CDC FTE
Planning	$6,750	$18,000
Analysis	$101,250
Design	$63,750
Development	$135,000	$4,500
Testing	$30,000	$ 900
Deployment	$1,100
Documentation	$24,400
Training	$8,200	$1,800
Maintenance	$4,700	$8,320
Total	$375,150	$33,520

15. Explanation for Program Changes or Adjustments

This is a new reporting system.

16. Plans for Tabulation and Publication and Project Time Schedule

A. Time schedule for the entire project

A 3-year clearance is requested for this required semi-annual data collection. Actual data collection scheduled to begin in Feb. 2007. The below Project Time Schedule table includes the start and end dates for collection of information and other actions as required.

A. 16-1 Project Time Schedule
Activity	Time Schedule
Letters sent to respondents	1 – 2 months after OMB approval
Completed training	2-4 months after OMB approval
Analyses and Validation	5 - 7 months after OMB approval
On-going Support (as required)	8 months after OMB approval

B. Publication plan

Information collected through the MIS will be reported in internal CDC documents and shared with state and territorial grantees.

C. Analysis plan

CDC will not use complex statistical methods for analyzing information. All information will be aggregated and reported in internal documents. Statistical analyses will be limited to simple tabulations.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The expiration date of OMB approval of the data collection will be displayed.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exemptions are being sought to the certification statement for this data collection.

B. Collections of Information Employing Statistical Methods

CDC will not use any statistical methods to select respondents because all funded states and territories will use the MIS system. Public law requires application submission and financial reporting by the actual recipients of funding. Statistical methods cannot be used to reduce burden or improve accuracy of results because of the nature of the program.

All 13 states and grantees are currently required to submit annual progress reports. The MIS will allow funded programs to submit their progress reports semi-annually by entering information into the system, thus eliminating the need for additional written reports. The MIS will enable CDC to identify training and technical assistance needs, monitor compliance with cooperative agreement requirements, evaluate progress made in achieving program-specific goals, and obtain information needed to respond to Congressional and other inquiries regarding program activities.

1. Respondent Universe and Sampling Methods

CDC does not plan to use any statistical methods to select any respondents because all funded states and territories will be required to use the progress reporting system.

2. Procedures for the Collection of Information

The information will be collected using the described password protected web-based system. Respondents will log into the system at their worksite computer and provide progress reporting information through prompted data entry points.

The respondents will receive training on use of the application and on the required report content prior to their first reporting deadline of February 2007. Respondents will be informed of their reporting deadlines via semi-annual notification letters received from the Procurement and Grants Office (PGO) and via emails sent by the CDC Division of Oral Health to all known users of the system. Respondents will not be re-interviewed or contacted for data validation.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Respondents are required to file twice yearly progress reports in order to continue to receive level federal funding in support of cooperative agreement 3022. Respondents are encouraged to use the web-based system to file these reports, but are not required to do so. Once data has been entered there will be a reduction in the burden hours to the state grantees; therefore, no efforts will be made to maximize respondent use rates. However, rates are expected to be 100%.

4. Tests of Procedures or Methods to be Undertaken

The system will undergo rigorous application testing, including fidelity and usability testing of system design, accuracy and comprehension testing of proposed data elements, and pilot testing of the online system. These tests will be performed using less than 10 respondents per test. Respondents will be culled from the external workgroup (see Attachment 3).

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and /or Analyzing Data

No individuals will be consulted on statistical aspects of the design as statistical methods will not be used in analysis of the information.

The individuals responsible for design of the data collection system include:

Karen Sicard, Division of Oral Health, Centers for Disease Control and Prevention, (770) 488-5839, [email protected]

Jeanne Casner, Northrop Grumman Mission Systems (contractor), (678) 530-3522, [email protected]

ATTACHMENT 1

APPLICABLE SECTIONS OF LAWS OR REGULATIONS

TITLE III—GENERAL POWERS AND DUTIES OF PUBLIC HEALTH SERVICE

Part A—RESEARCH AND INVESTIGATION

IN GENERAL

Sec 301. [241] (a) The Secretary shall conduct in the Service and encourage, cooperate with, and render assistance to other appropriate public authorities, scientific institutions, and scientists in the conduct of, and promote the coordination of, research, investigations, experiments, demonstrations, and studies relating to the causes, diagnosis, treatment, control, and prevention of physical and mental diseases and impairments of man, including water purification, sewage treatment, and pollution of lakes and streams. In carrying out the foregoing the Secretary is authorized to—

(1) collect and make available through publications and other appropriate means, information as to, and the practical application of, such research and other activities;

(2) make available research facilities of the Service to appropriate public authorities, and to health officials and scientists engaged in special study;

(3) make grants-in-aid to universities, hospitals, laboratories, and other public or private institutions, and to individuals for such research projects as are recommended by the advisory council to the entity of the Department supporting such projects and make, upon recommendation of the advisory council to the appropriate entity of the Department, grants-in-aid to public or nonprofit universities, hospitals, laboratories, and other institutions for the general support of their research;

(4) secure from time to time and for such periods as he deeps advisable, the assistance and advice of experts, scholars, and consultants from the United States or abroad;

(5) for purposes of study, admit and treat at institutions, hospitals, and stations of the Service, persons not otherwise eligible for such treatment;

(6) make available, to health officials, scientists, and appropriate public and other nonprofit institutions and organizations, technical advice and assistance on the application of statistical methods to experiments, studies, and surveys in health and medical fields;

(7) Enter into contracts, including contracts for research in accordance with and subject to the provisions of law applicable to contracts entered into by the military departments under title 10. United States Code, sections 2353 and 2354, except that determination, approval, and certification required thereby shall be by the Secretary of Health, Education, and Welfare, and

(8) adopt, upon recommendations of the advisory councils to the appropriate entities of the Department or, with respect to mental health, the National Advisory Mental Health Council, such additional means as the Secretary considers necessary or appropriate to carry out the purposes of this section.

The Secretary may make available to individuals and entities, for biomedical and behavioral research, substance and living organisms. Such substances and organisms shall be made available under such terms and conditions (including payment for them) as the Secretary determines appropriate.

(b)(1) The Secretary shall conduct and may support through grants and contracts studies and testing of substances for carcinogenicity, teratogencity, mutagenicity, and other harmful biological effects. In carrying out this paragraph, the Secretary shall consult with entities of the Federal Government, outside of the Department of Health, Education, and Welfare, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct for such entity studies and testing of substances for carcinogencity, teratogenicity, mutagenicity, and other harmful biological effects.

(2)(A) The Secretary shall establish a comprehensive program of research into the biological effect of low-level ionizing radiation under which program the Secretary shall conduct such research and may support such research by others through grants and contracts.

(B) The Secretary shall conduct a comprehensive review of Federal programs of research on the biological effects of ionizing radiation.

(3) The Secretary shall conduct and may support through grants and contracts research and studies on human nutrition, with particular emphasis on the role of nutrition in the prevention and treatment of disease and on the maintenance and promotion of health, and programs for the dissemination of information respecting human nutrition to health professionals and the public. In carrying out activities under this paragraph, the Secretary shall provide for the coordination of such of these activities as are performed by the different divisions within the Department of Health, Education, and welfare and shall consult with entities of the Federal Government, outside of the Department of Health, Education, and Welfare, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct and support such activities for such entity.

(4) The Secretary shall publish a biennial report which contains--

(A) a list of all substances (I) which either are known to be carcinogens or may reasonably be anticipated to be carcinogens and (ii) to which a significant number of persons residing in the United States are exposed;

(B) information concerning the nature of such exposure and the estimated number of persons exposed to such substances;

© a statement identifying (I) each substance contained in the list under subparagraph (A) for which no effluent, ambient, or exposure standard has been established by a Federal agency, and (ii) for each effluent, ambient, or exposure standard established by a Federal agency with respect to a substance contained in the list under subparagraph (A), the extent to which on the basis of available medical, scientific, or other data, such standard, and the implementation of such standard by the agency, decreases the risk to public health for exposure to the substance; and

(D) a description of (I) each request received during the year involved—

(I) from a Federal agency outside the Department of Health, Education, and Welfare for the Secretary, or

(II) from an entity within the Department of Health, Education, and Welfare to any other entity within the Department, to conduct research into, or testing for, the carcinogencity of substances or to provide information described in clause (ii) of subparagraph (C), and (ii) how the Secretary and each such other entity, respectively, have responded to each such request.

(5) The authority of the Secretary to enter into any contract for the conduct of any study, testing, program, research, or review, or assessment under this subsection shall be effective for any fiscal year only to such extent or in such amounts as are provided in advance in Appropriation Acts.

The Secretary may conduct biomedical research, directly or through grants or through grants or contracts for the identification, control, treatment, and prevention of diseases (including tropical diseases) which do not occur to a significant extent in the United States.

The Secretary may authorize persons engaged in biomedical, behavioral, clinical, or other research (including research on mental health, including research on the use and effect of alcohol and other psychoactive drugs) to protect the privacy of individuals who are the subject of such research by withholding from all persons not connected with the conduct of such research the names or other identifying characteristics of such individuals. Persons so authorized to protect the privacy of such individuals may not be compelled in any Federal, State, or local civil, criminal, administrative, legislative, or other proceedings to identify such individuals.

PROJECT GRANTS FOR PREVENTIVE HEALTH SERVICES

Sec. 317. (k)(1) The Secretary may make grants to States, political subdivisions of States, and other public and nonprofit private entities for—

(A) research into the prevention and control of diseases that may be prevented through vaccination;

(B) demonstration projects for the prevention and control of such diseases;

(D) education, training, and clinical skills improvement activities in the prevention and control of such diseases for health professionals (including allied health personnel).

(2) The Secretary may make grants to States, political subdivisions of States, and other public and nonprofit private entities for—

(A) research into the prevention and control of diseases and conditions;

(B) demonstration projects for the prevention and control of such diseases and conditions;

(D) education, training, and clinical skills improvement activities in the prevention and control of such diseases and conditions for health professionals (including allied health personnel).

(3) No grant may be made under this subsection unless an application therefore is submitted to the Secretary in such form, at such time, and containing such information as the Secretary may by regulation prescribe.

(4) Subsections (d), (e), and (f) shall apply to grants under this subsection in the same manner as such subsections apply to grants under subsection (a).

ATTACHMENT 2

FEDERAL REGISTER NOTICE ANNOUNCING

THE 60-DAY PUBLIC COMMENT PERIOD

ATTACHMENT 3

NAME, EMAIL ADDRESSES AND PHONE NUMBERS

OF ORAL HEALTH COOPERATIVE AGREEMENT 3022

STATE AND TERRITORIAL MIS WORK GROUP

Name	Email Address	Phone Number
Brad Whistler	[email protected]	(907)4658628
Donna Altshul	[email protected]	(518)474-1961
Laurie Leonard	[email protected]	(401)222-7633
Linda Altenhoff	[email protected]	(512)458-7111
Lynn Mouden	[email protected]	(501)280-4111
Maureen Ross	[email protected]	(401)222-7633
Tracy Anselmo	[email protected]	(303)692-2569
Tim Cook	[email protected]	(518)474-1961

Attachment 4

RESULTS OF THE USABILITY STUDY

DOH Management Information System (MIS)

Management Overview for Logistics, Analysis, and Reporting (MOLAR)

v1.0

Usability Study Results

Northrop Grumman Web Applications Team
Coordinating Center for Health Promotion (CoCHP)
Centers for Disease Control and Prevention (CDC)

Summary xiv

Method xiv

Purpose xiv

Objectives xiv

Issues and Recommendations xiv

Other Participant Comments xxii

Pre-Test & Post-Test Findings xxiii

Appendix 1: xxiv

Appendix 2: Participant Tasks xxvii

Appendix 3: Post-Test Questionnaire Results xxxi

Document History

Version	Date	Comments	Author
1	03/09/06	Initial Creation	Aimee Murphy

Summary

This document is a summary of the Division of Oral Health Management Overview for Logistics, Analysis, and Reporting (MOLAR) Usability Study Findings which describes the analysis of the data collected during testing. Eight state participants located throughout the United States were tested over the span of a week in February 2006. Because the navigational model and participant interface are based on existing management information systems which were rigorously tested the issues discovered in the MOLAR system were minor.

Also included in this document is a set of recommendations to address the issues discovered as a result of the study, as well as other findings based on participant observation, reactions and suggestions. All recommendations are based upon both quantitative (pre- and post-test questionnaire data) and qualitative (observations of remote participant behavior during the test and post-test discussions with participants) data.

Method

Five usability sessions were conducted. At three of the sessions two representatives from different states were partnered. The participant teams were presented with nine written task scenarios, typical of a task they would perform, and then asked to complete the task in the prototype. For each of these sessions, the co-discovery protocol was used. Co-discovery is a technique in which two participants work together to perform the tasks within a scenario. In the other two sessions, a single participant performed each task. All participants were encouraged to talk aloud (and to each other) as they worked. This format was used to increase verbal feedback while using the system.

Purpose

The purpose of this usability study is to ensure the proposed MOLAR design is both learnable and useable by the state public health staff that will utilize the system. From this study, we sought to understand whether the primary participant community could utilize the system with accuracy and ease. When usability issues were encountered, we worked to uncover the reason for the difficulties so we may provide recommendations to correct the problems in this report.

Emphasis was placed on the following aspects of participant interface design during usability testing:

Ease of use
Functional flow between screens
Consistency of terminology, labeling, and screen layouts
Error identification and recovery
Participant satisfaction ratings

Objectives

The objectives of the usability study were to determine:

if participants can complete each task
which click stream is most common for participants.
if everything easily available within the interface.
if the information architecture make sense to the participants.
If participants understand how to add, edit and delete information
If participants are able to successfully attach documents.

Issues and Recommendations

The usability study revealed issues that ranged from mild to moderate; none of the issues were classified as severe. Each issue is identified below. After each issue, a recommendation for improvement is provided. Other observations, participant responses and recommendations are also documented.

Issue #1: Staff Page Location
Priority Level: High

Issue Description

In 5 out of 5 tests, participants had a great deal of difficulty locating the page where the “Add Staff” link resided. All of the participants exhausted the sub-navigational options under the General Information tab, which included visiting and fully exploring Contact Information, Program Overview and Organizational Chart pages prior to visiting the Resources tab (Figure 1.1). One team of two participants tried to add a new staff member by editing the Program Contact Information.

Once participants located the Staff page, 4 out of 5 commented that they do not consider humans as resources. They consider resources to include only the non-human resources. Participants were able to add a new resource with ease, once they located the entry page. In all five tests, participants were adept at editing the staff information.

Participant Reactions

“I think of staff as a part of our General Program. I would never look for it in Resources.”
“Will the system require us to add more information than I’m accustomed to adding? It seems like they are asking for a lot of information, much more than I usually include.”
“Why do I have to enter all this extra information for staff?”
“People are resources? Well, I guess that makes sense when you really think about it.”
“To me, resources are like money, partners, etc. – not our staff members.”

Recommendations

Consider collecting staff information in the General Information section rather than the Resources section. The suggested placement of the ‘Staff’ link is as follows (Figure 1.1)

Figure 1.1

Issue #2: Placement of Add Objective
Priority Level: Moderate

Issue Description

When asked to add a new objective, participants indicated that they wanted to add an objective under a specific 5-year goal instead of using the general “add objective” link currently located at the upper right corner of the page. Figure 2.1 below illustrates the current layout.

Adding an objective for a specific 5-year goal could pose an additional challenges if an objective is related to more than one 5-year goal. When discussed with participants during post-test discussions, participants noted that it would be very rare that an objective relates to more than one 5-year goal.

Figure 2.1

Participant Reactions

“There’s no way for me to add my objective under coalitions. There’s no option next to the objective title.”
“The last time I added an objective, it (the system) automatically knew to put my objective under State Oral Health Plan. I’m not sure how that happened.”
“In our state, it is rare that an objective will support more than one five-year goal. We rarely see overlap.”
“I think it would be easier if you put the ‘add’ next to each goal. I thought this page was just a listing – sort of like a table of contents - and that I couldn’t add anything to it.”

Recommendations

Remove the general “Add Objective” link from the upper right corner of the page. Insert an “Add Objective” link next to each 5-year goal category as shown in Figure 2.2. When the add page is opened, the goal category would be pre-selected and participants could also select additional goal categories if necessary.

Figure 2.2

Issue #3: Updating Activity Status and Timeline on Progress Page

Priority Level: Moderate

Issue Description

On the activity revisions page, the following question appears:
*Does the activity status, start date or end date need to be revised? Yes/No

If the participant select yes, the system displays date and explanation text boxes for the revision of the following (see Figure 3.1)

Reason for status revision
Reason for activity start date change
Reason for activity end date change

Recommendations

Consider changing the question ‘Does the activity status, start date or end date need to be revised?’ to ‘What aspects of the activity needs to be revised?’ with the options ‘Status’, ‘Start Date’, and ‘End Date’ (figure 3.1 and 3.2). The remaining questions will be filtered depending upon which elements are selected therefore reducing the display of extraneous fields that the participant does not need to complete.

Figure 3.1

Figure 3.2

Issue #4: Identification of Active Tab and Secondary Menu Relationship
Priority Level: High

Issue Description

Several participants experienced difficulty interpreting the navigational cues embedded in the tabbed navigation because the active tab is a different background color than the secondary navigation. The background colors are incongruent and do not communicate the relationship between the primary and secondary navigational elements.

Recommendation

Secondary navigation background should be the same color as the active tab.

Figure 4.1

Issue #5: Content Issues
Priority Level: Medium

Issue Description:

Several participants had comments about the content of MOLAR. These comments are listed below:

In 3 out of 5 tests, participants were very confused about the Ethnicity question under “Identify the target population(s) from the burden report”. Many asked why the options were dealt with Hispanic or Latinos. One participant asked “Is it even worthwhile to ask this? Why do they need to know this?”
“Our Burden document covers people of all ages. Why are they asking target population ages? CDC wants our burden report to be comprehensive and cover everyone. We’re following their guidance to serve all ages, but then they ask us what ages we are targeting? This doesn’t make sense.”
When adding an objective, participants asked why the status defaulted to “proposed”. Three participants asked how their objective would be approved and who would approve it.
On the Add Objective page, one participant suggested changing the help text next to the question: *Measure of success “filtered based on selected 5 yr goal”. The participant suggested that we change “filtered based on selected 5 yr goal” to “For each 5-year goal selected, chose the measures of success”.

One participant team was particularly confused by the use of the term “product”. They did not have a strong sense of what we were asking for in this question.

Recommendation

CDC Work group discuss user content questions/concerns and advise. These content issues do not affect the user interface design.

Issue #6: Ability to Print screens
Priority Level: Low

Issue Description:

Some participants expressed concern about gathering the data necessary to complete their grant applications online. Several individuals noted that they currently divide up certain parts of the application among coalition coordinators, health communication specialist, etc. It was suggested that a print-friendly version of data entry pages be provided. This will allow the main MOLAR state user to communicate with team members regarding the type of information needed for the grant application.

Participant Reactions

“Our coalition coordinator, health communication specialists, etc are all in charge of different sections when we submit our application. This is going to be hard to divide responsibilities now because they will need to submit all of their information to a staff member who will enter it into your system. We need a way to communicate with all these other people what the CDC is asking for. Can we print the form pages so we can ask for the correct data ahead of time?”

Recommendation

Provide print-friendly versions of data entry pages in a future version of MOLAR.

Other Participant Comments

“It seems to be that our state puts more on paper than this system requires. I’m skeptical that they missed something.”
“Are you going to train us on this system? We’re going to need a definitions sheet to make sure we are on the same page with terminology.”
“The system looks like it could be interesting, especially since it will replace our current reporting system. I think it is easy to use if you know where you are going. It takes awhile to find your way around.”
“I’m feeling overwhelmed. This is a tremendous amount of work; I will have to find staff to devote to this. How can I account for this in my budget? The program reporting and application processes are daunting. Information is formatted differently, so this is all new. It is going to take an inordinate amount of time to learn this. We don’t have any money in our budget to spare but now we’ll have to fuss with this new system and it will take money that we don’t have. All in all, if I had staff, I’d be a happy camper. I’m just saying that fitting this in with everything else will be difficult.”
“This system seems pretty easy once you know your way around.”

“It would be really nice if the cursor jumped to the next field so I don’t have to use my mouse after each answer.”

“Can you make it jump to the next field automatically?”

Participants suggested that, when appropriate, we alphabetically rearrange our list boxes, checkboxes, scrolling check boxes, etc. to aid when searching for specific options.

Pre-Test & Post-Test Findings

A pre-test and post-test questionnaire was administered prior to and after the usability test, respectively. From these questionnaires, we were able to gather additional data regarding the background of the participants, their technical skill level, and their reactions to the MOLAR application. Listed below are some key findings and suggestions.

All participants
- Use PCs
- Use computer mostly at work
- Utilize IE 6.0 as their browser
- Spend more than 4 hours using a computer each day
- Listed checking email, using the Internet/Intranet, and software programs such as Word and Excel as their most frequent computer related activities.
7 participants rated their computer experience as Intermediate, while 1 rated as Expert.

5 participants rated their internet experience as Intermediate, while 3 said they were Experts.
Participants used one of these three ways to ask for help when having difficulty using a website.
- Refer to manuals
- Ask someone
- Contact site administrator

Participants liked the application for providing centralized program status information available throughout the year. They felt that site was easy to navigate and easy to read.
50% of participants were not sure where to look for certain things.
Participants asked for the addition of online Help features.
The availability of Participant’s guide, FAQ or definition sheet was one of the suggested improvements for the application.
The participants suggested the addition of an automatic email reminder to alert them when updates to the system are due.
The availability of the system year-round was appealing to participants.
Participants like the idea that MOLAR has the potential to reduce paperwork. Many commented that they looked forward to having a centralized area to store program information.
What did you like least about the MOLAR application?
- Were unsure where to look for certain things.
- Some participants felt this system would not save time.
- Some definitions didn’t match those in the participants’ current grant applications.

Appendix 1: Pre-Test Questionnaire Results

Participants in the usability testing answered a questionnaire before the beginning of the test. The questionnaire was used to understand background and experience of the participants. Eight participants returned the questionnaire. The results are below.

Questions	Responses
What is your current job title?	Program Administrator - 2 participants Bio Statistician - 1 participant Dental Director - 1 participant State Oral Health Chief - 1 participant CDC Program Coordinator - 1 participant Acting Director - 1 participant Evaluation Consultant - 1 participant
How long have you held your current position?	10+ years – 1 participant 5+ years – 2 participants 3+ years – 2 participants Les than 1 year – 3 participants
Have you been involved in discussions or creation of the MOLAR application?	Yes – 3 participants No – 5 participants
3a. How have you been involved in the process?	Observer Helped pick the name
In general, how do you feel about working with computers?	Don’t like – 0 participants No Strong like or dislike – 1 participant Like Working – 5 participants Other – 2 participants (frustrated when things do not work their way)
What type of computer do you use most often?	PC – 8 participants MAC - 0
On average, how much time do you spend using a computer each day?	More than 4 hours – 8 participants
Where do you use your computer most often?	At Work - 8 participants
How would you rate your computer experience?	Intermediate – 7 participants Expert – 1 participant
When having difficulty using a web site, do you ever use “online help” to solve your problem?	Yes – 5 participants No – 1 participants Sometimes – 2 participants
What other ways do you ask for help when having difficulties on a web site?	Refer Manuals Ask someone Contact site administrator
How many hours do you spend on the Internet per week (including office and personal use)?	1-5 hour – 3 participants 6-10 hour – 3 participants 15+ hour -2 participants
In general, what are the most frequent computer-related activities you do?	Email Internet/Intranet Software program/application – MS Word, MS Excel
What resolution is your monitor set to?	1024 x 768 – 4 participants 800 x 600 - 1 participant Didn’t know – 2 participants Did not answer – 1 participant
How would you rate your internet experience?	Intermediate – 5 participants Expert – 3 participants
What browser type and version do you most often use?	IE 6.0 – 8 participants
Which activities have you performed online?	Ordered a product/service from a business, government or educational entity by filling out a form on the web – 8 participants Made a purchase online for more than $100 – 7 participants Created a web page – 0 participants Customized a web page for yourself (e.g. MyYahoo, CNN Custom News) – 3 participants Changed your browser's "startup" or "home" page – 6 participants Changed your "cookie" preferences – 6 participants Participated in an online chat or discussion (not including email) – 6 participants Listened to a radio broadcast online – 4 participants Made a telephone call online – 2 participants Used a nationwide online directory to find an address or telephone number – 5 participants Taken a seminar or class about the Web or Internet – 4 participants Bought a book to learn more about the Web or Internet – 2 participants

Appendix 2: Participant Tasks

Task 1: Add a staff member

You need to add a third staff member, Hank Smith, MPH. Below is the information you will need:

Name: Hank Smith, MPH

Address: 2343 Burnt Mill Drive, Suite 100, Atlanta GA 30306

Telephone number: 404-555-1234 ext. 0002

Email address: [email protected]

Position Title: Program Manager

Overall OH Program Time Allocation: 100%

Cooperative Agreement Program Time Allocation: 100%

Primary Role: Program Manager

Other roles and time allocation: Grant writer: 10%

MIS Contact: 5%

Program Manager: 85%

% of salary funded by CDC DOH: 50%

Other funding sources: Permanent state dollars, CDC prevention block grant

Employment type: State employee

Date Started: 10/2004

Task 2: Add Burden Report Information

You need to add information about your state’s draft burden report. Use the information below to complete the task:

Title: Oral Health Burden in Georgia

Status: Draft

Anticipated Publish Date: 06/2006

Progress to date: We are in the final stages of data analysis after several months of data gathering and updating. Overall, the report is around 85% complete.

Task 3: Add Burden Report Information

Your burden report has now been published. Update your existing information with the following:

Title: The Burden of Oral Disease in the State of Georgia

Status: Published

Date Published: July 2006

Date Last Revised: June 2006

Web address: http://www.gaburdenreport.org

Dissemination: Academia/School

Business/industry sector

General public

Media

Target Populations:

Race: African American or Black

Asian

White

Ethnicity: Hispanic or Latino

Gender: Male, Female

Geography: City

Age: 20-49 years

Income: Medicaid eligible

Data Sources used:

ASTDD State Synopsis

My Water's Fluoride (MWF)

Chronic Disease Indicators (CDI)

Indicators consistent with:

NOHSS:

Percentage of people who visited the dentist or dental clinic within the past year.

ASTDD State Synopsis:

Number of dentists in the state.

WFRS:

Population served by public water system.

Burden report includes description of:

Oral health assets

Oral health burden

Oral health disparities

Task 4: Add an objective

Your state’s oral health program needs to develop a statewide OH plan pinpointing issues which will identify oral health needs, available resources, and strategies to address the need. To accomplish this, you need to create a SMART objective. Use the information below to complete this task:

Title: Develop Oral Health State Plan
5 Year Goal: Develop or update a comprehensive State Oral Health Plan
Measure of success: Plan addresses oral health infrastructure including current resources, gaps in resources and recommendations for their elimination
Baseline and Target: Your state plan has not yet been started and the target is to be 100% complete by the end date of this objective.
Evidence for successfully measuring target: The state plan document will be completed and published
Describe how this objective will establish….. : We will have a comprehensive state plan addressing our state’s oral health needs over the next 5 years.
Start date: 01/2006
End date: 01/2007

Task 5: Add another objective

Your state’s Oral Health Program needs additional partners on the statewide coalition to be consistent with the coalition framework as recommended by CDC. To accomplish this, you need to create a SMART objective. Use the information below to complete this task:

Title: Enhance statewide coalition
5 Year Goal: Establish and sustain a diverse statewide oral health coalition.
Measure of success: Coalition membership represents categories in the coalition framework
Baseline and Target: Your coalition has 10 existing members and plans to increase to at least 15 total members who span the framework categories
Evidence for successfully measuring target: The statewide coalition will be representative of all categories in the coalition framework, thus enhancing our state program’s reach.
Describe how this objective will establish….. : The framework will be diverse which will help to ensure that the program’s objectives will have the greatest impact.
Start date: 07/2006
End date: 06/2007

Task 6: Add an activity

An important step when developing your oral health state plan is to work with your partners and the coalitions to draft the plan and determine priorities. Use the information below to complete this task:

Title: Determine state priorities and draft plan

Description: Work with existing partners and coalitions to determine priorities to be included in the state plan and then to create a draft of the state plan.

The lead role is assigned to the activity is the program coordinator. No contactors working on this activity. The partners involved are the State Board of Education, Dentists Association of Georgia, and Hygienists Association of Georgia.

Partners are donating personnel to aid in this activity.

The activity duration is 01/2006 – 06/2006.

Task 7: Add activity progress for the activity previously entered (draft state plan)

You need to enter a progress record for the activity you previously entered. You have started work on the state plan, but need to extend the due date. Enter progress for this activity using the information below:

Date progress occurred: 02/01/2006.

Describe progress: Priorities for the state plan have been determined, but have not yet started drafting the report due to a lack in resources.

Revisions? Yes

Revise activity end date: 12/2006

Explain: We’ve had to extend the end date because our lead staff member has taken another position with the state health department and we are working to replace this staff member. We are currently interviewing potential candidates and hope to have a replacement by 4/2006.

Task 8: Add progress for the Objective that you entered (coalition) in Task 5.

Time has passed and it’s now August 2006. You need to enter progress on the objective you entered regarding enhancement of your statewide coalition. Use the information below to complete the task:

Date progress occurred: 07/06/2006

Describe progress: Held an introductory meeting for potential members to introduce them to the state oral health program’s goals and ask questions about the state program.

Target met? Since the objective is still ongoing, the target has not yet been met.

No current measure is applicable, barriers or plans to overcome barriers, no other outcomes, and no revisions to this objective.

Task 9: Add a product

Because of the success and positive outcome for the coalition objective, Lisa Moore, the program coordinator, has written an article called “Coalition Member Recruiting” to appear in JAPHA. You need to enter this article as a product in the MIS.

Title: Coalitions Recruiting

Description: This document contains lessons learned and success stories based on our program experiences when trying to enhance our existing coalition by recruiting new members and diversifying our membership.

Attach file

Title: Coalition recruiting

File location: Click Browse to find a file.

Date file last revised: 01/2006

Can this document be shared? Yes, you are willing to share this article with any other state programs who are interested.

Appendix 3: Post-Test Questionnaire Results

Participants in the usability testing answered a questionnaire after the testing was completed. The questions allowed participants to rank their feelings toward aspects of the interface using a Likert scale. Six participants returned the questionnaire. The results are below.

Questions	Responses Strongly Agree ------ Strongly Disagree
I was able to find what I needed quickly and easily.	Agree – 1 participant Somewhat agree – 2 participants Neither agree nor disagree – 2 participants Disagree – 1 participant
When navigating, I had a clear sense of where I was in the website.	Agree – 1 participant Somewhat agree – 3 participants Neither agree nor disagree – 1 participant Disagree – 1 participant Strongly disagree – 1 participant
The site is consistently designed.	Agree – 3 participants Somewhat agree – 2 participants Disagree – 1 participant
The site is easy to read.	Agree – 2 participants Somewhat agree – 2 participants Neither agree nor disagree – 1 participant Somewhat disagree – 1 participant
The link labels are easy to understand.	Somewhat agree – 4 participants Neither agree nor disagree – 1 participant Strongly disagree – 1 participant
The site was difficult to navigate.	Strongly Agree – 1 participant Somewhat agree – 1 participant Neither agree nor disagree – 2 participants Somewhat disagree – 1 participant Disagree – 1 participant
All major parts of the site are accessible from the home page.	Agree – 1 participant Somewhat agree – 3 participants Neither agree nor disagree – 1 participant
	Very Easy ------ Very Difficult
Rate how difficult it was to learn the MOLAR application.	Very easy – 1 participant Somewhat easy – 2 participants Neither easy nor difficult – 1 participant Somewhat difficult – 1 participant Difficult - 1 participant
Rate the understandability of terms and labels used throughout MOLAR.	Somewhat easy – 3 participants Neither easy nor difficult – 1 participant Difficult – 2 participants
	Very Satisfied ------- Very Unsatisfied
Rate your satisfaction with the information presented and its layout.	Satisfied – 2 participants Somewhat satisfied – 2 participants Somewhat unsatisfied – 2 participants
Rate your satisfaction with the appearance of the site.	Satisfied - 2 participants Somewhat satisfied – 3 participants Neither satisfied nor unsatisfied – 1 participant
Considering all factors (ease of learning, ease of use, ease of moving between pages, ease of locating information) provide an overall rating of your satisfaction with the site.	Satisfied – 1 participant Somewhat satisfied – 3 participants Somewhat unsatisfied – 2 participants

What did you like best about the Molar application?
- Pretty easy to navigate
- Able to add information throughout the month/year
- Easy to read. Layout was organized well
- Potential for paperwork reduction. Location for centralized program status information

What did you like least about the Molar application?

Wasn’t sure where to look for certain things
Don’t see it ever saving time
Some definitions didn’t match those in our current grant applications
Limitation in ability to cross reference objectives

What improvements would you recommend for MOLAR?

Print seems a little small and not bold enough
Participant’s guide, FAQ, or definition sheet
Too soon to tell

What future capabilities would you like to see?

Automatic reminders for updates
Cursor automatically moved from one field to the other when entering dates
Interaction with project officer. Ability to have multiple folks enter data into system
Need a person designated to this system

ATTACHMENT 5

PORPOSED MIS DATA COLLECTION INSTRUMENT

Form Approved

OMB NO.____________

Exp. Date ____________

Division of Oral Health

Semi-Annual Progress Report

Public reporting burden of this collection of information is estimated to average 9 hours per response (semi-annual and annual report), including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ASTDR Reports Clearance Officer; 1600 Clifton Road NE, MS D024, Atlanta, Georgia 30333; ATTN: PRA (0920-XXXX)

Overview

The following table defines the data proposed for collection through the CDC Division of Oral Health Information System (DOH IS). For each key section, the question and response options are identified. If the response option is labeled “text”, the responder can enter free form text. Questions marked with an asterisk (*) indicate a required question.

Information Sections

The data collected is grouped according to the key sections listed below.

General Program Information

Staff

Partners

Contracts

Statewide/Community-Based Coalition

Budget Detail And Justification

Systemic, Socio-political, and Policy Change Assessment

Disease Burden, Priority Population, and Unmet Needs

Data Sources

Work Plan Objectives

Work Plan Objective Progress

Work Plan Activities

Work Plan Activity Progress

Work Plan Products

General Program Information

Question	Response Options
Program Contact Information
Mailing address line 1*	Text
Mailing address line 2	Text
Mailing city*	Text
Mailing state*	Select from list of states
Mailing zip*	Text
Shipping address line 1*	Text
Shipping address line 2	Text
Shipping city*	Text
Shipping state*	Select from list of states
Shipping zip*	Text
Program telephone*	Number
Program fax	Number
Program web address	Text
Principle Investigator*	Text
Principle Investigator Telephone*	Text
Business Official*	Text
Business Official Telephone*	Text
Funded for fluoridation program (10A)?*	Yes/No, list first year of funding
Funded for sealant program (10B)?*	Yes/No, list first year of funding
Program Overview
Program type*	Select one: Capacity Building Basic Implementation
Program summary*	Text
Program goals*	Text
Organization Chart
File name*	Text
Type*	Select one: Overall state health structure State health agency structure Oral health program structure
Date last revised*	Month and Year

Staff

Question

Response Options

First name*

Text

Middle name

Text

Last name*

Text

Credentials

Text

Address same as program mailing address

Select one:

Yes

Address line 1*

Text

Address line 2*

Text

City*

Text

State*

Text

Zip*

Number

Telephone*

Number

E-mail address*

Text

Position title*

Text

Overall oral health program time allocation*

Percent

Program time allocation working on cooperative agreement*

Percent

Primary role within oral health program*

Select one:

Administrative support

Agency manager

Budget manager

Coalition coordinator

Community developer

Computer technology support

Cooperative agreement program contact

Data analyst

Data manager

Dental consultant

Dental director

Dental sealant coordinator

Epidemiologist

Evaluation specialist

Fluoridation engineer

Fluoridation specialist/coordinator

Grant writer

Health communication specialist

Health educator

MIS contact

Policy developer

Principle investigator

Program coordinator

Program manager

Regional consultants

Web designer

Other (specify)

Indicate all roles performed including the primary role of this staff member and the percent of overall program time allocation for each role. *

(the total of all roles FTE must add up to the overall FTE)

Select all that apply:

Role	% of Overall FTE
Administrative support	Percent
Agency manager	Percent
Budget manager	Percent
Coalition coordinator	Percent
Community developer	Percent
Computer technology support	Percent
Data analyst	Percent
Data manager	Percent
Dental consultant	Percent
Dental director	Percent
Dental sealant coordinator	Percent
Epidemiologist	Percent
Evaluation specialist	Percent
Fluoridation engineer	Percent
Fluoridation specialist/coordinator	Percent
Grant writer	Percent
Health communication specialist	Percent
Health educator	Percent
MIS contact	Percent
Policy developer	Percent
Program coordinator	Percent
Program manager	Percent
Regional consultants	Percent
Web designer	Percent
Other (specify)	Percent

What percent of the primary role’s overall FTE is funded by CDC DOH? *

Percent

Please identify what other sources fund this staff member's salary*

Select all that apply:

Permanent state dollars

One-time only state dollars

CDC/DOH core dollars

CDC/DOH supplemental dollars

Maternal Child Health block grant

CDC prevention block grant

Other (specify)

Employment type*

Select one:

State employee

State outsourced contract

Temporary state employee

Other (specify)

Date started with state oral health program*

Month and Year

Date finished with state oral health program

Month and Year

Curriculum vitae/resume*

Text – file name

Date last revised

Month and year

Partners

Question	Response Options
Partner organization*	Text
Contact first name*	Text
Contact last name*	Text
Address line 1	Text
Address line 2	Text
City	Text
State	Text
Zip	Number
Telephone	Number
E-mail address	Text
Website	Text
Partner Status*	Select one: Active Inactive
Is this partner a member of a statewide or community-based oral health coalition?*	Select one: Yes (Select coalition) No
Partner level*	Select one: National Regional State District/Local
Partner type*	Select up to 3: Academia/education Advocacy group An Individual Business/industry sector Civic organization Community based organization Community health center Cultural organization District or local government agency Environmental agency Faith-based organization Federal government agency Foundations Healthcare organization Nonprofit organization Other government agency Organization representing priority population Prevention research center Professional association Public health official Quality improvement organization State government agency Volunteer agency Other (specify)
Contributions*	Select all that apply: Communication network access Conference sponsor Consultation Data analysis Epidemiology Equipment Evaluation Funding Media Personnel Supplies and equipment Training/education Travel assistance Visibility (credibility) Other (specify)
Evidence of collaboration*	Select all that apply: Joint dedication of resources Letter of support Memorandum of agreement (MOA) Memorandum of understanding (MOU) Other (specify)

Contracts

Question	Response Options
Organization name*	Text
Contact first name*	Text
Contact last name*	Text
Address line 1	Text
Address line 2	Text
City	Text
State	Text
Zip	Number
Telephone	Number
E-mail address	Text
Website	Text
Is this contractor fulfilling the role of a staff member for the state health department*	Select one: Yes No
Primary role(s)*	Select all that apply: Administrative support Coalition coordination Community development Computer technology/support Data analysis Data collection Data management Dental sealant coordination Epidemiologist Evaluation Facilitator Fluoridation engineering Fluoridation coordination Grant writing Health communication Health education Meeting/conference facilitation Policy development Program consultant Public relations Regional consultants Training Web/Application designer Other (specify)
Contract amount*	Number
Contract Attachment*	File Name - Text
Date Last Revised*	Date
Type*	Select one or more: Meeting minutes Method of accountability Method of selection Period of performance Scope of work

Statewide/Community-Based Coalition

Question	Response Options
Coalition Name*	Text
Type*	Select one: Community Regional Statewide Other (Specify)
Number of members	Number
Member composition*	Select all that apply: Government: Social services Environmental health State/Local Health Department Interagency and/or Interdepartmental Steering Committee Other (specify) Community: Business leader Community water supervisor/manager Community-based clinic Faith-based organization Foundation Local community health department Other (specify) Education: Local school administrator Parent Teacher Association School nurse association Education Regional staff Other (specify) Third Party Payers: Insurance Managed care Medicaid Other (specify) Policy Makers: Federal legislator Local/community policy maker Policy advocate State legislator Other (specify)
Meeting frequency*	Select one: Monthly Quarterly Semi-annually Annually Other (specify)
Priority focus areas*	Select all that apply: Infants and toddlers Children Adolescents Adults Older adults Access Aging population Assessment Caries Communications/marketing Disparity Education Evaluation Fluoridation Funding Infection control Infrastructure Injury prevention Oral and systemic disease Oral cancer Periodontal disease Policy Program/system sustainability Sealants Surveillance Tobacco cessation Work force Other (specify)
Does a specific group within the coalition address any of the following priority areas?*	Infrastructure Yes No Fluoridation Yes No N/A, HP2010 has been met Sealants Yes No N/A, HP2010 has been met
List Any Coalition Sub-Groups	Text
Sustainability evidence type*	Select all that apply: 501c3 status By-laws Clerical staff support Established internal communication network Evaluation of coalition and coalition activities Funding and institutionalization Stakeholder maintenance/list Letter of support Leveraging resources Meeting minutes/schedules Membership maintenance/list Memorandum of agreement/understanding Policy development Products & impact SMART action plan development and implementation Visibility Written priorities/plans/strategies Written vision/mission statements Other (specify)
Attachment*	File Name - Text
Date Last Revised*	Date
Type*	Select all that apply: 501c3 status By-laws Clerical staff support Established internal communication network Evaluation of coalition and coalition activities Funding and institutionalization Stakeholder maintenance/list Letter of support Leveraging resources Meeting minutes/schedules Membership maintenance/list Memorandum of agreement/understanding Policy development Products & impact SMART action plan development and implementation Visibility Written priorities/plans/strategies Written vision/mission statements Other (specify)

Budget Detail and Justification

Question	Response Options
Personnel
Budget type	Display only
Personnel*	Select from list
Position Title	Display only
Yearly salary*	Number
% of time	Display only
Number of months per year*	Number
Amount	Number
Justification*	Text
Allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Fringe benefit rate*	Percent
Fringe amount	Number
Fringe allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Travel
Budget type*	Select one: Base funding Supplemental Carryover (Specify year)
Trip title*	Text
Type*	Select one: Instate Out of state
Number of people*	Number
Number of trips*	Number
Dates of Travel	Enter date range
Per diem	Number
Mileage	Number
Ground transportation	Number
Airfare	Number
Lodging	Number
Car rental	Number
Other	Number
Amount	Number
Justification*	Text
Allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Equipment
Budget type*	Select one: Base funding Supplemental Carryover (Specify year)
Equipment title*	Text
Number of units*	Number
Cost per unit*	Number
Amount	Number
Justification*	Text
Allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Supplies
Budget type*	Select one: Base funding Supplemental Carryover (Specify year)
Supply title*	Text
Number of units*	Number
Cost per unit*	Number
Amount	Number
Justification*	Text
Allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Contractual
Budget type*	Select one: Base funding Supplemental Carryover (Specify year)
Organization name	Select from list
Primary role	Display only
Amount*	Number
Justification*	Text
Scope of work*	Text
Method of accountability*	Text
Period of performance*	Enter date range
Method of determination*	Text
Allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Travel costs included?*	Select one: Yes No
Type*	Select one: Instate Out of state
Number of people*	Number
Number of trips*	Number
Dates of Travel	Enter date range
Per diem	Number
Mileage	Number
Ground transportation	Number
Airfare	Number
Lodging	Number
Car rental	Number
Other	Number
Amount	Number
Other
Budget type*	Select one: Base funding Supplemental Carryover (Specify year)
Description*	Text
Amount*	Number
Justification*	Text
Allocation*	Enter each Federal requested amount or % State cash amount or % State in-kind amount or % Other cash amount or % Other in-kind amount or %
Indirect Charges
Budget type*	Select one: Base funding Supplemental Carryover (Specify year)
Indirect charge rate*	Percent
Indirect charge base* (Object class categories against which the indirect rate is applied.)	Select all that apply: Personnel Fringe benefits Travel Equipment Supplies Contractual Other
Comments*	Text
Amount	Display only
Allocation*	Enter federal requested amount or %

Systemic, Socio-political, and Policy Change Assessment

Question	Response Options
Assessment title*	Text
Date of assessment*	Date
Next expected assessment date*	Date
Level*	Select all that apply: State Region within state Local Other (specify)
Frequency of assessment*	Select one: Quarterly Semi-annually Annually Bi-annually Every ___ years
Describe process for conducting the assessment (methodology)*	Text
Summarize opportunities identified (findings)*	Text
Change as a result of the assessment (use of findings)*	Text
Stakeholders involved in the developing, conducting, analyzing or evaluating the assessment?	Text
Additional assessment information	Upload attachment
Date last revised	Date

Disease Burden, Priority Population, and Unmet Needs

Question	Response Options
Title*	Text
Status*	Select one: Draft Published
The following questions relate to DRAFT burden documents
Anticipated Publish Date	Date
Describe Progress to Date	Text
The following questions relate to PUBLISHED burden documents
Date Published*	Date
Date Last Revised	Date
Upload or Web Address	Upload attachment or enter Web URL
Dissemination*	Select all that apply: Academia/school Advisory/partner group Business/industry sector Coalition Federal health government agency General public Governor and staff Hospital/health care agency Legislator Local health government agency Media National organization and state affiliate Other federal government agency Other local government agency Other state government agency Priority population organization Private/public policy maker State health government agency Third party payers Other (specify)
Identify the target population(s) from the burden report*	Race Select all that apply: African American or Black American Indian or Alaska Native Asian Native Hawaiian or Other Pacific Islander White Not specified Ethnicity Select one: Hispanic or Latino Not Hispanic or Latino Not specified Gender Select all that apply: Female Male Geography Select all that apply: City County/parish Community Other (specify) Age: Select all that apply: 0-5 years 6-11 years 12-19 years 20-49 years 50-64 years 65 + Not specified Income: Select all that apply: Medicaid eligible 100% of poverty (poor) 200% of poverty (near poor) At or below 235% of poverty Not Specified
Additional target population comments*	Text
Identify the data sources used for the burden report	Select all that apply: ASTDD State Synopsis Basic Screening Surveillance (BSS) Behavioral Risk Factor Surveillance System (BRFSS) Centers for Medicare and Medicaid Services (CMS) Chronic Disease Indicators (CDI) Dental, Oral and Craniofacial Data Resource Center (DRC) Health Plan Employer Data and Information Set (HEDIS) Hospital Discharge Data My Water's Fluoride (MWF) National Health and Nutrition Examination Survey (NHANES) National Immunization Survey (NIS) National Oral Health Surveillance System (NOHSS) Pregnancy Risk Assessment Monitoring System (PRAMS) U.S. Bureau of Census Vital statistics Woman, Infants, and Children (WIC) Youth Risk Behavior Surveillance System (YRBSS) Youth Tobacco Survey (YTS)
Burden report includes indicators consistent with*	Select all that apply: National Oral Health System (NOHSS) Percentage of people who visited the dentist or dental clinic within the past year. Percentage of people who had their teeth cleaned in the past year. Percentage of people aged 65 years and older who have lost all natural permanent teeth. Percentage of people served by public water systems who receive fluoridated water. Percentage of 3rd grade students with caries experience, including treated and untreated tooth decay. Percentage of 3rd grade students with untreated tooth decay. Percentage of 3rd grade students with dental sealants on at least one permanent molar tooth. Cancer of the oral cavity and pharynx No data available ASTDD State Synopsis Population served by public water system Percentage of people on public water systems receiving fluoridated water. Number of dental hygiene schools Number of community-based low-income dental clinics Number of school-based health centers with an oral health component Number of tribal, state, or local agencies with service populations of 250,000 or more Number of agencies with a dental program Number of dental programs directed by a dental professional Number of directors with an advanced public health degree Number of dentists in the state No data available Water Fluoridation Reporting System (WFRS) Communities and populations receiving new or replacement fluoridation equipment. Percent of fluoridated water systems consistently maintaining optimal levels of fluoride as defined by No data available
Burden report includes description of*	Select all that apply: Oral health burden Oral health unmet needs Oral health disparities
Additional burden document information or publications	Enter text (100 words/500 characters) -AND/OR- Upload file

State Plan

Question	Response Option
Plan status	Select one: Draft Published
The following questions relate to DRAFT plans
Working Title*	Text
Anticipated Publish Date*	Date
The following questions relate to PUBLISHED plans
Published Title*	Text
Timeframe*	Date
Date Published*	Date
Date Last Revised*	Date
Attach Plan	Upload file
Dissemination of Plan*	Select all that apply: Academia/school Advisory/partner group Business/industry sector Coalition Federal health government agency General public Governor and staff Hospital/health care agency Legislator Local health government agency Media National organization and state affiliate Other federal government agency Other local government agency Other state government agency Priority population organization Private/public policy maker State health government agency Third party payer Other (specify)
Content Areas*	Select all that apply: Burden of disease Caries Evaluation strategies and recommendations for monitoring the outcomes and impacts of plan implementation Healthy People 2010 objectives Implementation strategies Infection control Leveraging of resources Oral cancer Oral health infrastructure Partnerships Periodontal diseases Plan maintenance Priority populations School-based or school-linked sealant programs Strategies to address oral health promotion across the lifespan Strategies to identify best practices Water fluoridation Other (specify)
Does the plan include specific, measurable and time phased objectives?*	Select one: Yes No

Surveillance Plan

Question	Response Option
Plan status	Select one: Draft Final
The following questions relate to DRAFT plans
Working Title*	Text
Anticipated Completion Date*	Date
The following questions relate to FINAL plans
Title*	Text
Time Frame*	Dates
Date Completed*	Date
Date Last Revised	Date
Has a logic model been developed for the plan?*	Select one: Yes No Currently being developed
Attach Plan* (Attach logic model, surveillance grid and narrative)	Upload file
Identify the data sources used for the surveillance plan*	Select all that apply: [Display list of data sources already entered]

Evaluation Plan

Question	Response Option
Evaluation Type*	Select one: Overall (required for evaluation plan and logic model) Leadership Oral disease burden, health disparities, and unmet needs Comprehensive state oral health plan Statewide oral health coalition (required for logic model) Oral disease surveillance system (required for logic model) Opportunities for systemic, socio-political and/or policy change Partnerships Limited community water fluoridation program management State program accomplishments, best practices, lessons learned, and use of evaluation results Water fluoridation program (logic model and evaluation plan required if funded) Limited school-based or school-linked dental sealant program (logic model and evaluation plan required if funded) Other (Specify)
Stage of Plan*	Select one: Not started Planning Implementation
The following questions relate to the NOT STARTED stage
Anticipated Planning Date*	Date
The following questions relate to the PLANNING or IMPLEMENTATION stage
Time Frame (Required if stage = implementation)	Dates
Logic Model (Required if stage = implementation)	Select one: Yes No
Stakeholders Involved (Required if stage = implementation)	Select one: Yes No
Evaluation Questions (Required if stage = implementation)	Enter text (200 words/1000 characters)
Data Sources Used (Required if stage = implementation)	Select all that apply: [list of data sources already entered]
Tools Used (Required if stage = implementation)	Select all that apply: State Plan Oral Health State Plan Index State Plan Index Coalitions Starting a Coalition Checklist Initial Coalition Survey Risk Factors for Collaborative Participation Worksheet Coalition Effectiveness Inventory (CEI) Partnership Self-assessment Member Satisfaction Survey Meeting Effectiveness Inventory Sealants Sealant Provider Survey Sealant Placement Survey School/Community Follow up survey Staff and Volunteer Satisfaction Survey Tracking Program Implementation Sealant Program Cost Analysis/ImprovePro Sealant Follow-up form Surveillance Surveillance System Evaluation Tool Other (specify)
Evaluation Design Plan (Required if stage = implementation)	Select one: Yes No
Use of Evaluation	Enter text (200 words/1000 characters)
Attachments	Upload file
Date last revised*	Date
Type*	Select all that apply: Evaluation Plan Reports Presentation Tools Other (Specify)

Work Plan Objectives

Question Asked		Response Option
Specific
Objective Title*	Text
Objective Status*	Select one: Proposed In progress Completed Deferred Cancelled
Related 5-Year Goal*	Select all that apply: Develop Oral health program leadership capacity. Describe the oral disease burden, health disparities, and unmet needs in the State. Develop or update a comprehensive State Oral Health Plan. Establish and sustain a diverse Statewide oral health coalition. Develop or enhance oral disease surveillance system. Identify prevention opportunities for systemic, socio-political and/or policy change to improve oral health. Develop and coordinate partnerships to increase State-level and community capacity to address specific oral disease prevention interventions. Coordinate and implement limited community water fluoridation program management. Evaluate, document, and share State program accomplishments, best practices, lessons learned, and use of evaluation results. Develop and Implement a water fluoridation program. Develop, coordinate and implement limited school-based or school-linked dental sealant programs.
Measurable & Achievable
Measure of success*	Select all that apply (based upon selected 5-Year Goal): Develop Oral health program leadership capacity. existence of full-time dental director existence of .25 time epidemiologic support access to at least .50 time of a water fluoridation engineer/specialist or coordinator access to .50 to one time dental sealant coordinator access to .25 time capacity for health education, health communication access to .25 time support staff Describe the oral disease burden, health disparities, and unmet needs in the State. disease burden document is publicly available. disease burden document includes oral health status with indicators consistent with the National Oral Health System (NOHSS), the Water Fluoridation Reporting System (WFRS), and the ASTDD State Synopsis. Develop or update a comprehensive State Oral Health Plan. plan addresses oral health infrastructure including current plan addresses evaluation strategies and recommendations for monitoring the outcomes and impacts of plan implementation Establish and sustain a diverse Statewide oral health coalition. progress towards coalition sustainability Develop or enhance oral disease surveillance system. establishment of a plan for how data collection, analysis, and dissemination will support program activity, including a surveillance plan logic model consistent with the CDC Surveillance Logic model Identify prevention opportunities for systemic, socio-political and/or policy change to improve oral health. periodic assessments to demonstrate identification of socio-political and policy changes.
Baseline*	Text – OR – Select ‘Baseline unknown’
Target*	Text
Evidence for measuring target*	Text
If baseline is unknown, explain how it will be determined.	Text
Relevant
Describe how this objective will establish, strengthen or expand your program’s capacity to plan, implement, and evaluate population-based oral disease prevention and health promotion programs, targeting populations and oral disease burden.*	Text
Time-bound
Start Date*		Date
End Date*		Date

Work Plan Objective Progress and Results

Question Asked	Response Option
Progress
Date progress occurred*	Date
Describe progress*	Text
Has the objective’s target been met?*	Select one: Yes No Currently ongoing
Results if Objective Target is Met
Enter date met*	Date
Measure achieved*	Text
Facilitating factors for success*	Text
Describe barriers encountered while achieving the objective's target measure	Text
Describe any unanticipated outcomes or collateral effects	Text
Results if Objective Target is Not Met or Currently Ongoing
Current measure (if applicable)	Text
Describe barriers to achieving the objective's target measure	Text
Describe plans to overcome barriers	Text
Describe any unanticipated outcomes or collateral effects	Text
Objective Revisions
Does the objective status, start date, end date or target measure need to be revised?*	Select one: Yes No
Revise objective status	Select one: Proposed In Progress Completed Deferred Cancelled
Explain reason for revising status	Text Required only if status is revised to ‘Deferred’ or ‘Cancelled’
Revise objective start date	Date
Explain reason for revising start date	Text Required only if start date is delayed
Revise objective end date	Date
Explain reason for revising end date	Text Required only if end date is extended
Revise objective target measure	Text
Explain reason for revising target measure	Text Required for all target measure changes

Work Plan Activities

Question Asked	Response Option
Activity Title*	Text
Activity Description*	Text
Status*	Select one: Proposed In progress Completed Deferred Cancelled
Lead staff assigned to this activity*	Select one: [list of existing staff]
Other staff assigned to this activity	Select all that apply: [list of existing staff]
Contractors assigned to this activity	Select all that apply: [list of existing contractors]
Partners assigned to this activity	Select all that apply: [list of existing partners]
Describe partner involvement	Text
Start Date*	Date
End Date*	Date

Work Plan Activity Progress

Question Asked	Response Option
Progress
Date progress occurred*	Date
Describe progress*	Text
Activity Revisions
Does the activity status, start date or end date need to be revised?*	Select one: Yes No
Revise activity status	Select one: Proposed In Progress Completed Deferred Cancelled
Explain reason for revising status	Text Required only if status is revised to ‘Deferred’ or ‘Cancelled’
Revise activity start date	Date
Explain reason for revising start date	Text Required only if start date is delayed
Revise activity end date	Date
Explain reason for revising end date	Text Required only if end date is extended

Work Plan Products

Question Asked	Response Option
Products
Title*	Text
Description*	Text
Website Address	Text
Attachments	Upload File
Date file last revised	Date
Can this document be shared?	Select one: Yes No

File Type	application/msword
File Title	OFFICE OF MANAGEMENT AND BUDGET (OMB)
Author	Karen
Last Modified By	gzk8
File Modified	2007-01-31
File Created	2007-01-31