Supporting Statement A

NCHS received Office of Management and Budget (OMB) approval for NCHS Customer Surveys Generic Clearance (OMB No. 0920-0729, Exp. Date 09/30/2020) to conduct several activities to make progress toward meeting its Program Assessment Rating Tool (PART) objective to "measure satisfaction of key data users and policymakers." The surveys may be conducted among the following four categories: (1) "reimbursable collaborators," which includes agencies (mostly governmental) that fund components of NCHS surveys; (2) conference attendees, through both focus groups during the conferences and an email survey of attendees following the meetings; (3) federal "power users" of NCHS data, which include agencies that use NCHS data for analysis, policy, and other purposes; and (4) users of the NCHS website, both through "usability testing" at the NCHS Conferences and through Internet-based surveys of users of the NCHS website.

This proposed study falls under category 4: users of the NCHS website. Under this approval, OMB has agreed to expedite a generic information collection (GenIC) review of proposals for data collections, and OMB will generally review such requests within 10 business days.

United States (U.S.) vital statistics are collected and published by a decentralized, cooperative system with each individual state and independent registration area responsible for the registration of births, deaths, and fetal deaths. The National Center for Health Statistics (NCHS) is required by federal law (the Public Health Service Act [42 U.S.C. 242K]) to make an annual collection of data from the records of births and deaths in the registration areas and to encourage comparability of data across reporting areas. The collection of the data from these records is the basis of the National Vital Statistics System, the only source of uniform, continuous data related to vital events occurring in the U.S.

Uniformity of vital statistics data is achieved by the issuance of national standards. These standards are developed in collaboration with state vital statistics officials, the responsible national agency (NCHS), and other vital statistics stakeholders. The standard U.S. birth certificate is the principal means of achieving uniformity of vital statistics data for the nation. The states use the national standards to develop their own state-specific birth certificates and state vital statistics laws and regulations.

Information from a U.S. birth certificate is used extensively to track national trends in births by demographic characteristics, health care utilization, obstetric procedures, and maternal and infant health. In more recent years, these data have also been more widely used in obstetric and perinatal research. A chief advantage of birth certificate data is that information is collected for essentially every birth occurring in the country each year, allowing for analysis of subpopulations and rare conditions and events. These analyses are important to better understand maternal and infant health.^¹ The quality of birth certificate medical and health data, however, is of long-standing concern. Studies evaluating 1989 birth certificate revision-based data have consistently shown that the demographic and selected medical and health items (e.g., method of delivery, birthweight, plurality) are collected with a high degree of completeness and accuracy, but many of the health and medical items are underreported.

Accordingly, a key objective of the latest birth certificate revision (i.e., the 2003 U.S. Standard Certificate of Live Birth) was to standardize the data collection process and improve data quality. However, recent validity studies indicate wide variation in the quality of the medical and health information collected based on the 2003 revision, and several underperforming questions have been dropped. In addition, data from hospitals and state vital statistics offices have also identified lack of familiarity with standardized resources available for data collection, limited resources for training, and lack of awareness of the uses and importance of these data among hospital employees. As such, comprehensive, standardized online training for hospitals and birthing facilities across the U.S. was developed by NCHS, the National Association Public Health Statistics and Information Systems (NAPHSIS), and state vital statistics representatives, as a cost-effective approach for overcoming these barriers and improving data quality.

The vital statistics of the U.S. are critical to understanding the health of mothers and babies. Because data are collected for almost every U.S. birth via birth certificates and fetal death records, they provide the unique opportunity to examine subpopulations, as well as rare conditions. However, in order for the data to provide accurate information and to be of high quality, standardized methods for collection and reporting are necessary. To ensure a standardized process for the collection and/or reporting of data, individuals involved in the collection and/or reporting of birth certificate data must be trained in a standardized manner.

The goal of this GenIC is to (1) assess participants’ changes in knowledge, awareness, thoughts, and perceptions about the importance of birth certificate data and changes in understanding of best practices for reporting these data; and (2) assess participants’ impression of the eLearning course in terms of acceptability, convenience, and accessibility.

A.2 Purpose and Use of the Information Collection

In order to make improvements in data quality, NCHS has created an online eLearning course to train those responsible for the collection and/or reporting of birth certificate information on when to complete a birth certificate or report of fetal death, the benefits of reporting accurate medical and health information, the standard resources available for completing the birth certificates and records of fetal death, the approaches for improving birth certificate data quality, and how to apply criteria from the “Guide to Completing the Facility Worksheets for the Certificate of Live Birth and Report of Fetal Death” to increase the accuracy of reporting. As such, the purpose of this project is to thoroughly evaluate the effectiveness of a new, nationally standardized online eLearning course to increase the quality of birth data. Quality is defined as completeness and accuracy across the reported medical and health data collected on a birth certificate and report of fetal death.

The goal of the eLearning course, “Applying Best Practices for Reporting Medical and Health Information on Birth Certificates,” is to improve the quality of medical and health data collected on the birth certificate and report of fetal death. Improvements in the quality of information collected will improve surveillance and research on maternal and infant health in the U.S. The eLearning course was designed to underscore the value of the data for improving maternal and infant health in the U.S., as well as the importance of accurate data collection and standard resources and best practices to assist in this process.

The information will be collected from individuals representing four types of hospital staff: (1) birth registers (BR); (2) birth register managers (BRM); (3) labor and delivery nurses (including nurse/midwives) (L&D nurses); and (4) obstetricians and gynecologists (OB/GYN). Administrators from recruited hospitals will be sent introductory emails (Attachment A1a) requesting that they provide staffing and email information (Attachment A1b) that will be used to identify potential participants. Once these potential participants have been identified, the nine hospitals will send the contact information to the contractor via FedEx. The contractor will then contact potential participants for inclusion in the study. Data will be collected online through web-based surveys for all respondent types.

Up to 645 staff involved in the collection of birth certificate data will be invited to participate in completing the Online Baseline Survey until up to 359 responses (i.e., 37 BR, 12 BRM, 216 L&D nurses, and 94 OB/GYNs) have been obtained. All staff who completed the Online Baseline Survey (Attachment B2) and the eLearning course will be invited to participate in the Online Follow-Up Survey (Attachment D2) within one week of completing the eLearning course. The details on each data collection point are provided in Supporting Statement B.

The results from this research study will provide essential information to guide modifications to future versions of the eLearning training course, including improving the effectiveness of the modules to increase knowledge of best practices for birth reporting. Results will also help identify obstacles and barriers that may prevent hospital staff (BR, BM, L&D nurses, and OB/GYNs) from accurately documenting relevant birth information in the medical record and/or from directly completing information required for birth certificate reporting. Information obtained on any barriers may be used to develop outreach to hospital staff and administrators to overcome these barriers.

The outcomes of interest for this study are both process and short-term outcomes. The process outcomes include such items as where the online eLearning course was taken, how long it took to complete the eLearning course, and perceptions about the eLearning course. Perceptions will be assessed by asking participants about the ease of navigation, format of the eLearning course, who should complete the eLearning course, and whether any key concepts were missing. Additionally, participants’ response to the online eLearning course, including acceptability, convenience, and accessibility, will be assessed. These process outcomes will be assessed using the Online Follow-Up Survey (Attachment D2).

The short-term outcomes will include changes to the knowledge, awareness, thoughts, and perceptions about the importance of birth certificate data quality, and intent for data quality improvement. These will be assessed by comparing results from the Online Baseline Survey (Attachment B2) and Online Follow-Up Survey (Attachment D2).

Attachment E provides an overview of the types of outcomes that will be assessed by each of the instruments.

A.3 Use of Improved Information Technology and Burden Reduction

NCHS is committed to complying with the E-Government Act of 2002 to promote the use of technology. The Online Baseline Survey (Attachment B2) and Online Follow-Up Survey (Attachment D2) will be web-based and will be self-administered using electronic methods. Each of the surveys are programmed so that the respondents are able to save their responses and return to the survey from any computer. This feature reduces the burden on respondents by allowing the flexibility to complete at their convenience. Additionally, respondents have the option of taking the surveys on a smart phone or tablet. This places less of a burden requirement on respondents by allowing them to take the surveys at their convenience at any location with access to the internet or a cell phone data tower.

A.4 Efforts to Identify Duplication and Use of Similar Information

There are no current studies examining the effectiveness of the eLearning course to improve the quality of medical and health data collected on the birth certificate.

A.5 Impact on Small Businesses or Other Small Entities

Respondents include BR, BRM (e.g., those managing the BR and/or L&D nurses), L&D nurses, and OB/GYNs in nine selected hospitals in Massachusetts. Although we do not believe that any of these hospitals are small businesses, if any of them are, they should not be adversely affected by the surveys. In order to minimize burden on all participants, the number of items contained on the surveys has purposely been held to the minimum required to collect the necessary information.

A.6 Consequences of Collecting the Information Less Frequently

Without this effort, which has been planned to assess the newly created eLearning course to improve birth certificate data quality with the minimum possible burden, NCHS will not have the information necessary to improve the eLearning course to maximize its effectiveness and improve data collected for birth certificates.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances. This collection of information will be conducted in a manner consistent with the guidelines in the Code of Federal Regulations, 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.a Federal Register notice and comments

The 60-day notice was published in the Federal Register, Vol. 78, No. 44, March 8, 2017, pages 12963-12965. Only one public comment was received. A standard response was provided.

A.8.b Consultations Outside the Agency

Only internal consultation will be involved in this project.

A.9 Explanation of Any Payment or Gift to Respondents

No payment or gift will be offered to the respondents.

A.10 Protection of the Privacy and Confidentiality of Information Provided by Respondents

While names and email addresses will be used for the purposes of contacting and tracking participants, the actual surveys will not collect personally identifiable information (PII) about the participants (BR, BRM, L&D nurses, and OB/GYNs). Instead, the surveys will collect information on the participants’ professional insights about the eLearning course; the current quality of medical and health data collected on the birth certificate; and any changes that occurred to participants’ knowledge, awareness, and intentions about improving birth certificate data as a result of the eLearning course. In addition, the Online Baseline Survey (Attachment B2) will collect information on how long participants have been working at their hospitals, as well as their role at the hospital and in the collection of birth certificate information. Participants will be sent a unique link to each of the surveys (Attachments B1a and D1a) and a unique, randomly generated password (Attachments B1b and D1b) that will be required to access the survey. Links and passwords will be sent in separate emails. The link, generated by Qualtrics, uses a long string of letters and numbers generated for each data record. The contractor will create randomized passwords using random number and random letter generators in Excel.

Confidentiality will be provided to respondents as assured by Section 308(d) of the Public Health Service Act (42 USC 242m), as follows: “No information, if an establishment or person supplying the information or described in it is identifiable, obtained in the course of activities undertaken or supported under section 304, 306, or 307 may be used for any purpose other than the purpose for which it was supplied unless such establishment or person has consented (as determined under regulations of the Secretary) to its use for such other purpose and in the case of information obtained in the course of health statistical or epidemiological activities under section 304 or 306, such information may not be published or released in other form if the particular establishment or person supplying the information or described in it is identifiable unless such establishment or person has consented (as determined under regulations of the Secretary) to its publication or release in other form.” A Confidentiality Frequently Asked Questions (FAQ) document (Attachment A2) will also be provided and attached to all emails sent to hospitals and potential participants.

Also NCHS makes the following Confidentiality Pledge: “We take your privacy very seriously. All information that relates to or describes identifiable characteristics of individuals, a practice or an establishment will be used only for statistical purposes. NCHS staff, contractors and agents will not disclose or release responses in identifiable form without the consent of the individual or establishment in accordance with section 308(d) of the Public Health Service Act (42 U.S.C. 242m(d)) and the Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA, Title 5 of Public Law 107-347). In accordance with CIPSEA every NCHS employee, contractor and agent have taken an oath and is subject to a jail term of up to five years, a fine of up to $250,0000, or both if he or she willfully discloses ANY identifiable information about you.”

Data will be treated in a confidential manner. Upon receiving the data from the hospitals via FedEx, the contractor will create an Excel file containing the names and email addresses of hospital staff who are involved in the collection and/or reporting of birth certificate data. The Excel file will be stored on a quarantined virtual server separate from survey data. The Excel file will be encrypted using Bitlocker. Only authorized personnel (i.e., designated agents approved to work on the project) will have access to the Excel file stored on the quarantined virtual server.

Though the data collectors (the contractors) have many years of experience collecting and handling both qualitative and quantitative data, a training will take place prior to the data collection to remind staff of the purpose of the study and to discuss the importance of data confidentiality/privacy. Additionally, all data collectors are required to sign a data collector confidentiality agreement (Attachment A3).

Surveys completed during business hours will be immediately downloaded and deleted from the Qualtrics (described below) server; surveys completed outside of business hours will be downloaded and deleted at the start of business hours and twice daily on weekends. Downloaded survey data will be stored in a network folder on a quarantined virtual server with access limited by a security group of authorized domain users. Record-level data will not be shared with Massachusetts DPH/VR or the hospitals. Massachusetts DPH/VR will receive summarized reports (created by NCHS) in which information about staff category, job experience, and barriers faced will not be shown by hospital. Where respondents are fewer than five in any category, data will be suppressed. No personally identifiable information (PII) will be included on any of the surveys. PII (names and email addresses only) will be encrypted and stored separately from survey responses and will only be seen by NCHS and the contractor for purposes of contacting and tracking the participants for the study. The document with the participants’ contact information will be stored separately from all data. Data will only be accessible to the minimum number of staff necessary to perform the work. Data will not be shared or made available to any unauthorized personnel or other third party. Any contact information or other data from paper records will be transferred to electronic files for use during projects. Any paper records will be stored in a locked file drawer. After all reports have been completed, all data files will be encrypted and transferred to NCHS. The Contractor will archive all data files for 3 years in a secure file server.

Primary data collection of quantitative data consists of the Online Baseline Survey (Attachment B2) and Online Follow-Up Survey (Attachment D2) composed of up to 359 respondents using Qualtrics, which stores responses in an internal database. For quality control and assurance, the data will be directly exported into Statistical Package for Social Sciences (SPSS) (version 24) as an SPSS file or will be exported as a CSV and imported into Stata (version 14) for analysis. These automated processes will ensure data accuracy. Any differences found during this process will be discussed by the contractor and reconciled.

NCHS and the contractor will protect data files from unauthorized access that could lead to theft and disclosure of personally identifiable information (PII) by requiring an authorized username and password to access the data files. In order to protect confidentiality, the respondent’s name and email address will be encrypted and stored separately from the survey responses and will not be seen by anyone other than the contractor and NCHS.

All data cleaning and processing is primarily done by the contractor. The final data products will be accessed by NCHS through an encrypted connection with authorized usernames and passwords. Survey data will be encrypted using Bitlocker and delivered back to NCHS via an FIPS 140-2-compliant method. When confidential materials are moved between locations, records are maintained to ensure there is no loss in transit. After all reports have been completed, all data files will be encrypted and securely transferred to NCHS.

NCHS and subcontractor staff routinely employ technical, physical, and administrative measures to secure information and safeguard privacy and confidentiality. The measures include the following:

When confidential materials are moved between locations, records are maintained to ensure that there is no loss in transit.
Hard copies of confidential information are stored in secure areas when not in use.
Access to the data processing and storage areas is controlled, with only authorized personnel allowed in secure locations.
Individual data files are protected by passwords and other techniques, which prohibit access by non-approved project staff.
Building security forces are on duty 24 hours, seven days per week at all sites.
Access to nonpublic data is restricted to those who must have such access.
Data collectors, supervisors, and staff receive thorough training on legal and ethical obligations.
After all reports have been completed, all data files will be encrypted and transferred to NCHS. The Contractor will archive all data files for three years in a secure file server.

A.11 Institutional Review Board (IRB) and Justification for Sensitive Questions

This project has been reviewed by the NCHS Ethics Review Board (ERB) and was approved on 09/11/2018. This GenIC does not contain questions of a sensitive nature.

A.12 Estimates of Annualized Burden Hours and Costs

The introductory/invitation email that each of the recruited hospitals will receive from the Massachusetts DPH/VR (Attachment A1a) will require administrators to spend approximately 20 minutes to provide the staffing and email information (Attachment A1b) needed to identify potential participants.

Online Baseline Surveys (Attachment B2) will be conducted with up to 359 BR, BRM, L&D nurses, and OB/GYNs, and will take, on average, 30 minutes to complete.

The eLearning course is designed to be completed within 1 hour (see: http://www.cdc.gov/nchs/training/BirthCertificateElearning/); however, the course is not part of this study, and thus does not contribute to the overall burden.

The Online Follow-Up Survey (Attachment D2) will be conducted with up to 359 BR, BRM, L&D nurses, and OB/GYNs, and will take, on average, 30 minutes to complete.

The estimated annualized burden for this information collection is estimated to be 363 hours (Table A1).

Table A1. Estimated Annualized Burden Hours

Type of Respondent	Form	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Hospital Administrators	Staff Listing Form (Attachment A1b)	9	1	20/60	3
Birth Registrars, Managers of Birth Registrars, Labor and Delivery Nurses, Obstetricians and Gynecologists	Online Baseline Survey (Attachment B2)	359	1	30/60	180
Birth Registrars, Managers of Birth Registrars, Labor and Delivery Nurses, Obstetricians and Gynecologists	Online Follow-Up Survey (Attachment D2)	359	1	30/60	180
Total					363

The total cost to participants is estimated to be $19,492.97 (Table A2). The mean hourly wage estimates for completing the surveys mentioned in the burden hours table are based on information from the Bureau of Labor Statistics website (http://www.bls.gov). Specifically, the “May 2017 National Occupational Employment and Wage Estimates” was used. For Hospital Administrator, 11-9111 “Medical and Health Services Managers” was used; for BR, 29–2071 “Medical Records and Health Information Technicians” was used; for BRM, 29–2090 “Miscellaneous Health Technologists and Technicians” was used; for L&D nurses, 29–1141 “Registered Nurses” was used; and for OB/GYNs, we used 29–1064 “Obstetricians and Gynecologists.”

Table A2. Estimated Annualized Burden Costs

Type of Respondent	Form Name	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Hospital Administrators	Staff Listing Form (Attachment A1b)	3	$53.69	$161.07
Birth Registrars	Online Baseline Survey and Online Follow-up Survey	38	$20.59	$782.42
Managers of Birth Registrars	Online Baseline Survey and Online Follow-up Survey	12	$23.36	$280.32
Labor and Delivery Nurses	Online Baseline Survey and Online Follow-up Survey	216	$35.36	$7,637.76
Obstetricians and Gynecologists	Online Baseline Survey and Online Follow-up Survey	94	$113.10	$10,631.40
Total				$19,492.97

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Recordkeepers

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.

A.14 Annualized Cost to the Federal Government

Direct costs for this study include the contractor’s fee charged by Massachusetts DPH/VR and the contractor. This cost for Massachusetts DPH/VR ($25,000) will cover time spent contacting the hospitals to determine interest in participating, initiating contact with the hospitals selected to participate, and consultation on the design and timing of the study. It is expected that collecting and analyzing the data from both the baseline and follow-up participant surveys will cost NCHS approximately $199,614 in contracting expenses (including labor and other direct costs), for a total of approximately $224,614.28 (Table A3).

Table A3. Annualized Cost to the Federal Government

Staff (FTE)	Annualized Cost
Massachusetts DPH/VR - Time spent contacting the hospitals to determine interest in participating, initiating contact with the hospitals selected to participate, and consultation on the design and timing of the study	$25,000.00
Contracting Expenses - Instrument development, OMB and ERB package preparation, data collection, coding and entry, quality control, data analysis and report preparation	$199,614.28
Total	$224,614.28

A.15 Explanation for Program Changes or Adjustments

This is a GenIC. There are no program changes or adjustments.

A.16 Plans for Tabulation and Publication and Project Time Schedule

The data collected will be analyzed to inform NCHS of any potential changes to be made to the eLearning course. Instruments are designed to yield answers that include a Likert-type scale, Yes/No, multiple choice, and short answers, where necessary. The data will be analyzed based on three components: (1) descriptive characteristics of the sample, (2) effectiveness of the training in improving knowledge of birth reporting, and (3) participants’ assessment/reaction to the eLearning course.

Descriptive characteristics of the sample will be tabulated, and frequencies calculated for responses to questions about job title, years of experience, and roles within the data collection process.

The effectiveness of the training in improving knowledge of reporting medical and health information on birth certificates will be assessed by changes in overall baseline and follow-up scores. It will be evaluated using knowledge-check questions and scored based on the percentage of correct responses. Comparisons between baseline and follow-up scores will be assessed using a paired t-test. Scores will be examined by job title and years of experience. Post-test scores may also be regressed against pretest scores and adjusted for these additional characteristics using linear regression.

Reactions to the course will be tabulated. Initially, Likert scale frequencies will be examined for each response but may be grouped into Strongly Agree/Agree and Strongly Disagree/Disagree for presenting final results.

Some questions in the survey yield qualitative responses. Responses from these qualitative questions will be imported into NVivo qualitative software analysis package and categorized by question. Responses from each individual question will be analyzed for themes by at least two trained team members to ensure comprehensiveness and accuracy. To improve inter-rater reliability, a codebook will be developed collaboratively among team members after initial review of answer responses.

Massachusetts DPH/VR will receive summarized reports (created by NCHS). Massachusetts may share this summary report with the participating hospitals if they wish. No hospital-specific information will be shared outside NCHS. General results may be made available on the NCHS website. Disseminating results to the public includes summarized (deidentified) results that will be made available to the Birth Data Quality Workgroup, the National Association for Public Health Statistics and Information Systems (NAPHSIS), and NCHS to determine next steps for the eLearning course. NCHS may publish summarized study results in a report on any changes of birth certificate data quality and NCHS will present study results at an annual NAPHSIS meeting in the summer of 2019.

The study timeline is below in Table A4.

Table A4. Study Timeline

Fall 2018:	Massachusetts DPH/VR to send introductory letter to hospitals asking them to securely send lists to the contractor of names and contact information of hospital staff involved in the birth certificate data collection process.
Fall 2018:	Invite hospital staff to participate in the study; collect Online Baseline Survey responses until the desired sample size has been reached.
Fall 2018:	Administer Online Follow-Up Surveys.
Fall 2018/Winter 2019:	Data analysis; develop report of survey and interview analysis.

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

An exemption to display the OMB expiration date is not being requested.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

This study does not require any exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

1 Martin, J. A., Wilson, E. C., Osterman, M. J. K., Saadi, E. W., Sutton, S. R., & Hamilton, B. E. (2013). Assessing the quality of medical and health data from the 2003 birth certificate revision: Results from two states. Retrieved from http://www.cdc.gov/nchs/data/nvsr/nvsr62/nvsr62_02.pdf

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Molly Matthews-Ewald, PhD, MS
File Modified	0000-00-00
File Created	2021-01-20