Ssa

This information collection is being conducted using OMB No. 0920-0879 “Information Collections to Advance State, Tribal, Local and Territorial Governmental Agency System Performance, Capacity, and Program Delivery” nicknamed the “CSTLTS Generic.” The respondent universe for this information collection aligns with that of the CSTLTS Generic. Data will be collected from 57 respondents across 57 state, local, and territorial health departments/jurisdictions (see Attachment A). Respondents acting in their official capacities include public health laboratory directors who provide oversight for NGS testing conducted in their laboratories.

This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241) (1). This information collection falls under the essential public health service(s) of

1. Monitoring health status to identify community health problems

2. Diagnosing and investigating health problems and health hazards in the community

3. Informing, educating, and empowering people about health issues

4. Mobilizing community partnerships to identify and solve health problems

5. Development of policies and plans that support individual and community health efforts

6. Enforcement of laws and regulations that protect health and ensure safety

7. Linking people to needed personal health services and assure the provision of health care

when otherwise unavailable

8. Assuring a competent public health and personal health care workforce

9. Evaluating effectiveness, accessibility, and quality of personal and population-based

health services

10. Research for new insights and innovative solutions to health problems¹

While being broadly implemented across CDC programs as well as state, local, and territorial public health laboratories (PHLs), there are many challenges associated with implementing Next Generation Sequencing (NGS) based tests in laboratories. These challenges range from technical to ethical, with issues that include how to ensure test results are reproducible, how is data analyzed and who is qualified to perform the analysis, when should clinicians order NGS-based tests, and how are the patient and family counselled. To address some of the challenges in implementing testing, several federal agencies have answered the call for regulation of NGS-based tests. For example, the Centers for Medicare and Medicaid (CMS) oversees clinical laboratories that develop and offer genetic tests through the Clinical Laboratory Improvement Amendments (CLIA), and the Food and Drug Administration published guidance documents for certain NGS-based tests (2-4). In addition, several non-federal organizations have also released guidance documents detailing best practices and standard operating procedures for implementing NGS tests in clinical laboratories, this includes documents codeveloped by the College of American Pathologists (CAP) and the Association of Molecular Pathologists (AMP) (5), a technical guide developed by the World Health Organization on detecting of drug resistance using NGS (6), as well as guidance by the American Medical Informatics Association on the development and validation of bioinformatic pipelines (7).

While many of these documents provide guidance for specific conditions and/or assays, few address the entire testing process i.e., from library synthesis to sequencing to bioinformatic analysis. This lack of in-depth guidance was noted by Santani et al., on behalf of CAP and AMP, as contributing to variability in how laboratories implement existing regulatory standards and was listed as a major factor in the development of step-by-step guidance for variant detection in heritable diseases. The importance of comprehensive documentation on the NGS total testing process is a major concern as improperly developed, executed, and/or validated processes and procedure could result in reporting of inaccurate results, which would have adverse effects on patient-health outcomes. Thus, even with these existing regulations and guidance documents, there has been a call for additional guidance documents and tools that ensure quality and accuracy for the entire NGS testing process and are broadly applicable across all specialties.

To be of value to public health and disease surveillance, laboratory operations need to be reliable, tests need to be as accurate as possible, and test results must be promptly delivered. Failures in any of these could have consequences for patient and our population health. Quality management systems (QMSs) have been described by the International Organization for Standardization (ISO) and the Clinical Laboratory Standards Institute (CLSI) as “coordinated activities to direct and control an organization with regard to quality” (8). A QMS investigates the entire laboratory system from organization structure to facilities to assays, and analyzes all processes and procedures, including testing processes, to ensure quality. The use of QMS specific to NGS that establish and validate workflows has been demonstrated to overcome low quality samples and allowed the identification of somatic alleles important to guiding treatment decisions (9). Furthermore, many accreditation programs now encourage clinical laboratories to develop and follow QMS for their NGS-based tests (10).

The NGS Quality Initiative is an ongoing, collaborative effort between CDC’s Deputy Director for Infectious Diseases (DDID), the Center for Surveillance, Epidemiology, and Laboratory Services (CSELS) Division of Laboratory Services (DLS) and APHL to address the growing need in state, local, and territorial PHLs for QMSs that assure foundational quality in development and implementation of NGS-based tests (regardless of specialty or testing platform) by providing ready-to-implement guidance documents, standard operating procedures (SOPs), and forms that can be utilized by all laboratories. These documents and forms will be based on the CLSI’s Quality System Essentials (QSE), a framework for developing a QMS by identifying 12 categories that are essential for ensuring quality (11). The data collected in this generic information collection (GenIC) will aid in the development of the initiative’s products and identify which QSE PHLs need support in. The initiative’s goals align with the essential public health service of assuring a competent public health and personal health care workforce.

The purpose of this GenIC is to assess the current, NGS-specific QMS landscape at the state, local, and territorial level. Specifically, it will assess PHLs use of- and barriers in implementing-QMS and identify laboratory resources allocated for NGS. Findings will inform the development of tools and guidance documents that enable PHLs to ensure high quality sequencing data, identify which QSEs are most relevant to NGS quality, and identify the level of support required to implement those QSEs.

Overview of the Information Collection System

Data will be collected from 57 respondents via an online survey (see Attachment B – Online Survey Instrument Web Version and Attachment C – Online Survey Instrument Word Version). The instrument will be used to gather information from laboratory directors regarding the current, NGS-specific QMS landscape at the state, local, and territorial level, PHLs use of- and barriers in implementing-QMS, and will identify laboratory resources allocated for NGS.

The information collection instrument was pilot tested by two public health professionals. Feedback from this group was used to refine questions as needed, ensure accurate programming and skip patterns and establish the estimated time required to complete the information collection instrument.

Items of Information to be Collected

The data collection instrument consists of 45 main questions of various types, including dichotomous (yes/no), multiple response, ordinal scale, interval scale, and open-ended. Efforts were made to limit the number of questions requiring narrative responses whenever possible. The instrument will collect data on the following:

Section 1: Laboratory demographics – questions I-III
Section 2: Laboratory usage of QMS (by QSE) – questions 1-38
Section 3: Survey quality – questions 39-42

Purpose and Use of the Information Collection

The purpose of this GenIC is to assess the current, NGS-specific QMS landscape at the state, local, and territorial level. Specifically, it will assess PHLs use of- and barriers in implementing-QMS and identify laboratory resources allocated for NGS. Findings will inform the development of tools and guidance documents that enable PHLs to ensure high quality sequencing data, identify which QSEs are most relevant to NGS quality in their laboratories, and identify the level of support required to implement those QSEs.

Use of Improved Information Technology and Burden Reduction

Data will be collected via online survey. This method was chosen to reduce the overall burden on respondents by allowing respondents to complete the survey and submit their responses at a time and place of their choosing. The data collection instrument was designed to collect the minimum information necessary for the purposes of this project (i.e., limited to 45 questions).

Efforts to Identify Duplication and Use of Similar Information

Many existing NGS technical and guidance documents were developed for specific conditions and/or assays, few address the entire testing process. This lack of guidance for the NGS total testing process was noted by Santani et al., as contributing to variability in how laboratories implement existing regulatory standards and was listed as a major factor in the development of step-by-step guidance for variant detection in heritable diseases. The importance of comprehensive documentation on the NGS total testing process is a major concern as improperly developed, executed, and/or validated processes and procedure could result in reporting of inaccurate results. Thus, even with these existing regulations and guidance documents, there has been a call for additional guidance documents and tools that ensure quality and accuracy for the entire NGS testing process. APHL conducted a survey in 2018 that focused on NGS, but its primary purpose was technical information gathering and not sequencing quality or QMS. Members of the NGS Quality Initiative reviewed prior surveys and confirmed that they were non-duplicative. Additionally, an environmental scan, literature review, and search of OMB approved information collections available from the OMB/OIRA website (www.reginfo.gov) did not identify duplicative work.

Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

Consequences of Collecting the Information Less Frequently

This request is for a one time data collection. There are no legal obstacles to reduce the burden. If no data are collected, CDC will be unable to:

Understand the full landscape of use of NGS technologies, the existing QMS practices and resources, and the barriers to QMS implementation.
Develop, in collaboration with key partners, timely resources and evidence-based approaches that help state, local, and territorial PHLs improve their NGS-based tests.

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this data collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

This data collection is being conducted using the Generic Information Collection mechanism of the CSTLTS Generic Information Collection Service (CSTLTS Generic) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on April 27, 2017, Vol. 82, No. 80, pp 19371-19373. One non-substantive comment was received. CDC sent forward the standard CDC response.

CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.

Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

Protection of the Privacy and Confidentiality of Information Provided by Respondents

The Privacy Act does not apply to this data collection. STLT governmental staff and / or delegates will be speaking from their official roles.

This Gen IC does not involve the collection of personally identifiable information. Respondent’s business email address will be collected for the sole purpose of clarifying responses if needed (e.g., clarify which and why datasets were selected for process validation and/or clarify partial responses in open-ended questions). Responses to this question will be exclusively retained by APHL, will not be shared with CDC, and stored on secure, password protected servers that are accessible only to the study team. All other data collected will be stripped of any identifiers by APHL prior to sharing with CDC.

Institutional Review Board (IRB) and Justification for Sensitive Questions

No information will be collected that are of personal or sensitive nature. This data collection is not research involving human subjects.

Estimates of Annualized Burden Hours and Costs

The estimate for burden hours is based on a pilot test of the data collection instrument by two public health professionals. In the pilot test, the average time to complete the instrument including time for reviewing instructions, gathering needed information and completing the instrument, was approximately 39 minutes (range: 38 – 40). For the purposes of estimating burden hours, the upper limit of this range (i.e., 40 minutes) is used.

Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) Bureau of Labor Statistics for occupational employment for medical and diagnostic laboratory directors and managers (http://www.bls.gov/oes/current/oes_nat.htm). Based on DOL data, an average hourly wage of $54.68 (PHL directors, occupational code 11-9111) is estimated for all 57 respondents. To account for potential increases due to the COVID- 19 response, the hourly wage rate has been doubled to $109.36 to account for fringe benefits and overhead (https://aspe.hhs.gov/pdf-report/guidelines-regulatory-impact-analysis). Table A-12 shows estimated burden and cost information.

There will be a total of 57 respondents and 57 responses.

Table A-12: Estimated Annualized Burden Hours and Costs to Respondents

Data collection Instrument: Form Name

Type of Respondent

No. of Respondents

No. of Responses per Respondent

Average Burden per Response (in hours)

Total Burden Hours

Hourly Wage Rate

Total Respondent Costs

Online Survey Instrument

PHL Director

40 / 60

$109.36

$4156.00

TOTALS

$4156.00

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to the respondents other than their time to participate in each data collection.

Annualized Cost to the Government

There are no equipment or overhead costs. The only cost to the federal government would be the salary of CDC staff and contractors being used to develop the data collection instrument, collect data, and perform data analysis. The total estimated cost to the federal government is $7058.00. Table A-14 describes how this cost estimate was calculated.

Table A-14: Estimated Annualized Cost to the Federal Government

Staff (FTE)

Average Hours per Collection

Average Hourly Rate

Total Average Cost

Biologist – GS-13, Step 2;

Development of OMB package and will perform data analysis

$47.75 /hour

$955.00

Evaluation Fellow – GS-11, Step 1;

Plan and implementation of data collection, and develop evaluation plan for data collection

$32.42 /hour

$648.00

Deputy Director for Science – GS-15, Step 6;

Development of OMB package, plan and implement data collection, develop summary reports and presentations

$74.94 /hour

$300.00

Clinical Research Associate/Contractor - Booz Allen Hamilton;

Development of data collection tool and will perform data analysis

$47.75 / hour

$955.00

Sr. Specialist, Advanced Molecular Detection (APHL)

Development of data collection tool and serve as contact for PHL

$1200.00

Sr. Specialist, Monitoring and Evaluation (APHL)

Programming of data collection tool and data analysis

$3000.00

Estimated Total Cost of Information Collection

$7058.00

Explanation for Program Changes or Adjustments

This is a new data collection.

Plans for Tabulation and Publication and Project Time Schedule

As resources and respondents may be impacted by the COVID-19 pandemic, we propose that data collection begin in 11/02/2020. Data will be stored on secure, password-protected servers maintained by and only accessible to APHL. Once the survey period has closed all data collected will be stripped of any identifiers by APHL prior to sharing with CDC for analysis. Survey responses will be analyzed by descriptive and inferential statistics. Linking collected data to existing data sources by non-personal identifiers (i.e. laboratory characteristics) may be used to increase the overall utility of the data collected. CDC project staff will summarize key findings for presentation and reporting to the PHL community.

Project Time Schedule

Design instrument (COMPLETE)

Develop protocol, instructions, and analysis plan (COMPLETE)

Pilot test instrument (COMPLETE)
Prepare OMB package (COMPLETE)
Submit OMB package (COMPLETE)

OMB approval (TBD)
Conduct data collection (6 weeks)
Code data, validate data, and analyze data (2 weeks/8 consecutive weeks)
Prepare summary report(s) (2 weeks/10 consecutive weeks)
Disseminate results/reports (3 weeks/13 consecutive weeks)
Prepare manuscript and publication (if necessary) (8 weeks/21 consecutive weeks)

Reason(s) Display of OMB Expiration Date is Inappropriate

We are requesting no exemption.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

LIST OF ATTACHMENTS – Section A

Note: Attachments are included as separate files as instructed.

Attachment A – List of Public Health Laboratories Associated with APHL
Attachment B – Online Survey Instrument Web Version
Attachment C – Online Survey Instrument Word Version

REFERENCE LIST

1. Centers for Disease Control and Prevention (CDC). "National Public Health Performance Standards Program (NPHPSP): 10 Essential Public Health Services.". Available at http://www.cdc.gov/nphpsp/essentialservices.html. Accessed 8/14/14.

2. Institute NHGR. January 17, 2018 2019. Regulation of Genetic Tests, on National Institutes of Health. www.genome.gov/10002335/regulation-of-genetic-tests/. Accessed 04/01/2020.

3. Food U, Administration D. 2016. Use of Public Human Genetic Variant Databases to Support Clinical Validity for Next Generation Sequencing (NGS)-Based In Vitro Diagnostics: Draft Guidance for Stakeholders and Food and Drug Administration Staff”, 8 July 2016.

4. Food U, pdf DAJ--hwfgdmdgu. 2018. Considerations for design, development, and analytical validation of next generation sequencing (NGS)–based in vitro diagnostics (IVDs) intended to aid in the diagnosis of suspected germline diseases.

5. Santani A, Simen BB, Briggs M, Lebo M, Merker JD, Nikiforova M, Vasalos P, Voelkerding K, Pfeifer J, Funke B. 2018. Designing and Implementing NGS Tests for Inherited Disorders: A Practical Framework with Step-by-Step Guidance for Clinical Laboratories. J Mol Diagn doi:10.1016/j.jmoldx.2018.11.004.

6. Organization WH. 2018. The use of next-generation sequencing technologies for the detection of mutations associated with drug resistance in Mycobacterium tuberculosis complex: technical guide. World Health Organization,

7. Roy S, Coldren C, Karunamurthy A, Kip NS, Klee EW, Lincoln SE, Leon A, Pullambhatla M, Temple-Smolkin RL, Voelkerding KV, Wang C, Carter AB. 2018. Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines: A Joint Recommendation of the Association for Molecular Pathology and the College of American Pathologists. J Mol Diagn 20:4-27.

8. Organization WH. 2011. Laboratory Quality Management System. World Health Organization, Clinical and Laboratory Standards Institute, Centers for Disease Control and Prevention, France.

9. de Abreu FB, Peterson JD, Amos CI, Wells WA, Tsongalis GJ. 2016. Effective quality management practices in routine clinical next-generation sequencing. Clin Chem Lab Med 54:761-71.

10. Aziz N, Zhao Q, Bry L, Driscoll DK, Funke B, Gibson JS, Grody WW, Hegde MR, Hoeltge GA, Leonard DG, Merker JD, Nagarajan R, Palicki LA, Robetorye RS, Schrijver I, Weck KE, Voelkerding KV. 2015. College of American Pathologists' laboratory standards for next-generation sequencing clinical tests. Arch Pathol Lab Med 139:481-93.

11. Institute CLS. 03/02/2018 2018. How CLSI Got the QSEs, on CLSI. https://community.clsi.org/about/blog/how-clsi-got-the-qses/. Accessed 12/30/2019.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Determining the Landscape of Quality Management Systems for Public Health Laboratories Performing Next-Generation Sequencing
Author	Gilliam, Adzua H. (CDC/DDPHSIS/CSTLTS/OD)
File Modified	0000-00-00
File Created	2021-01-13