NIST ClinQAP Impact & Needs Assessment Questionniare - Supporting Statement

OMB Control No. #0693-0033 – NIST Generic Clearance for Program Evaluation Data Collections

Expiration Date 03/31/2016

National Institute of Standards and Technology (NIST)

Clinical Quality Assurance Program (ClinQAP)

Impact and Needs Assessment Questionnaire

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

A questionnaire was developed for the current and potential participants in the Clinical Quality Assurance Program, (ClinQAP) which is provided by the Chemical Sciences Division (CSD) of the Material Measurements Laboratory (MML) at the National Institute of Standards and Technology (NIST).

The ClinQAP is currently comprised of three subprograms that routinely provide interlaboratory assessments for measurements of micronutrients (e.g. vitamin A, vitamin C, vitamin E, carotenoids, coenzyme Q10), vitamin D metabolites, and fatty acids in human serum and plasma matrices. The longest-standing of these three subprograms, the Micronutrients Measurement Quality Assurance Program (MMQAP), was initiated in 1984 and has been operating essentially unchanged since 2000; 74 comparability studies have been conducted to date. The Vitamin D Metabolites Quality Assurance Program (VitDQAP) was formed in 2009 and has provided seven comparability studies, whereas the Fatty Acids Quality Assurance Program (FAQAP) has conducted one pilot study to date. The MMQAP, VitDQAP, and FAQAP are all in various “life-stages” ranging from mature (MMQAP) to infancy (FAQAP), but all have the common goal of improving the comparability of clinical measurements for selected analytes.

The CSD is currently performing needs- and impact-assessments for many of its programs and services. Internal assessments have been made by the Technical Project Leaders (TPLs) of the three respective subprograms, and all agree that the current ClinQAP has some potential limitations that should be addressed. Primarily, there is concern that the ClinQAPs currently address only a narrowly-focused group of analytes and matrices and hence a limited portion of the clinical research community. Furthermore, since the mature programs have already accomplished many of their intended research goals (i.e., improvement in the comparability of laboratory measurements), the continuation of the subprograms in their present format, particularly the MMQAP and the VitDQAP, might not be technically justifiable.

Rather than speculate on the community’s needs and whether these needs align with the CSD’s research interests and goals, the TPLs of the subprograms have developed a questionnaire for the current and potential participants of the ClinQAP. The questionnaire has two intentions: 1) to assess if the current ClinQAPs are necessary for meeting the performance validation requirements of the participating laboratories and 2) to monitor and support the emerging measurement needs of the clinical community with the goal of expanding the potential research areas and hence impact of the ClinQAP.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The questionnaire was developed by the ClinQAP program coordinator in collaboration with the three subprogram TPLs. When combined, the program coordinator and TPLs have over 50 years of experience coordinating quality assurance studies for the nutritional and clinical communities, and have maintained positive, collegial relationships with the participating laboratories. Therefore, the program coordinator and TPLs are well-qualified to construct a questionnaire to meet the intentions described in Question #1. The questionnaire passed several rounds of testing through the TPLs and was also tested with other CSD staff members with interest in clinical chemistry and the field of metabolomics, which are potential growth areas for the ClinQAP. Suggestions for improvement were incorporated into the questionnaire during each iteration of internal review and testing.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

Prior to the execution of the questionnaire, the current and prospective MMQAP, VitDQAP, and FAQAP participants will receive a preliminary email from their respective TPLs through their regular communication channels. For example, the VitDQAP has a dedicated NIST email address, vitdqap@nist,gov, which is exclusively used to communicate with participants of the VitDQAP. The email will contain a preliminary announcement of the ClinQAP and the questionnaire:

“Dear Colleagues,

Thank you for your participation or interest in participating in the <MMQAP, VitDQAP, or FAQAP>.

As you might be aware, NIST currently offers three quality assurance programs (QAPs) for the determination of micronutrients, vitamin D metabolites, and fatty acids in serum and plasma matrices, the MMQAP, VitDQAP, and the FAQAP, respectively. Given the similarity in the operations of these programs, NIST is consolidating them into one larger program, the NIST Clinical Quality Assurance Program, or ClinQAP. The primary goals of the ClinQAP are to support the comparability of clinical measurements through the MMQAP, VitDQAP, and the FAQAP, and to monitor and support the emerging measurement needs of the clinical community.

To help the ClinQAP meet its goals, a questionnaire has been designed requesting your feedback. The questionnaire will be conducted through a web-interface, and a link will be available soon. Please anticipate an email from [email protected] announcing the availability of the questionnaire.

In the meantime, if you have any questions regarding the ClinQAP or the questionnaire, please don’t hesitate to ask.

Sincerely,

<the TPLS of the MMQAP, VitDQAP, or FAQAP>”

The current and potential participants in the ClinQAP will then receive an email inviting them to participate in the survey. The text of the email will read as follows:

“Dear Colleagues,

We recently informed you of the consolidation of the MMQAP, VitDQAP, and the FAQAP into a new program, the Clinical Quality Assurance Program, or ClinQAP. The primary goals of the ClinQAP are to support the long-term reliability of clinical measurements (through the MMQAP, VitDQAP, and the FAQAP), and to monitor and support the emerging measurement needs of the clinical community.

A questionnaire has been designed requesting your feedback, which will help the ClinQAP to meet its goals. The questionnaire is NOW AVAILABLE and can be accessed from our website, http://www.nist.gov/mml/csd/clinqap.cfm, or by using the direct link below:

<URL to be determined>

We greatly appreciate your feedback and request your participation in this questionnaire by <month, day, 2014>. If you have any questions regarding the questionnaire, please don’t hesitate to ask by responding to this email.

Sincerely,

Mary Bedner, on behalf of the Clinical QA Coordinators”

The email requesting feedback via the questionnaire will be sent initially to the approximately 150 current and prospective participants of the ClinQAP. After that time, any additional prospective participants will also be asked by email to participate in the questionnaire until the generic clearance expires on 3/31/2016. We anticipate that approximately 250 laboratories will be offered an opportunity to participate in the questionnaire during that time period.

We anticipate a 75% response rate to the questionnaire, and periodic emails will be sent to the participants as a reminder to participate and to provide feedback.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The questionnaire will be conducted through a web interface, and the responses will be collected into a database that is stored on a local server (to be determined). The ClinQAP program coordinator, the subprogram TPLs, and the CSD contractor responsible for generating the web interface will have access to the database containing the responses. Since the collection instrument contains many open-ended questions, the ClinQAP coordinator and the TPLs anticipate a wide-range of responses that will not be easily generalized to the participant population. Furthermore, the participants in the ClinQAP are kept anonymous, and reporting the results of the questionnaire could potentially reveal their identities. Therefore, the results from the questionnaire will not be formally reported, but rather the TPLs will review the individual responses to assess:

1) the approximate percentage of respondents that feel the current subprograms are necessary/not necessary for meeting their laboratory performance validation requirements. If, for example, the majority of the participants in the current MMQAP indicate that services offered by the College of American Pathologists (CAP) are sufficient to meet their accreditation needs for vitamins A and E, we may stop offering comparability studies for these particular analytes. The MMQAP participants would then be informed that the decision was made to phase-out the program for vitamins A and E based on the feedback received in the questionnaire.

2) if there are any predominant trends in the participant responses to the open-ended questions. If, for example, several of the participants express difficulty measuring toxic metals (e.g., arsenic) in dried blood spots, we may design and coordinate an interlaboratory comparison study to address this measurement challenge.

3) the subset of participants who are interested in metabolomics research and/or measurements of human urine, who may then be invited specifically to participate in a pilot interlaboratory study that has been recently proposed in the CSD.

3