SECTION B: COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS.
1. Respondent Universe and Sampling Methods
Sickle Cell Disease and Newborn Screening Program (SCDNBSP) Measurement Strategy
The respondent universe includes all clients that test positive for sickle cell disease or carriers of the sickle cell gene mutation through the newborn screening program and are enrolled in the SCDNBS program and are receiving services at the six grantee networks. We will approach 100% of those that are eligible based on provision of informed consent and on a positive sickle cell disease or sickle cell trait diagnosis. While it is not possible to determine exactly how many clients will request services, we anticipate a sampling frame of approximately 200 clients per grantee site based on current case load. This sampling method may result in selection bias as the sample will contain only those clients with sickle cell disease or carriers of the sickle cell gene mutation who choose to participate. While not scientifically robust, the project is a demonstration project not a research project.
Sickle Cell Disease Treatment Demonstration Program (SCDTDP) Measurement and Quality Improvement Activities
The respondent universe for both activities includes the nine network sites and all the individuals with sickle cell disease and their caretakers enrolled in the networks of the SCDTDP. Because these data will also be useful to sites in monitoring individual patient outcomes and identifying patients in need of specific services, we will strongly encourage sites to capture data from all patients served at these sites; however, some sites do not have the resources to collect data for all patients served. Based on previous experience with these sites and approximations for adequate sample size, we are asking sites for a sample size of 100. We believe that a sample size of 100 will be adequate to detect trends over time in the measures being collected. For these sites, we will ask the site director to provide an estimate of the total number of clients, and provide on a monthly basis a set of sealed envelopes with randomly sequenced inclusion or exclusion information with an appropriate sampling fraction to generate the minimum number of required respondents. Clients are enrolled into the SCDTDP through multiple processes such as referrals and self selection. Because some of the SCDTDP and SCDNBSP grantees are partners, there is a possibility that these programs may have common clients.
2. Information Collection Procedures
Sickle Cell Disease and Newborn Screening Program (SCDNBSP) Measurement Strategy
The Minimum Database Project Sickle Cell Disease and the Minimum Database Project Sickle Cell Trait Questionnaires will be administered by grantees to clients or caregivers when they present for services. At the time of enrollment, SCDNBSP participants will be informed about the data collection and clients will be asked to participate in either the sickle cell disease questionnaire or the sickle cell trait questionnaire depending on their disease or carrier status. The program will enroll participants on a rolling basis such that new patients will be added to the program as they present for services and provide consent.
Data will be collected at two points annually for the Sickle Cell Disease Questionnaire when clients and caregivers are enrolled into the SCDNBS Program and at follow-up after enrollment.
Data will be collected once annually for the Sickle Cell Trait Questionnaire.
The Hemoglobinopathies Emerging Populations Form (Client Family Communication) serves as a stand-alone form for the other HRSA hemoglobinopathies programs such as the SCDTDP, with its content embedded in the Minimum Database Project Sickle Cell Disease and the Minimum Database Project Sickle Cell Trait questionnaires.
The HRSA hemoglobinopathies programs also plan to use this questionnaire in developing educational materials, prioritizing outreach activities and informing decisions for future funding requests.
The Minimum Database Project Sickle Cell Disease (MDP SCD) Questionnaire
This questionnaire collects data on clients with sickle cell disease being served by grantees and includes questions on demographics, diagnoses, insurance status, family and healthcare systems of care, recent healthcare utilization and outcomes, and services received (genetic counseling, education, social services) by the client and/family. The questionnaire is implemented at the grantee site with the client (if adult) or client’s guardian/caretaker.
The Minimum Database Project Sickle Cell Trait/Carrier (MDP SCT) Questionnaire
This questionnaire collects data on clients being served by grantees with or at risk for carriers of the sickle cell gene mutation. The questionnaire includes questions on demographics and provides grantees with an opportunity to document services received by the client and/or family including genetic counseling, education, and genetic testing. The questionnaire is implemented at the grantee site or by telephone with the client (if adult) or client’s guardian/caretaker.
The Hemoglobinopathies Emerging Populations (Client Family Communication) Questionnaire
This questionnaire collects data on the race, ethnicity, country of origin, and language spoken in the home of clients (if adult) or client’s guardian/caretaker (in infant or child) being served by grantees. The questionnaire will serve as a modular component for both of the above questionnaires and will be integrated within the National Coordinating and Evaluation Center (Evaluation Center, defined on page 2 of Part A) Minimum Dataset Project Sickle Cell Disease Questionnaire, and the Evaluation Center Minimum Dataset Project Sickle Cell Trait Questionnaire upon receipt of OMB approval. Approval is currently sought for the form on its own merits.
Quality Assurance
Quality assurance for all data related processes including data collection, entry, analysis and reporting will be conducted at three levels. The administrative level will involve developing quality control training modules for all data representatives involved in the data collection processes, (attachment D). The training modules that will complement the data collection forms will be offered via face-to-face meetings followed by webinar presentations and teleconferences. Teleconferences will be primarily used to troubleshoot issues concerning data collection and entry procedures. The training topics that will be covered during the initial face-to-face meeting and follow-up webinar presentations will include:
procedures to obtain consent from clients,
instructions and FAQs to explain procedures of the study,
instructions to explain and assist clients on various sections/items on the forms,
instructions on data recording on the hard copy form during client interview,
secure storage of completed hard copies of the forms,
instructions and how-to manual to securely log onto web server for data entry, and
how-to instructions for data entry and saving.
The technical level will involve implementing data control measures and communicating with all grantees on a regular basis to provide technical assistance and troubleshoot issues related to data collection and entry procedures. This level will include quality control processes as follows;
documentation of quarterly data submission flow,
error checking and coding by the data collection software system,
verification of data irregularities for 20 percent of data per variable,
communicating and resolving data irregularities with the data representatives,
ongoing technical assistance via teleconferences to discuss and troubleshoot data related issues with the data representatives, and
communicating challenges and resolutions during TA, with the supervisory staff.
The evaluative level will involve conducting statistical procedures for the purpose of evaluating stated objectives and intended outcomes of the data collection processes. This level will include quality control processes as follows:
data coding and editing to identify outliers and missing data,
missing data analysis to ensure completeness of data,
documentation and record-keeping of number of data submissions, tracking of technical assistance reports, and working with the technical team and data representatives from all grantee sites on data related issues,
frequency-based and multivariate analyses to prepare reports on a regular basis, and
communication of results with all grantees and supervisory staff on a regular basis.
Sickle Cell Disease Treatment Demonstration Program (SCDTDP) Measurement and Quality Improvement Activities
Sickle Cell Disease Treatment Demonstration Program Measurement
Data will be collected on an annual basis from patients obtaining care at the SCDTDP sites:
At baseline when the patients and caregivers are initially enrolled; and
Follow-up data will be collected every 12 months after enrollment.
Grantee networks will enroll participants on a rolling basis such that new patients/clients will be added to the study over a specified period. At the time of enrollment, SCDTDP participants will be informed about the data collection as part of the informed consent process. Three of the instruments selected and one developed by the Technical Working Group (TWG) include the:
SF-36 ®– an instrument developed for the Medical Outcomes Study in the 1980’s and which has been widely adopted for the measurement of functional health and well being in many other subsequent studies. The validity of SF-36 scales based on psychometric studies has been supported in results from clinical trials comparing scores for patients before and after treatment. The reliability of the eight scales and two summary measures has been estimated using both internal consistency and test-retest methods, and generally have exceeded the minimum standard of 0.70. This instrument is designed to be self-administered.
PedsQL ® - The Pediatric Quality of Life Inventory is a set of instruments developed by James W. Varni, Ph.D. to assess health related quality of life in pediatric populations. The 23-item scales were developed to measure the core dimensions of health as defined by the World Health Organization, and the measurement model consists of developmentally appropriate forms for children by age. The measurement must include both child self-report and parent proxy-report and has been widely used in pediatric health outcomes evaluation. Reliability estimates for the core scales in both the self- and proxy-report are greater than the .70 standard, and validity has been demonstrated in known groups comparisons and correlations with other measures of disease burden. Parents will complete the parent proxy instrument for children aged 2-18. Children aged 5-7 will complete the PedsQL Young Child Report. Children ages 8-12 will complete the PedsQL Child Report, and children ages 13-18 will complete the PedsQL Teen Report. These instruments are designed to be self-administered.
Medical Home Family Index – an instrument developed by the Center for Medical Home Improvement to measure patient/caregiver satisfaction with the care received from their primary care provider. This instrument is designed to be self-administered.
Utilization Questionnaire- an instrument developed by the Technical Working Group of the SCDTDP for the specific information needs of this study that could not be met with pre-existing instruments. This questionnaire collects information on the demographic characteristics of the patients enrolled in the SCDTDP, their health/disease status, and their use of health care services and sickle cell disease treatments. This instrument is designed to be administered by an interviewer who will collect the information either through an interview with the patient or caregiver (if patient is a minor), the medical record, or project database. Items that cannot be obtained through medical records or a project database will be obtained through an interview conducted during a regularly scheduled office visit.
Hemoglobinopathies Emerging Populations Form (Client Family Communication) - this questionnaire collects data on the race, ethnicity, country of origin, and language spoken in the home of clients (if adult) or client’s guardian/caretaker (in infant or child) being served by grantees.
Sickle Cell Disease Treatment Demonstration Program Quality Improvement Activities
QI Instrument: an instrument developed by the contractor for the specific information needs of the Hemoglobinopathy Learning Collaborative that could not be met with pre-existing instruments. This instrument collects information on clinical Quality Improvement data including: patient treatment plans, newborn screening rates, timely follow-up on positive screens, immunizations, and Preventative Care Program (PCP) visits. Additionally, the instrument collects team-focused information about the network team participating in the Learning Collaborative (e.g. percent of teams that met biweekly, percent that participated in calls, percent rated highly on team assessments). These data, from the QI Instrument, are to be self-reported on a monthly basis throughout the duration of the program.
Quality Assurance
The National Coordinating Center (NCC, described on page 5 of Part A) will provide training for the data collection staff approximately one to two months prior to the initiation of the data collection. The training topics will cover the procedures for explaining the study and the consent process instructing participants on the completion of the forms and providing instruction on completing individual items on each of the forms. The NCC will train on procedures for verifying the correct record, assuring data security, and standardized conventions for completing the form, (see attachment D).
In addition to training data collection staff, the NCC will develop a control system that will be the primary mechanism for tracking the flow of data and the progression of activities throughout the study. The control system will generate reports on the number of forms completed by the Networks and serve as the repository for the data once it has been uploaded to NCC servers. Other quality control procedures will also be implemented:
internal data editing specifications to detect problems such as out of range responses;
generation of frequencies on a periodic basis to detect any outliers or data inconsistencies that may require special attention; and
monitoring reports to facilitate discussion with the Network sites during periodic conference calls regarding any issues of performance or compliance.
3. Methods to Maximize Response Rates
Sickle Cell Disease and Newborn Screening Program (SCDNBSP) Measurement Strategy
Prior to data collection, all grantees have established a pre-existing relationship with the clients and their families. This facilitates recruitment and retention. Although recruitment is between 90%-100%, there have been instances where clients have not provided all of the information necessary for a complete interview. One of the main reasons for incomplete or missing responses has been the length of the survey and the time to complete it. We anticipate a response rate of 80% or more based on past experiences, on grantees recruitment, and on follow-up with clients. In the past, we have seen that due to intensive case management, grantees have been successful in interviewing 80% or more clients and their families who seek services at their site. In order to maximize response rates, we have reduced the length of the survey thereby minimizing the burden on the client. We also propose that grantees continue to maintain pre-existing relationships with their clients and especially with those who are most vulnerable to being lost to follow-up.
Sickle Cell Disease Treatment Demonstration Program (SCDTDP) Measurement and Quality Improvement Activities
Sickle Cell Disease Treatment Demonstration Program Measurement Strategy
The focus of the National Coordinating Center (NCC) will be to provide the SCDTDP Networks with technical assistance in the recruitment, retention and monitoring of their patient sample. A response rate of 80% is feasible based on previous experience with the Networks. First, the majority of the participants to be enrolled in this project typically have an established relationship of care with the Network providers. Sickle cell disease is rare in the population at large and requires a unique and complex set of treatments and therapies; therefore, patients tend to remain in one geographic region for an extended period of time and establish relationships with a specific set of providers. The pre-existing relationship between the network clinical providers (university hospital centers and federally qualified health centers) and the patients to be enrolled in the SCDTDP will facilitate recruitment and retention. There are some subpopulations with the sickle cell community such as immigrants, adults without a medical home or adolescents transitioning into adult care who may be more transient and therefore difficult to locate for follow-up. Networks will attempt to identify if these individuals are anticipating transitioning to a new area that are not currently served by a Network and work closely with these individuals to maintain up-to-date contact information and to transition them to new providers in their new area. The nature of sickle cell disease and the intensity of the SCDTDP interventions require the Networks to maintain frequent contact with their sickle cell disease patients (at least every other month). This frequency of contact may diminish the risk of loss to follow-up as contact information will be regularly updated during these visits.
Sickle Cell Disease Treatment Demonstration Program Quality Improvement Activities
The National Coordinating Center (NCC) will provide the SCDTDP Networks with technical assistance and support throughout the Learning Collaborative. A response rate of 90% is feasible based on previous experience leading similar Learning Collaboratives.
4. Tests of Procedures
Sickle Cell Disease and Newborn Screening Program (SCDNBSP) Measurement Strategy
Eight grantees tested the three questionnaires in June 2010 to ensure that the requests were clearly stated, that the content was relevant to the target population, and that the amount of time needed to complete them was reasonable. The findings indicated necessary revisions in a few of the elements – expanding the range of immunization options, adding other hemoglobinopathy traits as options, and the deletion of questions that were not relevant, such as date of birth of newborn with carriers of the sickle cell gene mutation. These changes are reflected in the current versions of the questionnaires.
Sickle Cell Disease Treatment Demonstration Program (SCDTDP) Measurement and Quality Improvement Activities
Sickle Cell Disease Treatment Demonstration Program Measurement Strategy
As noted earlier, the SF-36 ®, the PedsQL®, and the Medical Home Family Index have been previously used in other research and their psychometric properties rigorously evaluated (Ware, Snow, Kosinski 1992, Varni, Limbers, Burwinkle, 2007, Cooley et. al, 2003). Reference articles on these instruments can be provided on request.
The Utilization Questionnaire was developed for this study. It was cognitively tested in April 2008 with 9 participants. The findings indicated necessary revisions in a few of the elements – expanding the range of educational choices, adding other symptoms and complications, and the deletion of self-report as a source of immunization status.
Sickle Cell Disease Treatment Demonstration Program Quality Improvement Activities
The family of measures to improve quality of care for sickle cell disease were developed to be used in the Hemoglobinopathy Learning Collaborative. These measures were derived from established quality metrics for sickle cell disease and the measures align with National Heart Lung and Blood Institute’s Management of Sickle Cell Disease guidelines and American Academy of Pediatrics Policy Statement on the Health Supervision of Sickle Cell Disease.
5. Statistical Consultants
The data collection and analysis will be conducted under the overall leadership of the National Initiative for Children’s Healthcare Quality, working in concert with subcontractor Boston Medical Center. Names of key staff are listed below:
Project Director
Charles Homer, MD, MPH
National Initiative for Children’s Healthcare Quality
30 Winter St., 6th Floor
Boston, MA 02108
617-391-2714
Director of Applied Research and Evaluation
Marianne McPherson, PhD, MS
National Initiative for Children’s Healthcare Quality
30 Winter St., 6th Floor
Boston, MA 02108
617-391-2710
Director, Data Analysis
C. Jason Wang, MD, PhD
Adjunct Associate Professor of Pediatrics
Boston University Schools of Medicine and Public Health
Acting Associate Professor of Pediatrics,
Stanford University School of Medicine
[email protected]; [email protected]
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Health Resources and Services Administration/Maternal and Child Health Bureau |
| Author | EDU |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |