OMB Part B document RJP edits (0828)_revised 1-17-2013

January 30, 2021

SUPPORTING STATEMENT PART B

for

Evaluation of Fetal Alcohol Spectrum Disorders Regional Training Centers

New Request

Primary Contact:

Elizabeth P. Dang, MPH

CDC/NCBDDD

1600 Clifton Road, NE
Mailstop E-86

Atlanta, GA 30333

[email protected]

404-498-3947

Fax 404-498-3550

B.  Collections of Information Employing Statistical Methods

B. 1.    Respondent Universe and Sampling Methods

The universe of respondents is all participants of trainings conducted by the FASD Regional Training Centers of the two-year period remaining of the cooperative agreement. Training participants will be invited to complete the evaluation instrument that is specifically designed to match the target audience of each setting. Each FASD Regional Training Center plans to use a variety of evaluation instruments, depending on the target audience and the training setting. Each evaluation instrument contains a core set of items that are the same across all sites for assessing knowledge, practice behaviors, and comfort and self-efficacy to perform certain skills related to the prevention, identification, and treatment of FASDs. Through this approach, the FASD Regional Training Centers will have evaluation data at the individual training center level in order to identify strengths and weaknesses of the trainings, allowing for identification of areas to improve, revise, or expand. In addition, because of the core evaluation measures that will be collected consistently across all FASD Regional Training Centers, it will be possible to evaluate certain aspects of the collective FASD Regional Training Centers’ activities which will provide information regarding the effectiveness of the FASD Regional Training Centers’ training efforts as a whole and will assist CDC’s FAS Prevention Team with future program planning. See appendices for all proposed evaluation instruments. Data will not be weighted, only summary descriptive data will be reported.

B. 2.    Procedures for the Collection of Information

Training participants will receive an evaluation form to complete prior to and at the end of each training session from the presenter or from an FASD Regional Training Center staff member assisting with organization. Participants will also receive a 3- or 6-month follow-up evaluation (depending on the FASD Regional Training Center) to assess retention of knowledge over time as well as changes in practice behaviors and changes in confidence and self-efficacy to perform certain skills related to the prevention, identification, and treatment of FASDs. Most of the FASD Regional Training Centers’ 3- or 6-month follow-up assessments will be conducted via an online survey. To link the pre-test to the post-tests, participants will be assigned a temporary unique number that links the two forms, but that is otherwise not linked with any information about the participant.

B. 3.    Methods to Maximize Response Rates and Deal with Non-response

The program presenters will include a statement of the purpose of the evaluation and the need for collecting this information to improve programs and resources. Since the completion of the evaluation is conducted anonymously, it is not feasible to deal with non-response. 

B. 4.    Tests of Procedures or Methods to be Undertaken

The FASD Regional Training Centers will be collecting evaluation information on a multitude of trainings for medical and allied health students, residents, and practitioners. Each FASD Regional Training Center plans to use a variety of evaluation instruments, depending on the target audience and the training setting with the intent to evaluate their own training center’s activities. Each evaluation instrument also contains a core set of items that are the same across all sites for assessing knowledge, practice behaviors, and comfort and self-efficacy to perform certain skills related to the prevention, identification, and treatment of FASDs. Through this manner, it will be possible to evaluate certain aspects of the collective FASD Regional Training Centers’ activities using consistent measures. See appendices for all proposed evaluation instruments. 

Evaluation surveys in pencil-and-paper format, as well as online web-based evaluation surveys in some cases, will be given to participants of RTC trainings at pre-training, immediate post-training, and post-training follow-up. The evaluation assessments will be administered by the training organizers.

For the purpose of the evaluation, no individually identifiable information is being collected. Data collection, including for the 3- or 6-month follow up, will be anonymous; the evaluation forms themselves will have no identifying information or any link to names or contact information.

Paper and pencil surveys will be entered into electronic databases by each RTC with quality assurance (QA) methods such as double-pass entry in place. Surveys conducted electronically will be transferred to retention databases. All data will be checked for out of range values and missing value codes inserted as part of the QA process. Otherwise, data will be preserved as originally provided by participants.

Survey items result in a number of data types. Reporting methods will be dictated by the types of data described below. Centers will conduct analyses according to their programmatic needs. For review and planning, data will be summarized for all respondents for a given form and the data will be summarized across trainings. In comparing across trainings, results for all participants will be reported, though in conducting statistical tests, only participants with data available for all trainings being compared will be included. This will prevent errors due to self-selection bias.

Descriptive participant information is mostly categorical in nature and will be reported via frequency counts/percents within categories, with chi-square methods used to test differences among categories where appropriate.

Knowledge Questions are multiple choice items that will be scored as correct, incorrect, skipped or not tested. Note: participants are only tested on a subset of Knowledge Questions relevant to their training. Scoring results will be added to the database together with original item responses. A summary percent correct will be computed for each participant on each survey as: #correct/(#incorrect + #skipped). Pre-post and across-trainings statistical comparisons at the item level will use chi-square tests and at the participant level (percent correct) will use z-tests.

Two types of scale responses are included in surveys. Likert scales are 5-item labeled category scales. These result in ordinal type data and will be reported as frequencies/percents within categories and with appropriate charts. Pre-post and across trainings statistical comparisons will use Wilcoxon signed rank tests for two time points and Friedman tests for multiple points. Eleven-point self-rating interval scales are used for some items. These include a verbal anchor at each end with participants’ responses representing a number along the 11-point scale rather than a particular category. These scales result in interval level data (a number from 1 to 11) and can be reported as means and 95% CIs. Comparisons across trainings will be made with paired t-tests for two time points and repeated measures ANOVA for multiple points.

B. 5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Questionnaires and protocols were developed in collaboration with and are reviewed by staff of the National Center on Birth Defects and Developmental Disabilities and by members of the FASD Regional Training Centers:

CDC collaborators

Nancy Cheal, MS, PhD, Acting Team Leader, FAS Prevention Team, DBDDD/NCBDDD/CDC, 1600 Clifton Road, MS E-86, Atlanta, GA 30333 404-498-6764, [email protected]

Elizabeth Dang, MPH, Behavioral Scientist, FAS Prevention Team, DBDDD/NCBDDD/CDC, 1600 Clifton Road, MS E-86, Atlanta, GA 30333 404-498-3947, [email protected]

Leanna Fox, MPH, Public Health Advisor, Prevention Research Branch, DBDDD/NCBDDD/CDC,1600 Clifton Road, MS E-86, Atlanta, GA 30333 404-498-0604, [email protected]

Catherine Hutsell, MPH, Health Education Specialist, FAS Prevention Team, DBDDD/NCBDDD/CDC, 1600 Clifton Road, MS E-86, Atlanta, GA 30333 404-498-3825, [email protected]

FASD Regional Training Centers collaborators

Christiane Brems, PhD, ABPP, Co-Director, Behavioral Health Research & Services, UAA Director of Clinical Training and Professor of Psychology, Center for Behavioral Health Research & Services, University of Alaska Anchorage, 3401 E. 42^nd Street, Suite 200/201, Anchorage, AK 99508, [email protected]

Kristy Durkin, MSW, LCSW, Social Worker, Department of Family and Community Medicine, Meharry Medical College, 1005 D.B. Todd Jr. Blvd, Nashville, TN 37208-3599, [email protected]

Bridget Hanson, PhD, Research Assistant Professor, Center for Behavioral Health Research & Services, University of Alaska Anchorage, 3401 E. 42^nd Street, Suite 200/201, Anchorage, AK 99508, [email protected]

Joyce A. Hartje, PhD, Evaluation Research Manager, Center for the Application of Substance Abuse Technologies, University of Nevada, Reno, 800 Haskell St., First Floor, Reno, NV 89509, [email protected]

Mark Johnson, PhD, Co-Director, Behavioral Health Research & Services, Professor of Psychology, Center for Behavioral Health Research & Services, University of Alaska Anchorage, 3401 E. 42^nd Street, Suite 200/201, Anchorage, AK 99508, [email protected]

Robert Levine, MD, Professor of Family and Community Medicine, Director of the Research Division, Meharry Medical College, 1005 D.B. Todd Jr. Blvd., Nashville, TN 37208-3599, [email protected]

Ginger Mongeau, BBA, Data Manager, Center for Behavioral Health Research & Services, University of Alaska Anchorage, 3401 E. 42^nd Street, Suite 200/201, Anchorage, AK 99508, [email protected]

Becky Porter, MS, LPC, Center for Behavioral Health Research & Services, University of Alaska Anchorage

3401 E. 42^nd Street, Suite 200/201, Anchorage, AK 99508, [email protected]

Nancy Roget, MS, Principle Investigator/Executive Director, Center for the Application of Substance Abuse Technologies (CASAT), University of Nevada, Reno, 800 Haskell St., First Floor, Reno, NV 89509, [email protected]

Carolyn Szetela, PhD, Assistant Professor, Department of Professional and Medical Education

Program in Clinical and Research Ethics, Meharry Medical College, P.O. Box 1832, 1005 Dr. D.B. Todd Jr. Blvd., Nashville, TN  37208, [email protected]

Barbara Vardalas, MA, Associate Researcher, University of Wisconsin, Department of Family Medicine, 1100 Delaplaine Court, Madison, WI 53715, [email protected]

Georgiana Wilton, PhD, Associate Scientist, University of Wisconsin School of Medicine and Public Health, Department of Family Medicine, 1100 Delaplaine Court, Madison, WI 53715, [email protected]

Roger Zoorob, MD, MPH, FAAFP, Frank S. Royal Sr. Professor and Chair, Department of Family and Community Medicine, Meharry Medical College, Director, Program in Family Medicine, Vanderbilt University, 1005 D.B. Todd Jr. Blvd., Nashville, TN 37208-3599, [email protected]