Supporting Statement A for
Online Skills Training for PCPs on Substance Abuse (NIDA)
January 2011
Clinical Tools, Inc., with funding from the National Institute on Drug Abuse (NIDA)
Quandra Scudder
6001
Executive Boulevard
Suite 5230
Bethesda, MD 20892
Phone: 301-594-0394 or 301-526-8835
Fax: (301) 443-6277
e-mail: [email protected]
Table of Contents
A. JUSTIFICATION 4
A.1. CIRCUMSTANCES MAKING THE COLLECTION OF INFORMATION NECESSARY 4
A.2. PURPOSE AND USE OF INFORMATION COLLECTION 5
A.3. USE OF INFORMATION TECHNOLOGY AND BURDEN REDUCTION 10
A.4. EFFORTS TO IDENTIFY DUPLICATION AND USE OF SIMILAR INFORMATION AND USE OF SIMILAR INFORMATION 10
A.5. IMPACT ON SMALL BUSINESSES OR OTHER SMALL ENTITIES 11
A.6. CONSEQUENCES OF COLLECTING THE INFORMATION LESS FREQUENTLY 11
A.7. SPECIAL CIRCUMSTANCES RELATING TO THE GUIDELINES OF 5 CFR 1320.5 11
A.8. COMMENTS IN RESPONSE TO THE FEDERAL REGISTER NOTICE AND EFFORTS TO CONSULT OUTSIDE AGENCY 12
A.9. EXPLANATION OF ANY PAYMENT OF GIFT TO RESPONDENTS 14
A.10. ASSURANCE OF CONFIDENTIALITY PROVIDED TO RESPONDENTS 15
A.11. JUSTIFICATION FOR SENSITIVE QUESTIONS 15
A.12. ESTIMATES OF HOUR BURDEN INCLUDING ANNUALIZED HOURLY COSTS 16
A.13. ESTIMATE OF OTHER TOTAL ANNUAL COST BURDEN TO RESPONDENTS OR RECORD KEEPERS 16
A.14. ANNUALIZED COST TO THE FEDERAL GOVERNMENT 17
A.15. EXPLANATION FOR PROGRAM CHANGES OR ADJUSTMENTS 17
A.16. PLANS FOR TABULATIONS AND PUBLICATION AND PROJECT TIME SCHEDULE 17
A.17. REASON(S) DISPLAY OF OMB EXPIRATION DATE IS INAPPROPRIATE 18
A.18. EXCEPTIONS TO CERTIFICATION FOR PAPERWORK REDUCTION ACT SUBMISSIONS 19
LIST OF ATTACHMENTS
Attachment 1: References
Attachment 2: IRB Letter of Approval
Attachment 3: Screening Instrument
Attachment 4: Initial Contact Email/Participant Information
Attachment 5: Assessment Instrument
The proposed data collection is essential to the development and evaluation of a unique and innovative web-based program, SBIRTTraining.com, which would significantly contribute to the mission of the National Institute on Drug Abuse (NIDA). The Online Skills Training for PCPs on Substance Abuse project is authorized under U.S. code and supports NIDA's mission of “ensuring the rapid and effective dissemination and use of the results of that research to significantly improve prevention, treatment, and policy as they relate to drug abuse and addiction” (http://nida.nih.gov/about/aboutnida.html).
Authorization which makes collection of this data necessary is found in U.S.C. Title 42, Chapter 6A, Subchapter III, Part C, Subpart 15—National Institute on Drug Abuse which states:
(a) In general
The general purpose of the National Institute on Drug Abuse (hereafter in this subpart referred to as the “Institute”) is the conduct and support of biomedical and behavioral research, health services research, research training, and health information dissemination with respect to the prevention of drug abuse and the treatment of drug abusers.
(b) Research program
The research program established under this subpart shall encompass the social, behavioral, and biomedical etiology, mental and physical health consequences, and social and economic consequences of drug abuse.
With funding from NIDA
through a Small Business Innovation Research (SBIR) contract, this
project will develop online training and education materials for
primary care physicians (PCPs). “Primary care physician”
is defined as a physician who specializes in family medicine,
internal medicine, pediatrics in primary care, obstetrics or
gynecology in primary care, and emergency medicine for acute primary
care problems.
The educational goal of these materials is to
teach PCPs about screening, brief intervention, and treatment to
referral (SBIRT) for substance abuse. The Online
Skills Training for PCPs on Substance Abuse project
will create a multimedia, interactive program with didactic online
learning and skills training via video and Internet based chat
experiences mimicking the “Objective Structured Clinical
Examination” (OSCE) interactions familiar to PCPs from their
professional training.
In Phase I of this SBIR project, a curriculum for the online program was developed and reviewed by experts, and the feasibility of the technology transfer was demonstrated. In Phase II, the goals are to complete development of the online program and conduct an evaluation of the effects of the online program on learner knowledge, attitudes, and relevant clinical skills (such as screening for substance abuse). This application discusses the evaluation study. There are no previous data collections associated with this project.
If the evaluation shows the Online Skills Training for PCPs on Substance Abuse program to be effective, physicians will be much more interested in using the program when it is complete. Any physician with Internet access will be able to utilize a proven program that educates them about SBIRT. The data collected will also add to a limited but growing body of research investigating the effectiveness of the Internet as a medium for delivering professional education and training programs.
If the evaluation shows the program to be ineffective, we will be able to use the qualitative data collected to identify barriers to its effectiveness and use our remaining project time to correct these barriers and improve the program. In this case, the data collected will potentially have broad applications on the subject of barriers to the effectiveness of the Internet education and training programs. The data collected will allow the National Institute on Drug Abuse to assess the success of the project at developing useful education and training materials for PCPs, and will assist them with evaluating the success of this SBIR funded project.
This research will take place after the development of the website SBIRTTraining.com in order to evaluate effectiveness with the website. NIDA and Clinical Tools, Inc. (CTI) will use the data obtained from the proposed evaluation to assess the utility of the program to the target audience. The research time line includes several months to refine the activities based on research.
General Overview
An evaluation of the program will be conducted in a home or office setting with 80 primary care physicians. The proposed evaluation design is a two-group (intervention and control group), randomized trial with a crossover component. The major hypotheses for the proposed evaluation are: 1) PCPs in the intervention group will show greater gains in knowledge and improvements in attitude (from pre- to post-assessment) than participants in the control group; 2) PCPs in the intervention group will show greater gains in clinical skills as measures than those in the control group; and 3) PCPs in the intervention group will retain changes in knowledge, attitudes and skills over the follow-up period of six to nine weeks.
For the evaluation, 80 participants will be enrolled, with the expectation that 64 of the 80 participants will yield full data sets (completion rate of 80%-85%). The expected response rate for complete responses is based on previous experience of CTI researchers and consultants.
Subjects will be recruited via email to national professional membership groups, word of mouth, and advertisements in regional and national newspapers. Potential subjects will contact researchers directly and provide basic demographic information. Based on a Screening Instrument that clearly defines the recruitment guidelines, potential subjects will be invited to participate, and will be provided with information about the project and their participation in the study. See Attachment 3 for the screening instrument and Attachment 4 for the Initial Contact email/Participant Information. If they choose to participate, individuals will be randomly assigned to the control and intervention groups within the appropriate stratification subgroups (e.g., race, ethnicity, gender).
Evaluation Design: The proposed study design is a two-group, randomized trial with a cross over component, with the intervention group as a between-subjects factor, and time of assessment as a repeated measure. Participants will be randomly assigned to either an intervention group (Group A) or a control group (Group B). Subjects in Group A will complete knowledge and attitude measures and the OSCE style skills evaluation and Interpersonal skills evaluation via chat at baseline (pre-assessment), immediately after using the educational program (post-assessment), and at a follow up point approximately six weeks after completing the intervention (follow-up assessment) (please refer to the timetable in Table A.2-1 below). Subjects in Group B will complete the knowledge and attitude measures and the OSCE style evaluation at the beginning of the study (pre-assessment 1) and again at the six-week point of the study (pre-assessment 2). During the follow-up period, Group B subjects will then use the educational program for six weeks and complete the post-assessments.
Table A.2-1: Summative Evaluation Timetable
Partici-pants (Recruited over a six-month period prior to study) |
Initial Assessment |
Six-week Interval |
Second Assessment |
Six-week Interval
|
Third Assessment |
Start of Week 1 |
Weeks 1- 6 |
Start of Week 7 |
Weeks 7- 12 |
Week 13 |
|
Group A (40 PCPs) |
Pre-Assessment |
SBIRTTraining.com educational materials |
Post-Assessment |
Post-Intervention Interval |
Follow-up Assessment |
Group B (40 PCPs) |
Pre-Assessment |
No Intervention |
Pre-Assessment 2 |
SBIRTTraining. com educational materials |
Post-Assessment |
All subjects will be asked to complete the intervention in six weeks. The follow up period for Group A will be six weeks. Group B will have a six-week nonintervention period prior to their intervention. Thus, allowing one week for administering and completing the final evaluation, the study length for participants in either group will be 13 weeks from their enrollment in the study.
The crossover design allows participants in the control group to have access to the program after the initial test period. Administration of the pre-assessment at two points to the control group will provide information on the test-retest reliability of the measures. The follow-up assessment will provide data on whether or not the effects on knowledge and attitudes are maintained over a period of time.
Methods: Potential subjects will be electronically mailed an information packet containing a description of the research and contact information. Participants will be instructed to email back confirmation of interest and willingness to participate. Once these materials are received by CTI, willingness to participate will be verified via telephone or email by CTI staff members.
Each subject will then be provided, by e-mail, study directions and a participant ID number and password allowing access to the online program and assessment instrument. Participants will be required to sign in to the website each time they use the program.
Participants will complete the OSCE style chat interaction and assessment at three points during the study. There will be no time limit on completing the questionnaires. However, the OSCE is expected to take a minimum of 10 minutes and a maximum of 30 minutes to complete and the assessment is expected to take from 15 to 30 minutes.
Weekly reminders will be sent by e-mail to remind participants of participation requirements. The Principal Investigator and research staff at CTI will be available throughout the evaluation to answer any questions or to assist with any technical issues that may arise. The assessment will be completed online. The OSCE will occur over Internet-based “chat” (such as Google chat).
Assessment Instrument: The assessment instrument will measure the effects of the online program on knowledge and attitudes. The knowledge measure will contain multiple-choice questions assessing knowledge of the subjects addressed in the curriculum. For example:
Which of the following is true for the clinical guidelines from the NIAAA regarding screening?
A) All patients age 12 or older should be screened using a 10-item standardized scale
*B) All patients age 12 and older should be screened using a one-question pre-screen
C) Adolescent patients age 12 and older should be screened using the CRAFFT tool
D) Adolescent patients should not be screened unless they meet 1 or more risk factors
The attitude measure will assess attitudes towards treating patients with substance abuse. Each item will consist of a statement followed by a Likert scale on which subjects can indicate their level of agreement (from strongly agree to strongly disagree). For example, subjects will be asked to indicate their agreement with statements, such as, "Screening for substance abuse is too time-consuming to do routinely in my practice.” The scales will yield an average score for attitudes related to the assessment and treatment of substance use disorders.
Clinical skills will be measured using customized versions of the Interpersonal Skills Inventory and a SOAP note, a standard format for a medical note that a care provider would create after a patient encounter to describe subjective, objective, assessment, and plan elements of the clinical visit. The Self Assessment version of the ISI and the SOAP note will measure ability of the care provider to demonstrate appropriate clinical skills (patient interaction and documentation). See Attachment 5 for the Assessment Instrument.
Data Analysis: To assess the effects of the SBIRTTraining.com educational program on knowledge, a 2 (case groups) X 2 (assessment points) ANOVA on knowledge scores will be used. The hypothesis asserts that both groups will have similar knowledge scores at baseline, but Group A will show a greater gain in knowledge than Group B at the post-assessment point. The effects of the pattern of program use on knowledge will also be explored.
The effects of the SBIRTTraining.com program on attitudes towards assessing and treating substance abuse patients using best practice guidelines will be assessed using separate 2 (case groups) X 2 (assessment points) ANOVAs on attitude scores. It is predicted that the two groups will be similar on attitudes at baseline, but that the intervention group will show greater gains in attitude scores at the post-assessment point, indicating more positive views about following best practices and treatment guidelines. We will also analyze data from the usage logs to determine possible effects of the pattern of program use on attitudes.
Comparisons of Group A's scores on the post- assessment questionnaire (immediately after completion of the intervention) and its scores on the follow-up assessment questionnaire (six weeks following completion of the intervention) will be conducted to assess retention of observed changes in knowledge and attitudes. Changes in knowledge and attitudes will be assessed by comparing the average scores on a pre-assessment question versus the average on that same item in post-assessment scores. Our hypothesis asserts that there will be a statistically significant increase in knowledge and favorable attitudes for every item.
Lastly, the test-retest reliability of the knowledge and attitude measures will be examined. The reliability coefficient will be calculated by comparing Group B's pre-assessment 1 scores to pre-assessment 2 scores. Since this group will not have had an intervention between the two assessments, similar scores on both pre-assessments would indicate that the instrument is reliable, although some improvement due to the learning effect from simply taking the assessment is expected (Experiment Resources, 2008).
The data will be used by NIDA to determine the effectiveness of the education and training materials being evaluated.
To reduce respondent burden and to improve data quality, 100% of the data will be collected online. Subjects will not have the burden of the time involved in using mail to return their materials. Collection of data online will also speed data analysis; data will be available as soon as respondents answer, without the delay involved in awaiting responses via mail. Online data collection also decreases the chance of data entry error, because no separate data entry will be required. Data will be logged in real time as participants complete the forms.
Because all of the data including the demographic data that is collected as part of the screening process using the Screening Instrument will be collected online, a Privacy Impact Assessment is currently being conducted.
This SBIR project is developing a unique online continuing medical education program on the topic of screening, brief intervention treatment, and referral for substance abuse. There is no other evaluation of this new program. All information from this research will be reported directly to the funding agency.
This project is funded by a SBIR Phase II contract to CTI. Study participants are volunteer physicians. Some of the physicians will be from small businesses while others may be affiliated with larger institutions. It is anticipated that the online program will have a positive impact for the physicians coming from small businesses in terms of professional development through taking the training. Pre- and post-course assessments are a standard part of continuing medical education courses and so the experience will not take much more time than one of their usual continuing education experiences. Additional time for enrollment and the study assessment is expected to take less than an hour each time. Furthermore, there will be a positive impact at the local and state level by improving physician education and training. No other small businesses will be involved in our research.
In the summative evaluation, the pre-assessment, post-assessment, and follow-up assessment methodology contains the minimum number of data collection points required to evaluate the utility of the program. In order to assess the PCPs’ ability to retain what they learned by using the site, it is necessary to include the follow-up assessment. The crossover design allows us to expose all participants in the research to the possible benefits of the intervention. The data collection at three points (baseline, immediately post-intervention and at a follow-up point) is standard methodology for this type of research. If the proposed information collection is not completed, the utility of the web-based program cannot be assessed and physicians may lose the opportunity have an educational program with demonstrated benefits available for free via the Internet.
This proposed project fully complies with all guidelines of 5 CFR 1320.5.
The 60 Day Federal Register Notice of the proposed research was published July 28, 2010, Vol. 75, No. 144, on pages 44265-44266. One request for the instruments was received from the public. A copy of the data collection plans and instruments were provided, for information only and not for use.
Table A.8-1 below shows the names and contact information for all consultants on this project. Table A.8-2 below shows the role of each of the consultants. At this point, there have been no major problems that could not be resolved during consultation.
Table A.8-1: Consultants
Consultant |
Title |
Affiliation |
Telephone # |
Year Consulted |
Steve Applegate, MEd, MEd |
President, Instructional Designer |
Applegate Consulting |
(801) 349-2498 |
2009-2010 |
Peter Curtis, MD |
Adjunct Professor of Family Medicine |
University of Washington School of Medicine |
(919) 966-7890 |
2009-2010 |
Robert (Ted) Dietrich, MSS |
SBIRT Program Coordinator |
Denver Health and Hospital Authority |
(303) 436-6970 |
2009-2010 |
Peter Finelli, DO |
Private Practitioner and Addiction Medicine Specialist |
Private Practice in New Jersey |
(201) 796-4444 |
2009-2010 |
Ken Freedman, MD, MS, MBA |
Chief of Medicine |
Lemuel Shattuck Hospital |
(617) 971-3532 |
2009-2010 |
Peter Friedmann, MD |
Associate Professor of Medicine & Community Health |
Rhode Island Hospital |
(401) 465-9144 |
2009-2010 |
John Fromson, MD |
Associate Director of Post Graduate Medical Education |
Harvard University |
(617) 724-0408 |
2009-2010 |
Win May, MBBS, MS, PhD |
Associate Professor of Clinical Medical Education |
University of Southern California |
(323) 442-2381 |
2009-2010 |
Timothy McGrath, MD |
Private Family Practitioner |
Private practice in Mebane, NC |
(919) 563-2500 |
2009-2010 |
Giang T. Nguyen, MD, MPH, MSCE |
Assistant Professor of Family Medicine and Community Health |
University of Pennsylvania School of Medicine |
(215) 746-4157 |
2009-2010 |
Table
A.8-2: Consultant Roles
Consultant |
Program Content |
Assessment Instrument Development |
Data Analysis |
Steve Applegate, MEd, MEd |
X |
X |
|
Peter Curtis, MD |
X |
X |
|
Robert (Ted) Dietrich, MSS |
X |
X |
|
Peter Finelli, DO |
X |
X |
|
Ken Freedman, MD, MS, MBA |
X |
X |
|
Peter Friedmann, MD |
X |
X |
|
John Fromson, MD |
X |
X |
|
Win May, MBBS, MS, PhD |
X |
X |
|
Timothy McGrath, MD |
X |
X |
|
Giang T. Nguyen, MD, MPH, MSCE |
X |
X |
|
Tracy Shaw, MA |
|
X |
X |
Evaluative study participants will be eligible for an incentive of $225 ($75 upon enrollment in the study and completion of the first assessment, $75 for completing the assessment after completing the training, and $75 for the follow-up assessment). A participant who drops out prior to completing the first evaluation will receive no incentive, and a participant who drops out after completing two assessments will receive two incentives. Any assessment where less than 80% of the questions are answered will be ruled incomplete.
This monetary incentive is necessary to adequately compensate the participants, who are all physicians. The average salary for a primary care physician in the United States is $147,516 annually ($71/hour) (http://www.physicianssearch.com/physician/salary2.html). We selected a figure reflective of their hourly income in consideration of the extremely busy nature of physicians’ lives and thus the competing interests for their time. Research has shown that physicians are resistant to participation without financial incentives (Herber, et al, 2009). It is typical for health care professionals who provide consulting services to receive much higher hourly compensation rates than we are providing for their time involvement in this project. We have provides incentives to physicians at this rate on previous projects, for example, the Phase I portion of this project and Online Buprenorphine Practice Advisor for Physicians (NIDA Contract #HHSN271200555303C), and found it necessary in order to recruit a sufficient number of participants. Other, similar projects also compensate primary care providers at similar rates. For example, Short et al (2006) provided physicians $150 in incentives for taking three assessments and completing at least 4 hours of CME (Small Business Innovation and Research grant, National Institute of Mental Health (R44-MH62233). While they did not report the time involved in their published results, their assessment involved only paper surveys.
The proposed incentives are in no way meant to induce respondents to comply with unnecessary paperwork requests. Research has shown that offering incentives to participants greatly increases response rates as well as speed of response (Dillman, 1978; 2000). Participant incentives will be integral to obtaining desired response rates for all proposed studies.
The proposed research has been reviewed and approved as exempt by the Clinical Tools Institutional Review Board. The information provided will be kept confidential and will not be disclosed to anyone but the researchers conducting the study, except as otherwise required by law. Confidentiality will be maintained by using subject ID numbers rather than names on all data collection forms. Any identifying information inadvertently provided by participants will be promptly removed. Online data will be maintained on a secure server during the duration of the research, after which it will be deleted. Data linking subject ID with identifiers and all printed records will be kept in a locked filing cabinet accessible only to the Principal Investigator and project assistants involved in this research. All participants will complete informed consent documents that outline CTI's confidentiality procedures and will include information about circumstances under which participant information may be accessible to others. During analysis, data will be aggregated in such a way that no personally identifiable information can be obtained from all data reports or any published material. All electronic and paper data pertaining to identifiable data on subjects will be securely stored for three years, in keeping with NIH requirements, and then purged unless otherwise directed by NIDA. Standard human subjects guidelines will be followed. Confirmation of Human Subjects Institutional Review Board review and approval for exempt status are provided in Attachment 2.
The proposed project will fully comply with the Federal Registry Notice. The Privacy Act may apply to some data collection activities. Through coordination with the NIH Office of Management, the determination will be made as to whether the Privacy Act will be applied as indicated.
The proposed research does not involve asking participants sensitive questions. Although participants in the pilot test and evaluation will be asked questions concerning their professional knowledge and beliefs and attitudes about substance abuse assessment and treatment, no information about personal use of, or behaviors related to, illegal drugs will be solicited. It is essential to determine the program's effects, if any, on knowledge, attitude, and clinical skill related to SBIRT. No personal identifying information will be collected.
Table A.12-1 (below) shows the estimated burden of hours requested for this project, based on consultant input and previous testing of the website with fewer than 10 respondents.
Table A.12-1: Estimated Total Burden Hours Required
Type of Respondents |
Number of Respondents |
Frequency of Responses |
Average Time per Response |
Annual Hour Burden |
Primary Care Physicians (Evaluation) |
80 |
3 |
.75 |
180.00 |
Total |
80 |
- |
- |
180 |
Table A.12-2: Annualized Cost to Respondents
Annualized Cost To Respondents |
||||
Type of Respondents |
Number of Respondents |
Frequency of Response |
Hourly Wage Rate |
Respondent Cost |
Physicians |
80 |
3 |
$75.00* |
$13,500. |
|
Total $13,500. |
|||
*These figures are based upon seasonally adjusted hourly wage rates reported by the Bureau of Labor Statistics (www.bls.gov/cps).
There are no capital, start-up, operational, or maintenance costs to the respondents in providing the information required by this research.
Total costs associated with the project are estimated to be approximately $703,688 over a 2-year contract performance period. website development, formative research, pilot test of assessment instrument, an evaluation, creation of a final report, indirect costs, and the SBIR fixed fee. . In addition, it is estimated that one full-time-equivalent NIDA staff member will spend 20 percent of his or her time (520 hours) to manage and administer the project. Assuming an annual salary of $100,000, government personnel costs will be $20,000 over a 1-year period. The 1-year total project costs are $371,844 ($351,844 plus $20,000).
This is a new collection of information.
This project will require a maximum of 18 months from receiving OMB approval.
Table A.16-1: Projected Time Line
Project Time Schedule |
|
Activity |
Time Schedule |
Recruit Participants |
0 to 5 months after OMB approval |
Start Gathering Information |
1 month after OMB approval |
Stop Gathering Information |
8 months after OMB approval |
Analysis of Final Results |
9 months after OMB approval |
Completion of Report and Sending to NIDA |
12 months after OMB approval |
Project End Date and Website Completed |
12 months after OMB approval |
Submission for Publication in Peer Reviewed Journal |
13 months after OMB approval |
Publication in a Peer Reviewed Journal |
Approximately 19 months after OMB approval |
All results will be reported to the Project Officer and NIDA as required by the contract. The Principal Investigator may also pursue publication in the scientific literature and/or presentation at relevant conferences after consultation with NIDA.
The major hypotheses for the proposed evaluation are: 1) participants in the intervention group will show higher gains in knowledge (from pre- to post-assessment) compared to participants in the control group (Pre-Program Knowledge and Competency Measure); 2) participants in the intervention group will express more positive attitudes about following current best practice guidelines for SBIRT, as compared to the control group (Participant Attitude Survey); 3) participants in the intervention group will demonstrate more clinical skills as seen in the SOAP note completeness, that is, completeness in summarizing key findings in the case and documenting the treatment plan (Medical Record Patient Encounter Note – SOAP Note), and Modified Interpersonal Skills Inventory scores for their patient interview, as compared to the control group (Learner Self-Assessment Modified Interpersonal Skills Inventory); and 4) participants in the intervention group will retain changes in knowledge and attitudes over the follow-up period of six to nine weeks after using the intervention (Knowledge and Competency Measure).
To assess the effects of the SBIRTTraining.com online CME program on knowledge attitudes, attitude, and clinical skills separate 2 (case groups) X 2 (assessment points) ANOVAs will be used. It is anticipated that the two groups will have similar scores on the pre-assessments, but the intervention group will show greater gains than the control group in knowledge attitude, behavior, and clinical skills scores at the post-assessment point. Comparisons of post-assessment and follow-up mean scores among the intervention group will be conducted to assess retention of observed changes on all axes measured. The test-retest reliability coefficient for the assessment instrument will be established using the control group's pre-assessment 1 and pre-assessment 2 scores. Changes will be assessed by comparing the average scores on a pre-assessment question versus the average on that same item in post-assessment scores.
The OMB expiration date will be displayed on the assessment instrument.
There are no exceptions to the certification statement identified in OMB Form 83-I, item 19, “Certification for Paperwork Reduction Act Submissions.”
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement A for |
| Author | Meghan |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |