Supporting Statement A for
Activities Associated with Developing a Web-based Resource
for Youth about Clinical Research
(NHLBI)
November 24, 2015
Ellen
Rosenberg
Contracting
Officer’s Representative
Division of Cardiovascular
Sciences
National Heart, Lung and Blood Institute
Rockledge II, Room 8217
6701 Rockledge Drive
Tel:
301-594-1376
Email: [email protected]
Table of Contents
A. JUSTIFICATION 4
A.1 Circumstances Making the Collection of Information Necessary 4
A.2 Purpose and Use of the Information Collection 6
A.3 Use of Information Technology and Burden Reduction 9
A.4 Efforts to Identify Duplication and Use of Similar Information 9
A.5 Impact on Small Businesses or Other Small Entities 9
A.6 Consequences of Collecting the Information Less Frequently 10
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 10
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency 10
A.9 Explanation of Any Payment of Gift to Respondents 11
A.10 Assurance of Privacy Provided to Respondents 12
A.11 Justification for Sensitive Questions 13
A.12 Estimates of Hour Burden Including Annualized Hourly Costs 14
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record
keepers 16
A.14 Annualized Cost to the Federal Government 16
A.15 Explanation for Program Changes or Adjustments 17
A.16 Plans for Tabulation and Publication and Project Time Schedule 17
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 19
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 19
List of Attachments:
Attachment A: Individual Interview Flyer
Attachment B: Individual Interview Questions (Feature Stories)
Attachment C: Individual Interview Questions (Family Spotlights)
Attachment D: Individual Interview Questions (Comic Book)
Attachment E: Individual Interview Parent Permission Form
Attachment F: Individual Interview Child Assent Form
Attachment G: One-to-One Evaluation Study Flyer
Attachment H: One-to-One Evaluation Study Questionnaire
Attachment I: One-to-One Evaluation Study Parent Permission Form
Attachment J: One-to-One Evaluation Study Child Assent Form
Attachment K: Pre-Post Feedback Study Flyer
Attachment L: Pre-Post Feedback Study Questionnaire
Attachment M: Pre-Post Feedback Study Parent Permission Form
Attachment N: Pre-Post Feedback Study Child Assent Form
Attachment O: IRB Proposal
Attachment P: Web-Based Resource Screenshots (Feature Stories)
Attachment Q: Web-Based Resource Screenshots (Comic Book)
Attachment R: IRB Approval Page with Signature
A.1 Circumstances Making the Collection of Information Necessary
The National Heart, Lung and Blood Institute is represented under the legal authority U.S. Code Title 42, Chapter 6A, Subchapter III, Specific Provisions Respecting National Research Institutes, the Public Health and Welfare. The primary purpose of the Institute is to conduct and support research, training, health information dissemination and other programs with respect to heart, blood vessel, lung and blood diseases with respect to the use of blood and blood products and the management of blood resources. The National Heart, Lung and Blood Institute’s mission is to “provide global leadership for a research, training, and education programs to promote the prevention and treatment of heart, lung, and blood diseases and enhance the health of all individuals so that they can live longer and more fulfilling lives”.
The benefits and necessities for this particular research on pediatric clinical trials are congruent with NHLBI’s research goals and mission statement: attempting to assist in the enhancement of the health of individuals so that they can live longer and more fulfilling lives. The current lack of knowledge surrounding pediatric clinical trials can be dangerous and unhealthy towards the lives of youth, becoming a large public health need. The primary purpose of the proposed project is the development of a comprehensive web-based resource for youth with chronic illnesses or diseases designed to increase knowledge, self-efficacy, and positive attitudes towards participation in various clinical trials and research.
The knowledge gained from developing and testing this web-based resource will used to create a resource designed to help equip youth to make informed decisions about clinical research and increase motivation to participate in that research. In addition, the knowledge gained will be invaluable to the field of clinical research given the need for more clinical trials with youth.
There is a need for more clinical trials with youth. In fact, in a review of clinical trials conducted between 2007 and 2010, only 17.4% of the clinical trials were conducted with youth less than 18 years of age (Califf et al., 2012). In addition, “it is recognized that the quantity, quality, and relevance of data involving youth are substantially lower than for adults….” (p. 112, Hartling et al., 2012). This proposes a serious problem due to the fact that without more pediatric clinical trials, youth may be provided with drugs or treatments that have not been tested in clinical trials or are based on research findings with adults (Caldwell, Murphy, Butow, & Craig, 2004; Hartling et al., 2012). This is not only inappropriate but also dangerous given that youth may respond to treatment differently than adults (Caldwell et al., 2004; Smyth & Weindling, 1999). Thus, more clinical trials with youth are needed to enhance the quality of youth’s health care as a whole.
In order to better understand the importance of clinical trials, it is imperative to identify the benefits to youth and their families, as well as future generations of youth. Such benefits to youth may include: direct access to and benefits from new treatments that are not yet available, better medical care, and close monitoring of health (Caldwell et al., 2004; Morales-Olivas & Morales-Carpi, 2006). Those who participate in clinical trials (including those in the treatment and control groups) are less likely to experience complications or clinical events and have lowered mortality rates compared to those who do not participate (Caldwell, et al., 2004). Parents and youth may also have their own perceptions of the benefits associated with participation in clinical trials. Both parents and youth tend to perceive helping others with similar diseases or illnesses in the future as a benefit of their participation (Caldwell et al., 2004). Parents also perceive benefits as greater access to new treatments, health-care professionals, and health information, as well as better care for their youth.
There appear to be a number of benefits to participating in pediatric clinical trials; it may be that the rate of pediatric clinical trials is low because many families with youth who have chronic diseases/illnesses are not aware or informed of the benefits of participating in a clinical trial. It is also possible that the perceived costs of participation create barriers to participation, as discussed below. Once developed, the proposed web-based resource will attempt to educate youth in the area of clinical trials with the goal of increased participation in future trials.
Specifically, the proposed web-based resource will be an interactive, multimedia, developmentally appropriate resource for youth to be educated about pediatric clinical trials. The resource will be developed for youth aged 8 to 14 years. The theme of “investigative cyber-reporting” will be used throughout and will include youth making a series of decisions about different aspects of participating in clinical research studies. Youth will be tasked with the responsibility of learning all they can about clinical research trials in order to facilitate their knowledge and decision-making processes. Language typically used in journalism and design elements reminiscent of journalism will be incorporated into the content, design, and layout of the resource. There are three main components that will comprise the web-based resource. These include an interactive leaning module, full length video testimonials, and an electronic comic book.
The benefits and necessities for developing an educational tool designed to increase participation in pediatric clinical trials are congruent with NHLBI’s research goals and mission statement: attempting to assist in the enhancement of the health of individuals so that they can live longer and more fulfilling lives. The current lack of knowledge surrounding pediatric clinical trials can be dangerous and unhealthy towards the lives of youth, becoming a large public health need.
A research team at innovation Research and Training (iRT; a small business based in Durham, NC) will conduct the proposed developmental research. Three sequential studies comprise this developmental work:
2.1 Individual Interview Study
2.2 One-to-One Evaluation Study
2.3 Pre-Post Feedback Study
A.2.1. Individual Interview Study
Purpose
After the creation of the learning module (referred to as feature stories), video testimonials (referred to as family spotlights), and electronic comic book, youth will be interviewed to obtain feedback about the content in each component (Attachments B, C, and D). Working drafts of Feature Story #1 and the electronic comic book are attached as screenshots (Attachments P, Q). We will recruit nine youth who have not yet participated in clinical trials research and are diverse with respect to sex, race, age, gender, ethnicity, and health status to review the product in a series of individual interviews (Attachment A).
Use of Results
The information and knowledge gained through the individual interviews will be used in the further adaption and development of the web-based resource. At the end of the interviews, Drs. Parker and Scull will review the interview recordings and create a list of edits that need to be made to the content of the feature stories, family spotlights, and comic book based upon this feedback. All three prototypes will then be revised prior to undergoing alpha testing at IRT. The addition of this step allows for creating revisions to the content before the final product is overly committed.
A.2.2. One-to-One Evaluation Study
Purpose
The purpose of the One-to-One Evaluation Study with youth participants is to inform the web-based resource development process by providing opportunities to observe the use of the resource in live situations; obtain feedback from youth about their ease of use of and satisfaction with the resource; and to identify prominent errors with respect to the user interface and instructional systems. Five youth participants will be recruited (Attachment G) and asked to complete the web-based resource and then complete a questionnaire (Attachment H).
Use of Results
This information gathered in the One-to-One Evaluation Study will be used to further edit the web-based resource in an attempt for optimal development. An indication of the efficiency of the web-based resource will be evaluated through comparing participants’ length of time to complete each benchmark task, tallying the number of errors observed, and frequency of use of the Help menu. In addition, an indication of the effectiveness of the web-based resource will be evaluated based upon the percent of benchmark tasks completed. Scores for consumer satisfaction constructs (e.g., format, overall quality, effectiveness) will be calculated for each participant and descriptive statistics will be examined. Usability and satisfaction with the resource will be determined by examining if satisfaction scores are 3.5 above or for 80% or more of the scales. Open-ended responses will also be examined to obtain recommendations about specific areas for revisions to the web-based resource. This study will provide information on what works or doesn’t work in the resource, useful for guiding revisions to the resource.
A.2.3. Pre-Post Feedback Study
Purpose
The Pre-Post Feedback Study will provide insight into the overall feasibility of the web-based resource. Thirty five youth participants will be recruited (Attachment K), will review the web-based resource, and will be asked to complete pre- and post-test questionnaires that will assess their knowledge, attitudes, and self- efficacy as well as their interest in, engagement in, and satisfaction with the web-based resource (Attachment L).
Use of Results
The findings from this study will inform revisions and provide a solid foundation for proposing future developments and assessing the technical feasibility of the web-based resource. Analyses will examine the effects of the web-based resource on attitudes, cognitions, and behaviors, which will provide information on feasibility. The data compiled from the Pre-Post Feedback Study will also indicate the likely effectiveness of the web-based resource and identify areas for improvement. Also, if the youth participants are found to have gained in knowledge from pretest to posttest, it will indicate that the information contained in the web-based resource is sound. Finally, ratings and open-ended feedback from youth on the consumer satisfaction scales will pinpoint areas of the resource that have been perceived as exceptional as well as area in need of attention, which enables an iterative process of program development.
Youth will review parts of the web-based resource (see Appendices P and Q for screenshots of the web-based resource draft) and respond aloud to interview questions in the Individual Interview Study, and their responses will be audio-recorded. During the One-to-One Evaluation Study, youth will review the web-based resource (see Appendices P and Q for screenshots) and then complete an anonymous paper-based questionnaire. During the Pre-Post Feedback Study, youth will be asked to complete an anonymous paper-based questionnaire before completing the web-based resource (see Appendices P and Q for screenshots). After completing the web-based resource, the youth will be asked to fill out another anonymous paper-based questionnaire. These two questionnaires will be administered during the same data collection session, therefore not requiring a linking list, ensuring complete anonymity. A Privacy Impact Assessment (PIA) is currently underway.
The content and format of the proposed web-based resource is based on empirical literature on pediatric clinical trials as well as instructional design theory. This framework will guide the development of the proposed web-based resource. At the end of Phase I, it is hypothesized that youth who have completed the web-based resource will be more knowledgeable about clinical research, more self-efficacious about making decisions related to participating in clinical research, and has more positive attitudes about participating in clinical research. To our knowledge, no one else is collecting the same feedback data on this web-based resource. Because the proposed web-based resource and studies will be analyzing measures that have never been investigated before, there are no existing data that could serve the purpose of evaluating the efficacy of this web-based resource at this time.
No small businesses will be involved in this study.
This study contains three separate single-time research studies in which participants are interviewed or asked to complete questionnaires before and/or after using the web-based resource. Both the One-to One Evaluation and Individual Interview studies will provide immediate feedback to the research team based on youth opinions about the web-based resource. In addition, the Pre-Post Feedback Study will provide immediate feedback given that data collection will take place before and after youth review in the web-based resource (during the same data collection session). These study designs minimize the burden on participants and reduce the likelihood of attrition (compared with longer respondent re-contact intervals), while also allowing the research study to assess immediate feedback based on exposure to the web-based resource. In addition to this, the proposed studies are recruiting a specific and affected population; therefore lessening burden is increasingly critical.
The proposed data collection includes three major studies in which a total of 48 youth who have chronic diseases/disorders with no prior clinical research experience will be recruited to provide feedback on the web-based resource. Respondents’ participation in the data collection will take place within a four-month period, highlighted in the proposed timeline. The Individual Interview Study will take place between Months 7-8 of the proposed study schedule. The remaining two studies (One-to-One Evaluation Study and Pre-Post Feedback Study) will occur within the final four months of the proposed timeline. Response times for the Individual Interview Study will take up to 2 hours, the One-to-One Evaluation Study will take up to 2 hours, and the Pre-Post Feedback Study will take up to 4 hours.
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This data collection fully complies with 5 C.F.R. 1320.5.
The 60-day Notice was published in the Federal Register on March 12, 2015, Document Citation: 80 FR 13013, pp. 13013-13014. One comment was received. The respondent requested a chance to see the study materials and perhaps share them with their organization members (see attached). In response, the study materials were sent to the Contracting Officer’s Representative on this project, Dr. Ellen Rosenberg, who then forwarded the materials to the respondent. No further requests or comments were made during the remainder of the 60-day commenting period.
Additional consultation on the content and design of the web-based resource will be received from (1) Dr. Julie Thompson, a Pediatric Researcher in the School of Nursing at Duke University, (2) Dr. Vandana Shashi, an Associate Professor of Pediatrics at the Duke University School of Medicine, (3) Dr. Sarah Helms, a Postdoctoral Fellow at University of North Carolina-Chapel Hill, and (4) Dr. Mahnaz Moallem, a Professor of Instructional Technology at University of North Carolina-Wilmington.
Individual Interview Study
The interviews will take up to two hours each and participants will receive $25.
One-to-One Evaluation Study
The one-to-one evaluations will last up to 2 hours each and participants will receive $25.
Pre-Post Feedback Study
This study will take up to 4 hours and participants will receive $50.
Difficulty in participant recruitment and retention is anticipated. Therefore, the choice to use the proposed incentives was made with the following considerations in mind:
Voluntary nature of data collection: Youth are not required to participate in the data collection for these studies; rather, their participation is voluntary. Based on experiences with past studies within iRT, data collectors have determined that both adults and youth appreciated remuneration and incentives, which in turn increased their willingness to participate in data collection, their responsiveness when scheduling data collection times, and their focus and attention when participating in the actual data collection.
Difficulties in reaching and retaining a sample from a very small pool of participants: We anticipate that the proposed sample of participants will be difficult to reach and retain for these studies. To be eligible for these studies, youth (between ages of 8-14) must have a chronic illness, disorder, or disease. In addition, we are recruiting a diverse sample with respect to gender, race, and ethnicity. Finally, only youth who have already participated in a clinical trial will be eligible to participate. Thus, the pool of participants for these studies is very small, thereby increasing the difficulty with which to recruit participants.
Extreme burden on the respondent: Participation in these studies is anticipated to be very burdensome for this particular population of youth and their caregivers. First, we are recruiting children (aged 8 to 14) to participate in the study, which means that a caregiver must not only provide permission but also accompany the child to the data collection site. This will require the time of both the youth and the caregiver; thus, burdening two people rather than one. Second, we are recruiting children who have a chronic disease or illness. Family resources may already be overextended because of doctor appointments, medical procedures, or missing school or work. Participation in a study would place additional burden on these resources. Third, because the data collection is voluntary and administered in an informal setting, participating in the data collection may be perceived as an atypical activity for youth and parents, thus placing undue burden on the respondents. Therefore, providing incentives for participation will likely ease some of the burden on these participants.
Cost-savings to the government: Given the level of burden for these youth and their family members as well as the small pool of participants from which to recruit, as described above, the incentives must be compelling enough ensure adequate study sample sizes. If the incentive is not perceived to be worth the time of participants, particularly given the burden, it will make it very difficult to recruit the full sample. This, in turn, may lead to the need for additional study resources (i.e., time and money) to locate youth and caregivers willing to donate their time. Therefore, the small cost of the proposed incentives will save funds on this project in the long-term given that excessive resources (i.e., time and money) would not likely be required to recruit the full samples.
The data collected in the proposed studies will be anonymous, without any linking list or potential for the identities of the participants to be revealed. Every effort will be made to protect participants’ privacy, and the permission and assent forms will indicate this to participants. Participants will be advised of the procedures in place to maintain their privacy. Participants will be told that their answers will be kept private, securely stored in locked file cabinets and/or on password protected servers, and only accessible to research staff members who work on this project.
The research design and protocol (see IRB proposal; Attachment O) have been approved by the iRT Institutional Review Board (Attachment R).
Sensitive Questions
In the proposed project, there are no specific questions that are considered sensitive. The data collected across the studies will provide a foundation for assessing the technical feasibility of the web-based resource. First, the data compiled on the successful completion of the web-based resource will indicate the usability and feasibility of the web-based resource and identify areas for improvement. Second, if youth participants are found to have gained in knowledge from pretest to posttest, it will indicate that the information contained in the web-based resource is sound. Third, ratings and open-ended feedback from youth on the consumer satisfaction scales will pinpoint areas of the resource that have been perceived as exceptional as well as area in need of attention, which enables an iterative process of program development. These data will tell us if the resource is engaging, interesting, and effective for youth. Based upon the results of this study, the PD Team will evaluate which basic revisions need to be made so that the resource and its contents are comprehensive, appropriate, and reflect the areas requiring modifications to improve its feasibility. The final product will be a functional web-based resource for youth to learn about pediatric clinical research.
Personally Identifiable Information
This project uses anonymous audio-recordings and paper-based questionnaires, maximizing the protections to respondents’ privacy. For all data collected and analyzed as part of the studies, unique, randomly-selected identification numbers will be used in place of names to provide Privacy to participants. Participants’ contact information will be collected with the sole purpose of providing youth with the incentives for study participation and will be kept separate from the study data. No other personally identifiable information will be collected as part of this study. Participants’ de-identified responses will be stored separately from their contact information and consent forms. Permission and assent forms will be stored at the iRT office in a locked file cabinet that can only be accessed by project staff.
Consent Process and Privacy of Data
All participants will undergo informed consent procedures prior to data collection. Parents will provide permission for their child’s participation (Attachments E, I, and M). Youth will provide written assent (Attachments F, J, and N). Through the permission and assent process, all participants will be informed about the nature of the study and reminded that their participation is entirely voluntary. They will be informed about the possible benefits and consequences of participation. They will also be reminded that during data collection, they can skip any question they do not wish to answer for any reason and without consequences, and can also leave the study without consequence. Participants will be told that their answers will be kept anonymous, securely stored in locked file cabinets and/or on password protected servers, only accessible to research staff members who work on this project.
A12 – 1 Estimates of Hour Burden
Form Name |
Type of Respondents |
Number of Respondents |
Number of Responses per Respondent |
Average Burden Per Response (in hours) |
Total Annual Burden Hour |
Individual Interview Parent Permission Form |
parents |
9 |
1 |
5/60 |
1 |
One-to-One Evaluation Study Parent Permission Form |
parents |
5 |
1 |
5/60 |
0.42 |
Pre-Post Feedback Study Parent Permission |
parents |
34 |
1 |
5/60 |
3 |
Individual Interview Child Assent Form |
youth |
9 |
1 |
5/60 |
1 |
One-to-One Evaluation Study Child Assent Form |
youth |
5 |
1 |
5/60 |
0.42 |
Pre-Post Feedback Study Child Assent Form |
youth |
34 |
1 |
5/60 |
3 |
Individual Interview Questions (Feature Stories) |
youth |
3 |
1 |
2 |
6 |
Individual Interview Questions (Family Spotlights) |
youth |
3 |
1 |
2 |
6 |
Individual Interview Questions (Comic Book) |
youth |
3 |
1 |
2 |
6 |
One-to-One Evaluation Study Questionnaire |
youth |
5 |
1 |
2 |
10 |
Pre-Post Feedback Study Questionnaire |
youth |
34 |
1 |
4 |
136 |
Participants in all three studies will include youth (ranging in age from 8 to 14 years) with varying chronic illnesses, disease, or disorders who have not yet participated in a clinical trial.
For all three studies, parents will review and sign a parent permission form (Attachments E, I, M). This is expected to take up to 5 minutes. In addition, for all three studies, children will review and sign a child assent form (Attachments F, J, N). This is expected to take up to 5 minutes.
For the Individual Interview Study, youth will be recruited to participate (Attachment A). There are three separate parts to this study. First, three youth will be asked to review the Feature Stories and then will respond to a number of interview questions as led by a trained research team member (Attachment B). The review of the Feature Stories and interview is expected to take up to 2 hours. The project burden for this part of the study is 6 hours. Second, three youth will be asked to review the Family Spotlights and then will respond to a number of interview questions as led by a trained research team member (Attachment C). The review of the Family Spotlights and interview is expected to take up to 2 hours. The project burden for this part of the study is 6 hours. Third, three youth will be asked to review the Comic Book and then will respond to a number of interview questions as led by a trained research team member (Attachment D). The review of the Comic Book and interview is expected to take up to 2 hours. The project burden for this part of the study is 6 hours.
For the One-to-One Evaluation Study, five youth will be recruited to participate (Attachment G). Youth will be asked to complete the web-based resource and then will respond to a questionnaire pertaining to the usability of and satisfaction with the web-based resource. This will be led by a trained research team member. The one-to-one evaluations are expected to take up to 2 hours. The project burden for this study is 10 hours.
For the Pre-Post Feedback Study, 34 youth will be recruited to participate (Attachment K). Youth will be asked to complete a pre-test questionnaire (1 hour) and then complete the web-based resource (2 hours). Youth will then be asked to complete a post-test questionnaire and a consumer satisfaction questionnaire (1 hour). The Pre-Post Feedback Study is expected to last up to 4 hours. The project burden for this study is 136 hours.
The total project burden for the three studies is 172 hours of participant time.
A12 – 2 Annualized Cost to Respondents
Data are from the Bureau of Labor Statistics (http://www.bls.gov/cps/cpsaat39.htm). Participants include children aged 8 to 14 years. Hourly wage rates for participants were calculated at zero due to the fact that participants will receive an incentive for their participation at a rate of $25.00 for individual interviews, $25.00 for the one-to-one evaluations, and $50.00 for the pre-post feedback. The data collection appointment times will be flexible so that parents/caregivers can choose times outside of their working hours (e.g., in the afternoons, evenings, and weekends) to bring the youth participants to the study site, so there will be no loss of wages. Total projected cost to respondents is $0.00.
Type of Respondent |
Number of respondents |
Frequency of response |
Average time per response |
Hourly wage rate |
Respondent cost |
Child: Individual Interview Study |
9 |
1 |
2 |
$0.00 |
$0.00 |
Child: One-to-One Evaluation Study |
5 |
1 |
2 |
$0.00 |
$0.00 |
Child: Pre-Post Feedback Study |
34 |
1 |
4 |
$0.00 |
$0.00 |
Totals |
48 unique youth |
|
|
|
$0.00 |
There are no capital, start-up, operational, or maintenance costs to the respondents in providing the information required by this research.
The annual cost to the Federal Government is $98,561. This includes 10% of the annual full-time effort of a NHLBI Contracting Officer (estimated at $13,326) and 10% of a NHLBI Contracting Officer’s Representative (estimated at $8,236). This also includes the contractor cost for innovation Research and Training (estimated at $74,949) for the development of a web-based resource for educating youth about pediatric clinical trials. Other costs include incentives for study participants (estimated at $2,050).
Table A.14.1
Staff |
Grade/Step |
Salary |
% of Effort |
Fringe (if applicable) |
Total Cost to Gov’t |
Federal Oversight |
|
|
|
|
|
Contracting Officer |
14-5 |
133,264 |
10 |
|
13,326 |
Contracting Officer’s Representative |
12-5 |
82,359 |
10 |
|
8,236 |
|
|
|
|
|
|
Contractor Cost |
|
53,920 |
|
21,029 |
74,949 |
|
|
|
|
|
|
Travel |
|
|
|
|
|
Other Cost |
|
|
|
|
2,050 |
|
|
|
|
|
|
Total |
|
|
|
|
98,561 |
This is a new collection of information.
Outlined below is a proposed schedule of major events over the 12-month timeline.
|
Monthly Timeline |
|||||
Obj. 1: Develop content for web-based resource |
1-2 |
3-4 |
5-6 |
7-8 |
9-10 |
11-12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Obj. 2: Production and implementation of the web-based resource |
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(Study 1) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Obj. 3: Collect feasibility data on the web-based resource |
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(Study 3) |
|
|
|
|
|
|
|
|
|
|
|
|
|
Analyses.
After the development of the web-based resource, staff will continue to evaluate and analyze the results of the following studies:
Individual Interview Study. The information and knowledge gained through the individual interviews will be used in the further adaption and development of the web-based resource. At the end of the interviews, Drs. Parker and Scull will review the interview recordings and create a list of edits that need to be made to the content of the feature stories, family spotlights, and comic book based upon this feedback. All three prototypes will then be revised prior to undergoing alpha testing at IRT. The addition of this step allows for creating revisions to the content before the final product is overly committed.
One-to-One Evaluation Study. An indication of the efficiency of the web-based resource will be evaluated through examining scores on time it take youth to complete each benchmark task and tallying the number of errors observed. In addition, an indication of the effectiveness of the web-based resource will be evaluated based upon the percent of benchmark tasks completed. Scores for constructs related to consumer satisfaction (e.g., format, overall quality, effectiveness) will be calculated for each participant and descriptive statistics will be examined. Usability and satisfaction with the resource will be determined by examining if satisfaction scores are 3.5 above or for 80% or more of the scales. Open-ended responses will also be examined to obtain recommendations about specific areas for revisions to the web-based resource.
Pre-Post Feedback Study. One-way repeated measures ANOVAs will be conducted to examine change in youth’s knowledge, attitudes, and self-efficacy from pretest to posttest. Mean scores for consumer satisfaction constructs (e.g., format, overall quality, effectiveness) will be calculated and descriptive statistics will be examined. Usability and satisfaction with the resource will be determined by examining if mean satisfaction scores are 3.5 above or for 80% or more of the scales. Open-ended responses will also be examined to obtain recommendations about specific areas for revisions to the web-based resource.
Conclusions. The findings from these studies will provide a foundation for assessing the technical feasibility of the web-based resource. First, the data compiled on the successful completion of the benchmark tasks will provide an indication of the effectiveness of the web-based resource and identify areas for improvement. Second, if youth participants are found to have gained in knowledge, it will provide an indication that the information contained in the web-based resource is sound. Third, ratings and open-ended feedback from youth on the consumer satisfaction scales will pinpoint areas of the resource that have been perceived as exceptional as well as area in need of attention, which enables an iterative process of program development. Consumer satisfaction scores will provide an indication of whether the resource is engaging, interesting, and effective for youth.
Based upon the results of this study, the PD Team will evaluate which basic revisions need to be made so that the resource and its contents are comprehensive, appropriate, and reflect the areas requiring modifications to improve its feasibility. The final product will be a functional web-based resource for youth to learn about pediatric clinical research. The development of the product and results of four studies are expected to be completed by the end of the twelve-month project period (August 2015).
The OMB expiration date will be displayed on all assessment instruments.
There are no exceptions to the certification statement identified in OMB Form 83-I, item 19,
“Certification for Paperwork Reduction Act Submissions.”
| File Type | application/msword |
| File Title | Update from the site, March 2009 |
| Author | dealmeig |
| Last Modified By | Currie, Mikia (NIH/OD) [E] |
| File Modified | 2015-11-24 |
| File Created | 2015-11-24 |