Supporting Statement Part A -- Connecting PCP with Hard to Reach Adolescent Pop 9-12-2011

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

research that develops and presents scientific evidence regarding all aspects of health care; and
the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

The overall goal of this project is to explore the potential for emerging information technologies to improve the quality of adolescent health care. The project will address suboptimal adolescent care with respect to health risk behaviors, which can have serious health consequences. In particular, failure to address health risk behaviors among adolescents (e.g., smoking, substance abuse, poor diets, physical inactivity, and high-risk sexual behavior) contributes significantly to increased morbidity and mortality. Adolescents (11-17 years of age) constitute 17% of the population of the U.S., but they are responsible for only 7% of medical office visits. As a result, primary care providers have relatively less opportunity to evaluate and counsel adolescents in their offices than most other patients. Even when adolescents receive routine health care, open communication with their health care providers may be problematic. A national survey found that the majority of adolescent boys and girls in the U.S. report at least 1 of 8 potential health risks, but most (63%) had not spoken to their doctor about any of these . Improved engagement and communication between adolescents and their primary care providers could increase the likelihood that effective preventive services and health care are provided. It could also improve the efficiency of health care services for adolescents, in terms of appointments kept and adherence to recommended screening or treatment recommendations.

Technological interventions to improve care may be particularly appropriate for adolescents, since they are typically the early adopters of new technology . Use of in-office electronic screeners before appointments has proven useful . Outside of the office, youth have increasingly turned to the internet for health-related information, and have also rapidly adopted mobile technology and social media . Health plans (e.g., Kaiser Permanente) and practices have conducted early work in applying patient-centered web and mobile technologies. These projects have included interventions to decrease patient no-show rates, increase the use of sunscreen, and engage adolescents in diabetes management. Much work remains to be done, however, in understanding how primary care practices can best embrace advances in communications and information technology to improve health outcomes for adolescent patients.

This study is being conducted by AHRQ through its contractor, State Network of Colorado Ambulatory Practices and Partners (SNOCAP-USA), a practice-based research network (PBRN) based at the University of Colorado Denver, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to clinical practice, including primary care and practice-oriented research. 42 U.S.C. 299a(a)(1) and (4).

2. Purpose and Use of Information

The results from this exploratory project will be used to inform development of a manual to assist primary care practices in adopting interventions to improve the effectiveness of their outreach to and interactions with adolescent patients. Specifically, the project will explore potential improvements in adolescent care made possible by two complementary technologies, one outside the office and one in the office:

Engaging adolescents outside of the office: Clinics will engage with the adolescent community at large using a set of web, social media, and mobile technologies, including (i) a web page (prototype: https://sites.google.com/site/coloradoclinicsforyouth) for more static content such as information about practices and health-related commentary from practice clinicians and staff, (ii) a Facebook page (prototype: http://www.facebook.com/pages/Colorado-Clinics-For-Youth/118475374871964) for social interaction about health topics including topical content that will engage adolescents in conversations about general, not personal, health behaviors and encouraging youth to discuss these issues with their primary care practitioners at clinic visits, and (iii) a Twitter site (prototype: http://twitter.com/CoClinics4Youth) that will allow youth to use mobile phones with text messaging to subscribe to Facebook posts.
Engaging adolescents in the office: In the waiting room before scheduled appointments, clinics will have adolescent patients complete the Rapid Assessment for Adolescent Preventive Services © (RAAPS) questionnaire using a web-based interface licensed from the University of Michigan. RAAPS was developed by the University of Michigan Regional Alliance for Healthy Schools to improve clinical recognition of adolescent behaviors that should be addressed, but often are missed, in primary care so that personal counseling may be provided. The utility of RAAPS has been formally reported in the academic literature, and it is currently in use in over 30 health centers .

For this project, these technologies will be adopted by four primary care practice sites that have a substantial number of adolescent patients. Evaluation of practical issues and outcomes will include assessments of:

Lessons learned by clinic managers, staff, and clinicians on how to optimize the effective and efficient use of the technologies in practice,
The perceived utility of the technologies by adolescents,
Whether adoption of the technologies may increase in adolescent visits to primary care practices: operationally, whether adoption is associated with an increase in the proportion of visits to the practice that are for adolescent patients,
Whether adoption of these technologies may increase the identification of health risks during visits, and
A nested adolescent behavior and communication survey to determine whether the period of technology adoption is associated with improvements in doctor-patient communication and health behavior in four domains: (1) diet, (2) physical activity, (3) substance abuse (smoking, alcohol, and use of other recreational drugs), and (4) sexual health.

Table A-1 lists the activities and data collections to be conducted and their purposes.

Table A-1: Activities and Data Collections and their Purposes

Data collection	Technology Intervention	Assessment
Data collection	Technology Intervention	Process Measure	Lessons learned	Perceived utility	Increased visits	Increased identified risks	Improved health behavior
1. RAAPS questionnaire	X
2. Process measures for web technologies		X
3. Extraction of medical record data					X	X	X
4. Consent-assent form							X
5. Adolescent behavior and communication survey							X
6. Post-visit satisfaction survey				X	X	X
7. Adolescent focus groups	X
8. Adolescent “think aloud” sessions				X
9. Clinician semi-structured interviews			X
10. Administrator-staff semi-structured interviews.			X
11. Semi-structured interviews for the draft manual			X

Of note, all of these activities and data collections have successfully completed review by the Colorado Multiple Institutional Review Board (Protocol #10-1322), including review of their compliance with the Health Insurance Portability and Accountability Act (HIPAA). [PIA has been completed]. Specifically, these activities and data collections consist of the following:

RAAPS questionnaire. Practices will use the 21-item RAAPS questionnaire for in-office pre-visit screening (see Attachment B for a paper version of the screening questions). Data collected as part of this screening is for clinical care and operations. Practices will use the RAAPS data from individual patients to augment communication during clinic visits, and can also, at their discretion, create practice-wide reports on the health behaviors reported by their adolescent patients.
Process measures for web technologies. For each of the web technologies used (the web page, Facebook page, and Twitter site), data on the number of unique visitors, the frequency of their visits, and their activities (e.g. whether they create a new post or “like” postings) will be obtained by the research team. These data will not include personally identifiable information (e.g. the user’s username, birth date, IP address, etc.). OMB clearance is not required for this data collection.
Extraction of electronic practice and medical record data. Staff members at each practice will use their clinical information systems to extract data for use by the research team. Data to be extracted consist of

The number of unique patients seen by the practice (total and for patients aged 13-18) in the 12 months prior to the implementation start date and the 12 months following the implementation start date. These data will be used to assess whether the intervention increased the number and proportion of adolescent visits in the practices.
Clinic notes for adolescents seen in the 12 months prior to implementation start date and for adolescents seen in the 12 months following the implementation start date. Clinic notes will be made accessible by printing notes from electronic medical records, which are used by all four practices participating in this project. The notes will be reviewed and abstracted by the research team to assess whether the intervention had the intended effect of increasing adolescent visits to primary care and the identification of potential health risks during visits. Data to be extracted are described in Attachment C.
Contact information for patients seen in the 18 months prior to the start date for implementation of RAAPS and the web technologies. These data will be used by the project staff to prepare mailings to recruit patients for the adolescent behavior and communication survey.

Consent-assent form. This is used to obtain consent from the parent or guardian and assent from the adolescent to participate in the adolescent behavior and communication survey. Of note, these forms have already completed the review process by the Colorado Multiple Institutional Review Board (see Attachments D and E).
Adolescent behavior and communication survey. For adolescent patients (and parents/guardians, where applicable) who provide consent-assent, a questionnaire (by mail, with an online option) will be administered twice: once at baseline and again six months after the intervention (see Attachments F and G). The purpose of this survey is to measure the adolescent’s level of comfort with discussing their health with their clinician, their level of satisfaction with their medical care, and their self-reported health behaviors, to see how this changes after the intervention.
Post-visit satisfaction survey. Practices will provide adolescents with a brief, post-card sized anonymous questionnaire at every office visit during the study period (see Attachment H). The purpose is to assess the perceived utility of the RAAPS questionnaire, and whether the visit was related to the project’s web technologies.
Adolescent focus groups. Eight adolescents (two from each practice) will provide feedback on the web page, Facebook, and Twitter pages. There will be one in-person group meeting pre-implementation, followed by a series of 3 additional asynchronous group discussions conducted via the web at three-month intervals (see Attachment I). These provide a process for user-centered design and refinement of the of web technologies.
Adolescent “think-aloud” sessions. These sessions, which will be conducted near the end of the study period, will involve a set of eight adolescent patients (two from each practice) that did not participate in the focus groups. Subjects will come to the practice for individual sessions in which they will be asked to say aloud what they are thinking about the web technologies as they navigate them as they typically would. The purpose is to assess the perceived utility of the components of the web, Facebook, and Twitter pages (see Attachment J).
Clinician semi-structured interviews. At each site, individual interviews will be conducted with two clinicians (eight clinicians total). The purpose is to assess clinician perceptions of the effects of the RAAPS questionnaire and the web technologies on the clinical encounter and the care they provide (see Attachment K).
Administrator-staff semi-structured interviews. At each site, semi-structured interviews will be conducted with the practice manager and a front-desk staff member. The purpose is to assess the effect of the interventions on the check in process and other business processes (see Attachment L).
Semi-structured interviews for the draft manual. The draft manual of best practices in primary care for adoption of web and assessment technologies (such as the RAAPS questionnaire) developed by the research team will be sent to the practice manager and the practice director (lead clinician) of each site. Their feedback will be solicited by telephone roughly two weeks later. This “member checking” enhances the validity of the manual’s conclusions and recommendations (see Attachment M).

3. Use of Improved Information Technology

Information technology will be used to reduce the burden of quantitative data collection wherever possible:

Appointment information (before and after the intervention period) will be obtained using practice information systems.
Adolescents will be given the option of completing the adolescent behavior and communication survey online (using REDCap Survey) in addition to a paper version. The paper versions are shown in Attachments F and G; the electronic version is in development.
We will use the electronic version of RAAPS for screening
Process data for out-of-office interventions are available through electronic tools (e.g. Google Analytics and Facebook Insights)

4. Efforts to Identify Duplication

A thorough review of all the extant literature was conducted prior to development of this study. The results of that literature review are documented in Section A.1. No studies with a similar combination of objectives, design, setting and study participants were found. Hence, the proposed study is not duplicative.

5. Impact on Small Businesses or Other Small Entities

Participating practices are likely to benefit from assistance in adopting RAAPS, which has proven to be a useful clinical tool to improve adolescent care in multiple settings. The electronic version provides tools that allow practices easily to produce reports on the patient and practice/population level.

Practices will distribute the post-visit satisfaction survey at the same time that they assist patients in completing the RAAPS questionnaire. The postcard will utilize “business reply mail,” with SNOCAP-USA paying postage for those mailed postcards.

Practices will assist the SNOCAP-USA research team in chart review by pulling charts (in the case of paper medical records) or printing notes (in the case of electronic medical records).

Interviews with key informants will be brief, focusing on the issues most relevant to each informant.

6. Consequences of Collecting the Information Less Frequently

Overall, this is a onetime pre-post information collection, and there are no plans to repeat it.

Use of a pre-post design (vs. a single cross-sectional sample) is necessary to allow assessment of changes in behaviors and behavioral mediators related to the interventions. This will also allow this project to use the same outcome measure (six-month change in self-reported behavior) as used in the previous study of a screener alone , enabling inferences to be drawn about the incremental benefit of adding out-of-office outreach/engagement to the in-office screener.

7. Special Circumstances

There are no special circumstances related to the guidelines of 5 CFR 1320.5(d)(2).

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. Federal Register Notice

As required by 5 CFR 1320.8(d), a notice was published in the Federal Register on January 13^th, 2011 for 60 days (see Attachment N). One non-substantive comment was received.

B. Outside Consultation

Miriam Dickinson PhD, a biostatistician from the University of Colorado Department of Family Medicine, was consulted on statistical issues.

9. Payments/ Gifts to Respondents

Several payments are being made in these data collection efforts:

This project is part of AHRQ’s Practice-Based Research Network (PBRN) initiative. The PBRNs provide AHRQ with a unique infrastructure for conducting research in primary care settings. With the support of AHRQ for the past decade, they have developed the capacity and experience to successfully conduct this type of research. Many of our programs that focus on quality improvements in medical practices do not routinely offer respondent incentives. However, because this project focuses on primary care, and in fact on primary care in pediatric/adolescent practices, where the level of competing demands is very high, we believe an incentive is appropriate. Specifically, while practices participating in this type of quality improvement research do derive some benefit from the experience, it is also true that primary care practices, in particular, have limited resources for exactly the type of research that may improve the quality of medical care for Americans. In addition to the burden on the practices associated with extracting medical chart data there is in indirect burden on the practice due to the presence of the project staff in a busy medical office setting, and the need to arrange for the conduct of many interviews with adolescent patients. As such AHRQ proposes a practice-level incentive of $500, rather than an incentive to physicians, other practitioners or office staff who are being asked to complete questionnaires.
For adolescent survey participants, a gift of $10 will be attached to the baseline survey, and a gift of $20 will be attached to the six-month follow-up survey in the form of gift cards. (To allow auditing, the University of Colorado strongly favors the use of gift cards for incentives, and will only approve cash incentives in rare circumstances). Use of such an incentive is necessary to achieve an adequate response rate among participants for this project because (1) it includes sensitive questions about health behaviors, and (2) it involves adolescents, a group that is known to present challenges in recruitment and retention. Use of incentives in this manner is recommended by standard texts on survey methods . Similar incentives were employed in the comparable study that is the model for this project .
Adolescents participating in the adolescent focus groups will each receive a $75 incentive for their participation.
Adolescents participating in the adolescent “think aloud” sessions for will each receive a $20 incentive for their participation.

10. Assurance of Confidentiality

Individuals and organizations contacted will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

Confidentiality of the data collected will be maintained as follows:

Adolescent behavior and communication survey:

Surveys will only be sent to participants who have parent/guardian consent.
Surveys will be labeled with study identifiers, and will not include personally identifiable information such as name, medical record number, birth date, or address.
Data from the surveys will be stored in a secure, HIPAA-compliant system (REDCap for abstracted paper surveys, REDCap Survey for surveys completed online).
The study identifiers will be linked to patient identifiers (name and address) in a separate secure database. After all mailings are complete, this file will be erased, removing all connections between the study identifiers and personally identifiable information.

The electronic version of RAAPS uses encryption for transmission of data, and stores data in a secure, HIPAA-compliant manner. Role-based access to RAAPS data is provided to selected practice staff, and only data for that staff member’s practice is accessible.
The post-visit satisfaction survey will be completely anonymous. Abstracted data will be stored in REDCap.

11. Questions of a Sensitive Nature

RAAPS queries patients about sensitive issues in the four domains of interest in this project (diet, physical activity, substance abuse [tobacco, alcohol, and drugs], and sexual health) as well as other important but sensitive issues . These sensitive questions are necessary because addressing these issues in primary care is essential for comprehensive adolescent care , yet these issues often remain unaddressed in traditional care models .

Likewise, the behavioral and communication survey must include sensitive questions in order to assess the degree to which the information technology interventions actually improve adolescent behavior and mediators of that behavior. These domains will primarily be assessed using well-validated questionnaire instruments derived from COMBO (Common Measures, Better Outcomes) , which were used in Olson’s study of the screener alone . For domains not addressed in COMBO (drug use and sexual health) questions derived from CDC’s YRBS (Youth Risk Behavior Survey) will be used. The 2011 YRBS survey is OMB approved (OMB #0920-0493).

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annualized burden hours for the respondents’ time to participate in this research. Among the 776 adolescent patients across the 4 participating practices, 310 are expected to complete the RAAPS questionnaire, which takes about 12 minutes to complete, at each office visit (on average there will be an estimated 1.25 office visits per patient). Practice staff members will take about 5 minutes to register patients in RAAPS and to assist them in completing it. Practice staff members will perform the extraction of medical record data pre-implementation, and again post-implementation, for 50 patients. This task is estimated to require 4 hours per practice (slightly less than 5 minutes per patient record).

The consent-assent form for participation in the adolescent behavior and communication survey will be sent to the homes of all adolescents in the practice’s panels. The estimated average time for reading and responding to the form is 15 minutes. The adolescent behavior and communication survey will be completed twice, pre and post intervention, by 233 adolescent patients and requires 15 minutes to complete. The post-visit satisfaction survey will be completed by each of the 310 participating adolescent patients after each office visit and will take 1 minute to complete.

A series of four focus groups will be held with 8 adolescent patients over the course of the study period with each session lasting about 1.5 hours. In addition to the focus groups one “think aloud” session will be held with a group of 8 adolescent patients and will also take 1.5 hours.

Feedback from the practice staff and the clinicians will be obtained through 3 different semi-structured interviews. Two staff members from each of the 4 practices will participate in these interviews. The clinician and administrator-staff semi-structured interviews will each last 30 minutes. Semi-structured interviews for the draft manual will require about one hour total (30 minutes to review the manual and 30 minutes to participate in the interview). The total annualized burden is estimated to be 479 hours.

Exhibit 2 shows the estimated annualized cost burden associated with the respondents’ time to participate in this research. The total annual cost burden is estimated to be $7,980.

Exhibit 1. Estimated annualized burden hours

Activity/Data Collection	Number of Respondents	Number of Responses per Respondent	Hours per Response	Total Burden Hours
RAAPS questionnaire (patient)	310	1.25	12/60	78
RAAPS questionnaire (practice)	4	97	5/60	32
Extraction of medical record data	4	2	4	32
Adolescent behavior and communication survey consent-assent form	776	1	15/60	194
Adolescent behavior and communication survey	186	2	15/60	93
Post-visit satisfaction survey	310	1.25	1/60	6
Adolescent focus groups	8	4	1.5	48
Adolescent “think-aloud” sessions	8	1	1.5	12
Clinician semi-structured interviews	4	2	30/60	4
Administrator-staff semi-structured interviews	4	2	30/60	4
Semi-structured interviews for the draft manual	4	2	1	8
Total	1,618	na	na	511

Exhibit 2. Estimated annualized cost burden

Activity/Data Collection	Number of Respondents	Total Burden Hours	Average Hourly Wage Rate¹	Total Cost Burden
RAAPS questionnaire (patient)	310	78	$9.01²	$703
RAAPS questionnaire (practice)	4	32	14.34³	$459
Extraction of medical record data	4	32	$18.15⁴	$581
Adolescent behavior and communication survey consent-assent form	776	194	$22.11⁵	$4,289
Adolescent behavior and communication survey	186	93	$9.01²	$838
Post-visit satisfaction survey	310	6	$9.01²	$54
Adolescent focus groups	8	48	$9.01²	$432
Adolescent “think-aloud” sessions	8	12	$9.01²	$108
Clinician semi-structured interviews	4	4	$84.53⁶	$338
Administrator-staff semi-structured interviews	4	4	$29.63⁷	$119
Semi-structured interviews for the draft manual	4	8	$64.75⁸	$518
Total	1,618	511	na	$8,439

¹Mean hourly and wage costs for Colorado were derived from the Bureau of Labor and Statistics National Compensation Survey for May 2009 (http://www.bls.gov/oes/current/oes_co.htm).

²Hourly rate for an entry level worker (occupation code 35-0000) estimates the cost of time for adolescents, although many will not be employed.

³Average of the hourly rates for a receptionist (43-4171) and a medical assistant (31-9092).

⁴Hourly rate for medical records and health information technician (29-2071).

⁵Hourly rate for the mean for all occupations (00-0000) estimates the cost of time for the parent or guardian of the adolescent.

⁶Average of hourly rates for a family medicine practitioner (29-1062) and a general internist (29-1063).

⁷Average of (1) the hourly rate for a medical and health services manager (11-9111) and (2) the average of the hourly rates for a receptionist (43-4171) and a medical assistant (31-9092).

⁸Average of (1) the hourly rate for a medical and health services manager (11-9110) and (2) the average of the hourly rates for a family medicine practitioner (29-1062) and a general internist (29-1063).

13. Estimates of Annualized Respondent and Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. The only cost to the respondent will be that associated with their time to respond to the information collection.

14. Annualized Cost to the Federal Government

Exhibit 3 shows the estimated total and annualized cost to the Federal Government for conducting this research. These estimates include the costs associated with the project such as the preparation of survey administration procedures, labor costs, administrative expenses, costs associated with copying, postage, and telephone expenses, data management and analysis, and preparation of final reports. The annualized and total costs are identical since the data collection period will last for one year. The total cost is estimated to be $436,524.

Exhibit 3. Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Project Development	$72,364	$72,364
Data Collection Activities	$48,904	$48,904
Data Processing and Analysis	$73,937	$73,937
Publication of Results	$21,890	$21,890
Project Management	$75,733	$75,733
Overhead	$143,696	$143,696
Total	$436,524	$436,524

15. Explanation for Program Changes or Adjustments

This is a new collection of information.

16. Time Schedule and Dissemination Plan

AHRQ’s plans to summarize and disseminate this information include: requiring the contractor to 1) submit a manual of best practices for adoption of web and screener technologies in primary care; 2) submit a Final Report including detailed results, contractor conclusions, and contractor suggestions for additional research and immediately actionable lessons; and 3) write at least one manuscript of findings and submit it to a peer-reviewed journal. Given the exploratory nature of the project, the manual will be discussed in academic, PBRN, and AHRQ-sponsored conferences and subjected to further evaluation before widespread dissemination.

Assuming that we receive OMB approval by July 1, 2011, we will adhere to the following schedule of project activities. We will assist the practices in mailing information-consent letters starting July 1, 2011. We will mail baseline surveys to adolescent participants beginning July 15, 2011, and six-month follow up surveys starting January 15, 2012. We will prepare practices for adoption of the information technology components (RAAPS and out-of-office components) for initial deployment August 1, 2011. Deployment will continue for a study period of 12 months. Practices will distribute the post-visit surveys with the RAAPS questionnaire throughout this period.

Analysis of surveys will be conducted in early 2012. On-site data collection (chart review and qualitative efforts) will start August 1, 2012. Analysis of these data will continue through December 31, 2012. A draft manual will be submitted by January 12, 2012 and a final version will be submitted by February 12, 2010. A draft manuscript for publication will be submitted April 12, 2013 and a final version will be submitted May 12, 2013. We anticipate publication in scholarly journals such as Pediatrics or the Annals of Family Medicine.

Dissemination of findings through academic and AHRQ-sponsored conferences and PBRN activities will be conducted starting October 2011.

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments

Attachment A -- Healthcare Research and Quality Act of 1999

Attachment B -- RAAPS Questionnaire

Attachment C -- Extraction of Medical Record Data

Attachment D -- Adolescent Behavior and Communication Survey Information Letter

Attachment E -- Adolescent Behavior and Communication Survey Consent-Assent Form

Attachment F -- Adolescent Behavior and Communication Survey Baseline

Attachment G -- Adolescent Behavior and Communication Survey 6 Months

Attachment H -- Post-Visit Survey

Attachment I -- Adolescent Focus Group Guide

Attachment J -- Adolescent Think Aloud Guide

Attachment K -- Clinician Interview Guide

Attachment L -- Administrator-Staff Interview Guide

Attachment M -- Review of Draft Manual Interview Guide

References

American Medical Association. (1997). Guidelines for Adolescent Preventive Services (GAPS): Recommendations Monograph. Chicago: American Medical Association.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method (3rd ed.). Hoboken, NJ: Wiley.

Eaton, D. K., Brener, N. D., Kann, L., Denniston, M. M., McManus, T., Kyle, T. M., et al. (2010). Comparison of paper-and-pencil versus Web administration of the Youth Risk Behavior Survey (YRBS): risk behavior prevalence estimates. Eval Rev, 34(2), 137-153.

Fernald, D. H., Froshaug, D. B., Dickinson, L. M., Balasubramanian, B. A., Dodoo, M. S., Holtrop, J. S., et al. (2008). Common measures, better outcomes (COMBO): a field test of brief health behavior measures in primary care. Am J Prev Med, 35(5 Suppl), S414-422.

Hawn, C. (2009). Take two aspirin and tweet me in the morning: how Twitter, Facebook, and other social media are reshaping health care. Health Aff (Millwood), 28(2), 361-368.

Irwin, C. E., Jr., Adams, S. H., Park, M. J., & Newacheck, P. W. (2009). Preventive care for adolescents: few get visits and fewer get services. Pediatrics, 123(4), e565-572.

Klein, J. D., & Wilson, K. M. (2002). Delivering quality care: adolescents' discussion of health risks with their providers. J Adolesc Health, 30(3), 190-195.

Lehrer, J. A., Pantell, R., Tebb, K., & Shafer, M. A. (2007). Forgone health care among U.S. adolescents: associations between risk characteristics and confidentiality concern. J Adolesc Health, 40(3), 218-226.

Lenhart, A., Ling, R., Campbell, S., & Purcell, K. (2010). Teens and Mobile Phones. Washington, D.C.: Pew Research Center.

Lenhart, A., Purcell, K., Smith, A., & Zickuhr, K. (2010). Social Media and Young Adults. Washington, DC: Pew Research Center.

Montalto, N. J. (1998). Implementing the guidelines for adolescent preventive services. Am Fam Physician, 57(9), 2181-2188, 2189-2190.

Olson, A. L., Gaffney, C. A., Lee, P. W., & Starr, P. (2008). Changing adolescent health behaviors: the healthy teens counseling approach. Am J Prev Med, 35(5 Suppl), S359-364.

Salerno, J. (2008). Reducing risk behaviors in adolescents. New tool offers rapid assessment. Adv Nurse Pract, 16(4), 59-60, 62, 64.

Skinner, H., Biscope, S., Poland, B., & Goldberg, E. (2003). How adolescents use technology for health information: implications for health professionals from focus group studies. J Med Internet Res, 5(4), e32.

Sussman, M. P., Jones, S. E., Wilson, T. W., & Kann, L. (2002). The Youth Risk Behavior Surveillance System: updating policy and program applications. J Sch Health, 72(1), 13-17.

Yi, C. H., Martyn, K., Salerno, J., & Darling-Fisher, C. S. (2009). Development and clinical use of Rapid Assessment for Adolescent Preventive Services (RAAPS) questionnaire in school-based health centers. J Pediatr Health Care, 23(1), 2-9.

File Type	application/msword
File Title	SUPPORTING STATEMENT
Author	AHCPR
Last Modified By	bbarker
File Modified	2011-09-12
File Created	2011-09-12