Doc com_Supporting Statement B_Revised 7 8 11

NIDA’s Study of Substance Abuse doc.com Module Project

Supporting Statement B for

THE NATIONAL INSTITUTE ON DRUG ABUSE’S (NIDA)

STUDY OF SUBSTANCE ABUSE DOC.COM MODULE PROJECT

April 2011

Name: Elisabeth Davis, MPH

Address: Office of Science Policy and Communications

National Institute on Drug Abuse

National Institutes of Health

6001 Executive Boulevard

Bethesda, MD 20824-9591

Telephone: (301) 594-6317

Fax: (301) 443-7397

Email: [email protected]

Table of Contents

B. Collections of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods B-4

B.2 Procedures for the Collection of Information B-6

B.3 Methods to Maximize Response Rates and Deal with Nonresponse B-12

B.4 Test of Procedures or Methods to be Undertaken B-13

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting

and/or Analyzing Data B-13

References for Section B …………………………………………………………….B-14

List of Tables

Table 5: Power Derivations Assuming 7 Sections per Group with 15 Students per Section (p. B-8)

Table 6: Power Derivations Assuming 7 Sections per Group with 10 Students per Section (p. B-9)

Table 7: Power Derivations Assuming 7 Sections per Group with 5 Residents per Section (p. B-9)

Table 8: Schedule of Events for Medical Student (Group 1) Research (p. B-11)

Table 9: Schedule of Events for Resident (Group 2) Research (p. B-11)

LIST OF ATTACHMENTS

Attachment B1:

Attachment B1-1: Target Populations, Sampling Universes, Estimated Sample Sizes, and Sampling Methods

Attachment B2:

Attachment B2-1: Survey 1 for Medical Students (Pre-Survey 1 or 2)

Attachment B2-2: Survey 2 for Medical Students (Post-Survey 1 or 2)

Attachment B2-3: Survey 1 for Residents (Pre-Survey)

Attachment B2-4: Survey 2 for Residents (Post-Survey)

Attachment B3: Pilot Test Results of Survey by Medical Students

Attachment B4: Pilot Test Results of Survey by Residents

B.1 Respondent Universe and Sampling Methods

The aim of this study is to sample two important populations of healthcare professionals: medical students (Group 1) and residents in primary care residency programs (Group 2). This research will assess the efficacy of the NIDA substance abuse doc.com module. It will also provide an excellent set of data on which to design a much larger research study of the same intervention but with larger sample sizes. The latter will provide a much more diverse sampling of both universes (i.e., all U.S. medical students and medical residents).

Due to constraints of time and budget, a convenience sample of medical students from the two medical schools that are conducting this project will be surveyed. For the same reason, a convenience sample of primary care residents, who are affiliated with these same two medical schools and work primarily in the respective main teaching hospitals, will be surveyed.

Despite sampling only subjects from two medical schools and affiliated residency programs, the results of the study will still be generalizable, to a certain extent, to the universe of U.S. medical students and primary care residents. This is because all medical schools must meet national standards of the Liaison Committee on Medical Education (LCME, http://www.lcme.org/), the national accrediting body for all 132 allopathic U.S. medical schools. Likewise, all internal medicine and family medicine residency training programs must meet national standards of the Accreditation Council for Graduate Medical Education (ACGME, http://www.acgme.org/), the national accrediting body for all graduate medical education (GME) programs (residencies and fellowships) that are affiliated with U.S. medical schools.

However, there are still likely to be limits to the extent of this generalizability. For example, national data indicate that Penn Med and DUCOM medical students have more formal hours of instruction in their preclinical curricula that are devoted to education related to substance use disorders than other U.S. medical schools.

According to the latest national survey available in this regard, Penn Med has 23 hours of required formal education in substance use disorders for medical students in their preclinical years while DUCOM students have 13 such hours in their preclinical curriculum years. In contrast, data from 126 U.S. medical schools that were collected by the 2008 annual survey by the LCME indicate that, on average, there are 10 hours (“sessions”) of teaching related to substance abuse in the required preclinical years (http://services.aamc.org/currdir/section2/2008hottopics.pdf).

NIDA envisions that this project will be the first phase of a multi-phase research program to evaluate the NIDA substance abuse doc.com module in a much wider sampling of medical schools and residency programs. The expectation is that, in the future, with a larger sample that includes more medical schools, the results of the currently proposed research would be replicated and would allow for adjustment for important co-variables (e.g., hours of pre-clinical education in substance abuse, career plans), as well as a priori subgroup analyses.

Examples of additional variables representing a broader diversity of medical schools include: public versus private; geographic diversity; research-oriented versus non-research oriented; and primary care oriented versus non-primary care oriented. Examples of variables representing diversity of residency programs in primary care include public versus private versus Veterans Administration hospitals; geographic diversity; affiliation with research-oriented versus non-research oriented medical schools; and primary care oriented versus non-primary care oriented tracks within internal medicine residencies.

In addition, the vision is that future phases of this research project will include evaluation of the efficacy of these educational interventions in a sample of practicing primary care physicians. Although this population of practicing physicians was not included in the current project, due to limits in scope, time, and funds, the medical students and residents can be regarded as representing populations that will become the practicing physicians in the future.

For Group 1 (medical students) the total population is 400 students, representing 150 students from Penn Med and 250 students from DUCOM. Thus, there should be a sufficient population of medical students to reach the target sample size of 210. For Group 2 (residents) the total population is 308 residents, representing 174 residents from Penn Med and 134 residents from DUCOM. Thus, there should be a sufficient population of residents to reach the target sample size of 70. More information on how the target sample sizes were obtained can be found in Section B.2 below.

It is expected that the proposal’s methods will result in response rates of at least 80% of eligible respondents for both Groups 1 and 2. This estimate is based on previous experience in doing medical education research in these populations at Penn Med and DUCOM as part of the NIDA Centers of Excellence (CoE) for Physician Information activities and other research activities. This expected rate refers both to the rate of enrollment into the research study of eligible medical students or residents and to the rate of completion of all of the pre- and post-Intervention surveys that are dictated by the protocol. The study has not been previously conducted so no prior response rates are available. It is expected that the study will include women as subjects with an expected percentage of enrollment that reflects their proportion of the class. This is approximately 50% of both Penn Med and DUCOM medical students and residents. It is also expected that the study will include subjects with diverse ethnic and racial backgrounds that reflect these subgroups in their respective proportion of the class. The surveys will capture these demographic data (e.g., race/ethnicity, gender) for medical students and residents. In addition, the surveys will capture the age of the respondents. The analytic plan will take these demographic variables into account during the analysis phase as co-variables.

It is beyond the aim, scope, timeline, and budget of this project to sample the entire universe of U.S. medical students or U.S. medical residents in ACGME certified residency training programs.

B.2 Procedures for the Collection of Information

All participants in Group 1 and Group 2 will complete the informed consent form. Medical students who participate in Group 1 will complete surveys using either electronic or paper and pencil methods, whether in the control or intervention arms. Medical students at Penn Med will complete all four surveys on paper during class or clerkship rotation at the following points in time:

• Pre-Survey 1: Prior to the beginning of clinical year.

• Pre-Survey 2: On the first day of family medicine rotation.

• Post-Survey 1: Penn Med: On the last day of family medicine rotation.

• Post-Survey 2: Penn Med: At the end of the year of clinical rotations.

At DUCOM, the following survey administration schedule will be implemented:

• Pre-Survey 1: Electronic survey administration; prior to the beginning clinical year.

• Pre-Survey 2: Paper and pencil administration of survey; on the first day of ambulatory medicine rotation.

• Post-Survey 1: Electronic administration of survey; on the last day of ambulatory medicine rotation.

• Post-Survey 2: Electronic administration of survey; at or near the end of the year of clinical rotations.

The procedure that will be used to collect information for Group 2 (residents) will be password accessible online surveys that residents in both the intervention and control arms will complete at time points before and after the intervention in the intervention arm and at the same time points for residents in the control arm. As specified in section A.6, the resident surveys will be administered as follows:

Resident Pre-Survey:

• DUCOM: At or near the start of outpatient primary care rotation.

• Penn Med: On the first day of primary care medicine rotation during orientation.

Resident Post-Survey:

• DUCOM: On or near the end of outpatient primary care rotation.

• Penn Med: On the last day of primary care medicine rotation or within one week of completing the rotation.

As described in Sections A.1, A.2, and A.16, the research design stipulates the outcome variables will be changes in the Pre-Survey versus Post-Survey in the intervention arm compared to changes in the same Pre- and Post-Surveys in the control arm. Students and residents will be randomized to control and intervention arms in “clusters.”

In Group 1 of the study, the clusters are groups of students who are together for 4-week periods on a Family Medicine clerkship (Penn Med) or ambulatory Medicine clerkship (DUCOM). The number of students in each cluster at each school may vary from month to month between 10 and 15 students.

In Group 2 of the study, the clusters are groups of residents who are together for 2 to 4 week periods at a Family Medicine clinic (Penn Med) or an Internal Medicine clinic (Penn Med and DUCOM). The number of residents at each clinic at each school may vary from month to month between 4 and 10 residents.

These clusters will be assigned to an intervention or control arm through the use of randomized block permutations schema with a 1:1 ratio of intervention to controls. The subjects will not be blinded to the randomization assignment because of the transparent nature of the intervention.

Data Collection Plan

The project will use a repeated measures approach to assess the educational intervention’s efficacy. Online surveys will be preferentially used to reduce burden on respondents.

The study will achieve its specific aims, as described in Section A.1, by collecting information from individual subjects (either medical students in Group 1 or primary care residents in Group 2) by use of surveys that are administered before (“Pre-Survey”) and after (“Post-Survey”) the intervention in the intervention arm and before and after the standard curriculum at the same time points in the control arm.

The study will compare the changes in knowledge, attitudes, and skills, as measured by differences between the Pre-Survey and Post-Survey in the intervention arm with the analogous changes that occur in the control arm.

Sample Size and Power

Subject to the assumptions that are described below, the sample size calculations indicate that Group 1 of the study will need a total of 140 to 210 students, i.e., between 70 and 105 students in each arm of the study, to achieve 80% power. If one assumes a response rate of 80%, then Group 1 will need 262 students as its population to recruit from, i.e., 80% x the population to enroll from = 210 (or 210/0.8 = 262.5). The Table (Attachment B1-1) indicates that the plan is to enroll from a total population of 400 students, representing 150 students from Penn Med and 250 students from DUCOM. Thus, there should be a sufficient population of medical students to enroll from in order to reach the target sample size of 210.

Likewise, subject to the conditions described below, Group 2 of the study will need a total of 70 residents, i.e., 35 residents in each arm of the study, to achieve 80% power. If one assumes a response rate of 80%, then Group 2 will need 88 residents as its population to recruit from, i.e., 80% x the population to enroll from = 70 (or 88 x 0.8 = 70). The Table (Attachment B1-1) indicates that the plan is to enroll from a total population of 308 residents, representing 174 residents from Penn Med and 134 residents from DUCOM. Thus, there should be a sufficient population of residents to enroll from in order to reach the target sample size of 70.

These power calculations address the nested structure of the data in order to provide accurate estimates of power. For this study design, the results of surveys by students or residents within a cluster are assumed to be correlated. This correlation has an important impact on the resulting tests of significance (Kraemer, 1981). When this within-subject correlation is properly incorporated, the hierarchical linear modeling (HLM) analysis takes full advantage of all information obtained from each subject within a cluster, thereby more accurately reflecting the true statistical power over methods that ignore the clustering of the subjects within each cluster (Gibbons et al., 1993). Using the method described by Diggle et al. (2002), Ahn et al. (2001) derived formulas for power estimation for nested models. The latter were utilized for the power calculations described above, and for Tables 5-7, below.

In these calculations, the within-cluster correlations among the subjects receiving the intervention at the same time will be accounted for. Power calculations are based on various effect sizes and within group correlation estimates. Effect sizes are represented by the standardized difference between the two groups (Intervention versus Control) in the pre-post change of outcome variable (which are scores on the surveys). It is equivalent to asking “By how many standard deviations (or fraction of standard deviations) are the two means (one for each group) separated?”

The power calculations in Tables 5-7 to detect a significant difference between the two arms (Intervention versus Control) assume the following parameters: (a) alpha-level set at 0.05, (b) sample size per cluster set as 5 for residents and 10 or 15 for students; (c) number of clusters per study arm set as 7; (d) effect size ranging from 0.4 to 0.8 pooled standard deviations; (e) within cluster correlation ranging from 0.05 to 0.35.

Table 5: Power Derivations Assuming 7 Clusters per Study Arm with 15 Students per Cluster*

Effect Size**	0.4	0.5	0.6	0.7	0.8
Correlation^
0.05	0.60369	0.79350	0.91533	0.97320	0.99354
0.10	0.46449	0.64750	0.80130	0.90557	0.96260
0.15	0.37682	0.53890	0.69469	0.82140	0.90861
0.20	0.31804	0.45958	0.60650	0.73953	0.84461
0.25	0.27636	0.40047	0.53563	0.66676	0.78011
0.30	0.24542	0.35523	0.47867	0.60421	0.71971
0.35	0.22161	0.31973	0.43244	0.55098	0.66511

*105 subjects in each arm of the study (15 students per cluster x 7 clusters) or a total of 210 subjects (combined arms)

**Expressed as a proportion of a pooled standard deviation ranging from 0.4 to 0.8

^ Correlation = Correlation among subjects in the same cluster ranging from 0.05 to 0.35

From Table 5, if the assumption is a maximal within cluster correlation of 0.15 (highlighted row), there is at least 80% power to detect an effect of 0.68 or larger (by extrapolating between columns 0.6 and 0.7). Therefore, an average difference of 68% of a pooled standard deviation or larger between the two study arms (intervention versus control) is detectable at an alpha=0.05 level with a sample size of 105 students in each arm or a total of 210 students).

Table 6: Power Derivations Assuming 7 Clusters per Study Arm with 10 Students per Cluster*

Effect Size** Correlation.^	0.4	0.5	0.6	0.7	0.8
0.05	0.50209	0.69025	0.83839	0.93045	0.97561
0.10	0.40394	0.57379	0.73080	0.85185	0.92971
0.15	0.33860	0.48789	0.63892	0.77080	0.87022
0.20	0.29262	0.42379	0.56409	0.69669	0.80743
0.25	0.25872	0.37481	0.50360	0.63201	0.74709
0.30	0.23278	0.33646	0.45439	0.57651	0.69165
0.35	0.21234	0.30575	0.41391	0.52906	0.64181

*70 subjects in each arm of the study (10 students per cluster x 7 clusters) or a total of 140 students (combined arms)

**Expressed as a proportion of a pooled standard deviation ranging from 0.4 to 0.8.

^Correlation = Correlation among subjects in the same cluster ranging from 0.05 to 0.35

If the cluster size is assumed to be slightly smaller (i.e., 10 students per cluster), with the same assumed maximum within-cluster correlation of 0.15, the study has at least 80% power to detect an effect of 0.73 or larger (by extrapolating between columns 0.7 and 0.8 in Table 6). Therefore, an average difference of 73% of a pooled standard deviation or larger between the two arms (intervention versus control) is detectable at an alpha=0.05 level with a sample size of 70 per study arm or a total of 140 students.

The power and sample size calculations for the residents (Group 2) is similar to the above analytic approach described for the medical students (i.e., implementing the clustered design), but the calculations are based on only 5 residents per cluster (Table 7).

Table 7: Power Derivations Assuming 7 Clusters per Study Arm with 5 Residents per Cluster*

Effect Size** Correlation^	0.6	0.7	0.8	0.9	1.0
0.05	0.62980	0.76214	0.86326	0.93016	0.96848
0.10	0.56409	0.69669	0.80743	0.88915	0.94244
0.15	0.50971	0.63873	0.75358	0.84531	0.91105
0.20	0.46449	0.58810	0.70349	0.80130	0.87659
0.25	0.42656	0.54406	0.65780	0.75875	0.84088
0.30	0.39445	0.50571	0.61651	0.71850	0.80522
0.35	0.36701	0.47220	0.57937	0.68093	0.77045

*35 subjects in each arm of the study (5 residents per cluster x 7 clusters) or a total of 70 residents (combined arms)

**Expressed as a proportion of a pooled standard deviation ranging from 0.6 to 1.0

^Correlation = Correlation among subjects in the same cluster ranging from 0.05 to 0.35

For the residents, a higher within cluster correlation of 0.25 (highlighted row in Table 7) is assumed due to the more advanced training of the residents as well as the smaller cluster size. Based on these settings, the study has at least 80% power to detect an effect of 0.95 or larger by extrapolating between columns 0.9 and 1.0. Therefore, an average difference of 95% of a pooled standard deviation or larger between the two treatment arms is detectable at an alpha=0.05 level, i.e., it can detect close to one standard deviation difference between the two arms of the study. Note that this is larger than that detectable in the study of the students (Tables 5 and 6) due to the smaller sample size (n= 35 in each arm for the residents versus 70 or 105 in each arm for the students) and larger within cluster correlation (0.25 in the residents versus 0.15 in the students).

Both of these power and sample size calculations indicate that the expected number of participants will allow each arm of the study to detect an effect size less than one pooled standard deviation. Expressing effect size in terms of one standard deviation or fraction of a standard deviation is a customary approach to sample size and power calculations when there are no preliminary data, as is the case with this project.

Data Collection Instruments

All measurement instruments are designed to meet the specific aims of the study that are presented in section A.1. In brief, the surveys will be designed to answer the study evaluation questions. Specifically, all surveys will be designed to measure respondents’ knowledge, attitudes, and indirect measures of communication practices with patients with substance use disorders. In addition, measurement instruments will collect demographic data (e.g., age, gender, year of education or training, ethnicity, and race) and type of residency or career plans (e.g., areas of specialization) as applicable.

For all written measurement instruments (i.e., surveys), the questions have been organized and instructions clarified to make it easy to complete each question with minimal burden. There are a total of two instruments for Group 1 (medical students) and two instruments for Group 2 (residents) (see Attachments B2-1, B2-2, B2-3, B2-4 for copies of these instruments). The Group 1 instruments will be administered twice each for a total of four times per subject. The Group 2 instruments will be administered once each for a total of two times per subject (see Tables 8 and 9).

Table 8: Schedule of Events for Medical Student (Group 1) Research

	Timeline	Event	Intervention Arm+	Control Arm+
1	At the start of first clinical year of training*	Pre-Survey 1	X	X
2	On the first day of the specified clinical rotation^	Pre-Survey 2	X	X
3	During weeks 2 and 3 of four-week clerkship^	NIDA Online Module Exposure	X
4	During weeks 2 and 3 of four-week clerkship^	Control Exposure		X
5	On the last day of the specified clinical rotation^	Post-Survey 1	X	X
6	At or near the end of the year of clinical rotations*	Post-Survey 2	X	X

⁺Medical students will be divided into Intervention and Control Arms by use of cluster randomization in which a cluster consists of all medical students who are together during a specific four week clerkship at Penn Med or DUCOM.

*First clinical year at Penn Med is from January to December. First clinical year at DUCOM is from July to June. Note: Study will start at the earliest point in time during the first year of clinical training at both schools after OMB approval.

^Family Medicine clerkship at Penn Med; Ambulatory Medicine clerkship at DUCOM.

Table 9: Schedule of Events for Residents (Group 2) Research

	Timeline	Event	Intervention Arm+	Control Arm+
1	DUCOM: At or near the start of outpatient primary care rotation; Penn Med: On the first day of primary care medicine rotation during orientation*	Pre-Survey	X	X
2	During week 1 of two-week assignment or during weeks 1 and 2 of four-week assignment^	NIDA Online Module Exposure	X
3	During week 1 of two-week assignment or during weeks 1 and 2 of four-week assignment^	Control Exposure (Standard Curriculum)		X
4	DUCOM: On or near the end of outpatient primary care rotation; Penn Med: On the last day of primary care medicine rotation or within one week of completing the rotation *	Post-Survey	X	X

⁺Residents will be divided into Intervention and Control Arms by use of cluster randomization in which a cluster consist of all residents who are together during a given two or four week ambulatory medicine assignment at a single clinical site of practice and supervision.

*Outpatient assignments are 2 or 4 weeks long for Penn Med-affiliated residents and 4 weeks long for DUCOM affiliated residents. Residency schedules use a July to June academic year. Note: Study will start at the earliest point in time during the academic year at both schools after OMB approval.

^Family Medicine and Internal Medicine residents at Penn Med; Internal Medicine residents at DUCOM.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

Maximizing response rates is a priority for the investigators. The study conducted for the project will employ incentives (educational and non-educational) to ensure acceptable response rates are achieved (i.e., 80% response rates). Response rates will be determined by dividing the number of respondents who completed both the Pre- and Post-surveys and the intervention (if in the intervention arm) by the total number of potential subjects (i.e., those students or residents who meet all inclusion criteria and none of the exclusion criteria, received a survey, were asked to participate in the survey, etc.) and multiplying by 100. See Section B.2 for more information on the sample size calculations.

Incentives (educational and non-educational) will be used to maximize response rates. The non-educational, non-monetary incentives are discussed in Section A.9. Educational incentives will also be used because both populations of subjects are learners. In order to avoid any feelings of coercion by medical students or residents, participants are being offered incentives to participate in this study (see Section A.9). In addition, all participants will benefit from exposure to the knowledge that they gain from accessing the doc.com module either in the intervention arm or the control arm (following the study completion). Finally, a statement is included in the email invitations that specifically addresses that no penalty shall be incurred by those who choose not to participate.

Commitments have been obtained from the teachers of the medical students in the targeted ambulatory clerkships, and the faculty preceptors (teachers) of the residents, to participate as consultants in this project and to provide local encouragement and enthusiasm for participation of as many students or residents in the study as possible. These faculty members are all primary care physicians who truly value the potential of the NIDA CoE online teaching module and recognize that primary care physicians need to be much better prepared to diagnose and treat patients who abuse substances.

Participating in the study offers the opportunity (50:50 chance) to obtain additional education (during the intervention) related to substance use disorders, which is an important and challenging clinical disorder. Medical students who are randomized to the control arm will be exposed to a clinical topic or case different from the NIDA substance abuse doc.com module. Furthermore, researchers believe that both medical students and residents will have an inherent educational interest and curiosity to enroll in the study because of the opportunity for the unique learning experience that the study offers. All participants will receive access to the substance use disorder educational information provided to the intervention arm by the time the study is completed.

To increase the likelihood of participation, non-monetary incentives will be offered to complete the doc.com survey and the doc.com module, as described in Section A.9.

To address any non-responses, additional email requests for participation will be sent (see Section A for attachments). This will be follow-up by the in-person recruitment during the educational session. Based on previous experience, this approach should result in the desired response rates (i.e., at least 80%).

In the unexpected event that the response rates do not reach the expected 80%, we will do the following:

1. We would first assess if the response rates are different or not in the 2 arms of the study, stratified within the categories of participants (Penn and Drexel and medical students and residents).

2. We would assess consent rates vs. participation rates across these subgroups.

3. We would assess the impact of a lower response rate by doing sensitivity analyses with imputation that the associated factors are present or absent to various degrees in the non-responders. These sensitivity analyses would be informed by explicit assumptions. How the lower response rate would impact the study’s conclusions would depend on the results of these sensitivity analyses.

B.4 Test of Procedures or Methods to be Undertaken

Pilot tests of all measurement instruments and data collection procedures have been conducted with subsamples of the target populations for the study. The purpose of the pilot testing was to estimate the time required to complete the survey and to identify confusing or problematic questions or sequences. See Attachments B3 and B4 for details of pilot testing in medical students and residents, respectively.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals are responsible for statistical aspects of design and analyses:

J. Richard Landis, Ph.D., the Project’s Lead Biostatistician and Director of the Data Coordinating Center, has overall responsibility for interviewing, data collection, management, and analyses.

J. Richard Landis, Ph.D.

Professor and Director, Division of Biostatistics

Department of Biostatistics and Epidemiology

Center for Clinical Epidemiology and Biostatistics

University of Pennsylvania School of Medicine

600 Blockley Hall

423 Guardian Drive, Philadelphia, PA 19104-6021

Voice: 215-573-4922

FAX: 215-573-4865

Email: [email protected]

Robert Gallop, Ph.D, Senior Biostatistician at Center for Clinical Epidemiology and Biostatistics, is responsible for conducting the planned data analysis under Dr. Landis’ supervision and guidance.

Robert Gallop, Ph.D.

Associate Professor of Statistics

Department of Mathematics and Applied Statistics

West Chester University of Pennsylvania, West Chester, PA 19383

Voice: 610-436-2419

Email: [email protected]

REFERENCES

Ahn C, Overall JE and Tonidandel S. Sample size and power calculations in repeated measurement analysis, Computer Methods and Programs in Biomedicine, 2001; 64, 121-124.

Diggle PJ, Heagerty P, Liang KY, and Zeger SL. Analysis of longitudinal data. Second edition. New York (USA): Oxford University Press, Oxford Statistical Science Series 25; 2002.

Gibbons RD, Hedeker D, Elkin I, Waternaux C, Kramer HC, Greenhouse JB, Shea T, Imber SD, Sotsky SM, and Watkins JT. Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Archives of General Psychiatry, 1993; 50, 739-750.

Kraemer H. Coping strategies in psychiatric clinical research. J Consult Clin Psychology, 49: 309-319, 1981.

B - 16