Children’s Mental Health Initiative National Evaluation
Supporting Statement
Although most funded CMHI grantees are expected to participate in the Evaluation to some degree, the extent of each grantee’s participation will be limited as much as possible. (See Table 1).
Table 1. Participants: Respondents for each data collection activity by grant cohort, data source, and frequency of data collection
Data Collection Activity |
*Grant Type |
Data Sources |
Frequency |
All eligible grantees |
|||
Key Partner Interview |
All grantees |
High-level key partners
|
Twice for all grants: during the first 12 months and the last 12-18 months of grant funding.
|
SOCESS |
All grantees |
Key Partners
|
Baseline within the grant’s first 18 months, then annually through end of grant funding, for all grants. |
Network Analysis Survey |
All grantees
|
|
Twice: Baseline within the grant’s first 18 months of grant with follow up 2-3 years later |
Financial Mapping Interview |
All grantees |
Financial administrators |
Twice: Baseline within the grant’s first 18 months with follow up 2 years later |
Financial Plan Interview |
All grantees |
Financial Plan Manager |
Interviews years 2 , 3 and 4 |
Child and family outcome instruments
|
Children and Families receiving grant funded services |
|
Intake, discharge, 6 and 12 months (while receiving SOC services)
GIS: Intake |
Convenience Sample |
|
||
Benchmark Tool |
Volunteer grantees
|
Data compiled by personnel working with state Medicaid and MH Authority reporting and payment systems |
Twice: Baseline for two cohorts within the grant’s first 18 months. Follow-up 2 years later for the first cohort. |
Sample Size and Power Analysis for the Child and Family Outcome Component.
For the child and family outcome component, it is important that CMHS draws enough participants from each grantee to ensure the evaluation will be able to detect the impact of the SOC initiative on child and family outcomes. If the number of participants is too small, significant differences of an important magnitude might go undetected. The effect sizes of the phenomena of interest form the basis of determining the minimum number of participants needed through a statistical power analysis.1 In order to obtain complete follow-up data on 74 participants per grantee, it will be necessary to enroll 90 families into the evaluation for each grantee (based on a 90% retention rate at each follow-up data collection point). If SAMHSA assumes that grantees will serve 45 children for each full year of service delivery, 112 children will be served during the 2.5 years of enrollment period (i.e., the first six months will be start-up and the last year will be follow-up data collection). An initial response rate of 85% will allow the enrollment of 90 families.
CMHS conducted power analyses to determine the appropriate sample size. The overall goals of the Evaluation are twofold. CMHS believes that individual grantees should obtain a sufficient sample to conduct meaningful analyses for their own use. CMHS also needs to obtain sufficient data to conduct cross-site analyses related to the overall evaluation questions. Therefore, CMHS ran separate power analyses for these two separate yet related domains.
For individual grantee power analyses, CMHS uses the G*Power application to estimate the needed sample size using the following assumptions. CMHS assumes an average of three time points of data to be used in repeated-measures ANOVA. CMHS also assumes a retention rate of approximately 90% at each time point (for an overall baseline to 12 month follow-up retention rate of 81.5%), power of .80, an effect size of .26 or higher, repeated measures correlation of .5 or lower, and level-1 (time) variability of 1.0. Using these assumptions results in an estimated final (complete) sample size of 74 at each individual grantee needed in order to detect between-group differences in change over time in communities for their local analytic purposes.
For cross-site analyses, CMHS used the Optimal Design application to estimate the Minimum Detectable Effect Size (MDES) using the following assumptions: 69 Grantee Sites, three time points of data, a 3-level longitudinal Multilevel Growth Model testing a quadratic trend, a retention rate of approximately 90% at each time point and an overall baseline-to-12 month follow up retention rate of 81.5%, power of .80, residual variability of .3, level-1 (time) variability of 1.0, and 74 people for each grantee. The MDES ranges from .17 (very small effects) for an ICC of .10, .22 for an ICC of .20, and .27 for an ICC of .30.
Each participating grantee will be expected to recruit a sufficient number of children and families to ensure enrollment of 90 children and families in each grantee (or 74 after attrition). Complete data on 74 children and families in each of the 69 Grantee Sites will result in a final sample of 5,106 client families with complete data at the end of the year 2018. This sample size will be large enough to ensure the ability to detect changes in outcomes over time at both the local and national levels.
2. Information Collection Procedures
SAMHSA has contracted with Westat to conduct the Evaluation. Westat, and its subcontractors and consultants (listed in Section B.5), are referred to throughout this document as the CMHI National Evaluation. Child and family level data will be collected by local service provider agencies. The CMHI National Evaluation will conduct all other data collection activities directly with respondents. The CMHI National Evaluation will provide training and TA regarding child and family outcome instruments added to the CMHI portal and support grantees in the collection of child and family outcome data. The CMHI National Evaluation will receive de-identified client-level data from all grantees. Table 1 shows each data collection activity by respondent and data collection interval.
Implementation Assessment
The evaluation is designed using a strategic framework (adapted from Stroul and Friedman 2011 and Stroul, Dodge, Goldman, Rider and Friedman, 2015) that provides five analytic dimensions: 1) policies, 2) services/supports, 3) financing, 4) training/workforce, and 5) strategic communications). These dimensions cut across the State System, Local System and Service Delivery levels and together link to a range of proximal and distal outcomes. The evaluation will identify and assess the mechanisms and strategies employed to implement and expand systems of care, and explore the impact on system performance and child and family outcomes. Evaluation activities are framed by the five strategic areas to examine whether specific mechanisms and strategies lead to proximal and distal outcomes. System of care principles are woven throughout the framework at both the State and Local levels. The evaluation tools are designed to allow analysis across levels.
Key Partner Interview. The Key Partner Interview organizes qualitative data collection into these five areas and will allow within and across grantee evaluation of the implementation and impact of activities in these areas. The semi-structured interview will be conducted by phone with administrators, youth and family representatives and child agency representatives. This interview will be conducted twice, once at baseline during the first twelve months of grant implementation and once during the final 12-18 months of grant implementation.
SOCESS (System of Care Expansion and Sustainability Survey). The SOCESS is designed to capture self-report implementation data using the strategic framework adopted by the 2015 National Evaluation that consists of five analytic dimensions: 1) policies, 2) services/supports, 3) financing, 4) training/workforce, and 5) strategic communications) as discussed above. The SOCESS organizes self-report data collection into these five areas and will allow within and across grantee evaluation of the implementation and impact of activities in these areas.
This self-report survey will be administered via the online CMHI portal. Respondents will rate items on a Likert-type scale. Respondents will include representatives from approximately 54 grantee organizations and representatives from family and youth organizations, child-serving sectors, advocacy organizations for diverse populations, provider organizations, and financial officers, among others. Evaluation staff will identify potential respondents through previous evaluation efforts (e.g., document review, Key Partner Interviews) and invite those partners to participate in this component. Grantees will complete this survey in the first 12 to 18 months of funding and annually thereafter through the end of their funding period or June 2018, whichever comes first.
Network Analysis and GIS Component
Network Analysis Survey. This survey will be administered online to grantees via the CMHI portal described in Section A.3. Respondents will be high-level participants such as project directors, heads of child-serving agencies, and leaders of family and youth organizations. The survey will collect data on agencies and organizations with whom the respondent interacts as part of the SOC implementation and expansion effort. The list of these partner agencies and organizations will be developed based on document review, interviews with key partners, and other data collection efforts. In addition, respondents will have the opportunity to identify additional agencies/organizations with which they interact. For each survey item, respondents will report the extent to which their agency/organization engages in that activity with other agencies. In addition, the CMHI National Evaluation will ask respondents to indicate whether those relationships are formalized (i.e., whether there are written agreements, memoranda of understanding, or contracts). The initial survey will be conducted within the first 18 months of the grant’s funding, with the second administration 2 to 3 years later.
GIS. Child and family. For children/youth receiving services and their caregivers, grantee staff members already obtain addresses as part of routine intake procedures. To obscure families’ identity, the CMHI National Evaluation will provide grantee staff with software that converts home addresses to census block groups. Site staff will enter census block group data into the CMHI portal (but not addresses). This information will be collected at baseline only. Client/ family addresses will not be part of the evaluation dataset or transmitted by the grantees to SAMHSA or its contractors.
Grantees will participate in this Evaluation component in years 2 and 4 of their funding cycles.
Financial Mapping and Benchmark Component
Financial Mapping. The CMHI National Evaluation will make information requests and conduct semi-structured interviews with key grantee administrators and staff. Specifically, data will be collected on children’s MH funding sources for all states, counties, and tribes with CMHI grantees during the first 18 months of the grant and the next to last or last 12 months of the grant funding period. The CMHI National Evaluation will review publicly available information to develop a preliminary list of children’s mental health services in the state, county or tribe. This list will be sent to interview respondents at least a week prior to the interview with a request to make any needed corrections. The corrected list will then be incorporated into the interview schedules. The interview schedules will be shared with state and county agencies, and with tribal representatives who can describe Medicaid-funded, MH Authority-funded, and Indian Health Service-funded services in the form of a WebEx. In addition, the CMHI National Evaluation will also speak to representatives from family organizations about their funding sources and provider associations to learn what services are covered by commercial insurance plans. Key information from interviews with Mental Health agencies, Medicaid agencies and tribal authorities will be summarized in a matrix, and sent back to respondents for validation.
Financing Plan. The Team will collect data on grantees’ strategic financing planning process through an interview in years 2, 3 and 4. This analysis will include funding sources considered and the reasons for excluding any potential funding sources, agreements achieved to braid or pool funding, and barriers and facilitators to planning. The final interview will be conducted in the fourth year of the grant focusing on how the financial planning process supported or hindered attainment of sustainable financing.
Benchmark Component. Each grantee volunteering to participate in the Benchmark Component will receive preparation support for and begin cost data collection upon OMB approval. Data will be collected during the first 18 months for two cohorts. Data will be collected during the third and beginning of the final year of the grant funding period for the first cohort. Volunteer state or county MH and Medicaid agencies will collect and report a core set of data that will be used to calculate access, utilization, and costs for child MH services. The CMHI National Evaluation will provide states and counties with contact information to reach Evaluation staff if they have any questions about the data request. Evaluation staff has considerable experience in collecting these types of data and can effectively clarify any confusion or help to address limitations or problems that states may encounter when generating the requested information.
Child and Family Outcome Component
We will use SAMHSA’s existing data reporting requirements for the SAMHSA National Outcomes Measures (NOMS) system to identify persons for whom data will be collected for The Child and Family Outcome Study. Clients will need to: (1) receive services through a selected local service system within a funded grantee; (2) meet the local system’s service program eligibility criteria for SOC services; (3) be between age 0 and 26 years; (4) have a MH diagnosis; (5) not have a sibling already participating in the Evaluation; (6) have a participating caregiver if the client is age 0 to 17 years old; and (7) provide informed consent/assent, as appropriate based on client age. Data collection for this Evaluation component will begin soon after OMB approval.
Child and family data will be collected at intake, 6-months, and 12-months post service entry (as long as the child/youth is still receiving services). Data will also be collected at discharge if the child/youth leaves services before the 6- or 12-month data collection point. Evaluation staff will collect these follow-up data from caregivers of minor children and adolescents (age 0 to 17) and from youth and young adults age 11 to 26.
Several steps will be taken to maximize response rates and reduce non-response bias for all data collection efforts. The CMHI National Evaluation will lead and/or be available to support each data collection process. The CMHI National Evaluation will provide ongoing technical assistance and remain available to grantees and other respondents to respond to questions and provide clarification or guidance whenever needed.
For most data collection activities, the CMHI National Evaluation will collect data from participants involved in the planning, implementation, and expansion of SOCs. Efforts to maximize response rates are presented here by type of data collection method, as these apply across evaluation components.
Requesting documents. Document requests will be combined across other Evaluation components to minimize the number of requests and to avoid duplicate requests..
Identifying respondents among participants. The CMHI National Evaluation will work with the grantee’s project director to identify the appropriate people to interview. All respondents will be partners in the planning, implementation, and expansion of systems of care and will participate in the evaluation as part of the performance of their roles.
Scheduling interviews. The CMHI National Evaluation will be flexible in scheduling interviews, provide a copy of the interview schedule ahead of time, and respect the specified time limits. To make the best use of informants’ time, the CMHI National Evaluation will review available documents and perform web searches to collect publicly available information prior to the interview. To keep logistics and costs manageable, interviews will be conducted with individual informants by telephone, Skype, or video-conferencing.
Site liaison model. Individual CMHI National Evaluation staff will serve as a site liaison to each participating grantee to facilitate communication in ways that the CMHI National Evaluation anticipates will enhance response rates, data quality, and grantee motivation. In addition, the site liaison model will enable the CMHI National Evaluation to understand the grantees more comprehensively, which will be of value when interpreting findings.
The CMHI National Evaluation anticipates that grantees and other participants will be particularly motivated to participate in several data collection efforts of the Evaluation. Examples relevant to specific Evaluation components are as follows:
Financial Mapping. The CMHI National Evaluation anticipates that most informants will be interested in finding ways to financially sustain their SOC and will be motivated to participate in the Financial Mapping component. The CMHI National Evaluation will follow up with the people interviewed to share the draft financial map to confirm the CMHI National Evaluation’s understanding of the state’s use of funds, which the CMHI National Evaluation anticipate will further enhance motivation to participate
Benchmark Component. In the past, the CMHI National Evaluation has successfully collected similar data from over 31 state and county MH authorities and/or Medicaid agencies, who also participated on a voluntary basis. Grantees that elect to participate will be able to benchmark their state’s use of children’s MH resources against other participating states. The CMHI National Evaluation believes that states with well-developed information systems that can readily compile the needed data will be interested in the rare opportunity to compare how they use inpatient and residential care to other states. States in the cohort that will be benchmarked twice will also have the opportunity to document how expansion of their SOC may have changed their service use pattern and expenditure rates. This information may be valuable in demonstrating the business case for SOC to legislators and other participants.
The selection of data collection activities was based on a review of those used during the earlier National CMHI Evaluation (OMB Nos. 0930-0192, 0930-0209, 0930-0257, 0930-0280, and 0930-0349) in consultation with individuals involved in both evaluations; an assessment of measurement quality as reported in the literature; and decisions about data collection activities were made in conjunction with expert reviewers, consumers, and family members. These consultants are listed in Section B5. In addition most data collection activities proposed by this request have been thoroughly tested previously to minimize burden and refine the current collection of information. Testing consisted of pilot testing interviews and review of protocols by experts on systems of care. The extensive previous testing makes use of any further pre-testing unnecessary. Feedback from the previous testing was used to clarify individual questions, including re-wording items and adding definitions of terms, and additional information was added to instructions and introductory sections of the tools to provide additional clarity. Grantee participants also provided feedback on the presentation and display of the data collection tools (particularly those displayed online) to make the administration more user-friendly. For example, grantees indicated that it was helpful to display the Key Partner interview questions on the computer through WebEx so that they could read the questions at the same time the interviewer asked them.
The CMHI National Evaluation has full responsibility for the development of the overall statistical design, and assumes oversight responsibility for data collection and analysis for this Evaluation. Training, TA, and monitoring of data collection will be provided by the CMHI National Evaluation. The individual responsible for overseeing the entire evaluation, including all aspects of the design, data collection and analysis, and who had some involvement in the prior CMHI Evaluation, is the Principal Investigator:
Abram Rosenblatt, Ph.D.
Senior Study Director
Westat RB 4129
1600 Research Boulevard
Rockville, MD 20850
Office: (301) 517-4065
The following additional individuals will serve as statistical consultants to this project:
Ana Maria Brannan, Ph.D.
Associate Professor
Indiana University
201 N. Rose Avenue
Bloomington, Indiana 47405-1006
Office: (812) 856-8119
Alison Cuellar, Ph.D. (Health Economist, Financial Components, Consultant)
Associate Professor
Department
of Health Administration and Policy
George Mason University
4400
University Drive
MS: 1J3
Fairfax, VA 22030-4444
Office: (703) 993-5048
Michael Giangrande, M.G.I.S. (GIS Specialist, GIS Component)
Senior Study Director
Westat
RW 3546
1600 Research Boulevard
Rockville, MD 20850
Office: (301) 610-5107
Michael Pullmann, Ph.D. (Mixed Methods Specialist, Child and Family Outcome Data)
Research Assistant Professor
Public Behavioral Health and Justice Policy
Department of Psychiatry and Behavioral Science
University of Washington
2815 Eastlake Ave. East,
Suite 200
Seattle, WA 98195-8015
Office: (206) 685-0408
Michael Steketee, Ph.D. (Content Expert, Network Analysis)
Senior Study Director
Westat
1600 Research Boulevard
Rockville, MD 20850
240-453-2603
Data Collection and Analysis of Information:
Chandria Jones, Ph.D. (Project Manager)
Senior Study Director
Westat
RB 4107
1600 Research Boulevard
Rockville, MD 20850
(301) 251-4253
Johanna Bergan (Content Expert, Youth)
Executive Director
Youth
M.O.V.E. National
PO Box 215
Decorah, IA 52101
Office: (800) 580-6199 ext. 101
Eric Burns, Ph.D. (Content Expert, Wraparound Services, Consultant)
Associate Professor
University
of Washington Dept. of Psychiatry & Behavioral Sciences
Division
Public Behavioral Health & Justice Policy
2815 Eastlake
Avenue East, Suite 200
Seattle, WA 98102
Box 358015
Office: (206) 685-2477
Allen Daniels, Ed.D. (Content Expert, Health Care Systems, Consultant)
Senior Health Care Systems Specialist
Westat
1600 Research Boulevard
RB 4118
Rockville, MD 20850 -3129
Office: (513) 319-5614
Richard Dougherty, Ph.D. (Content Expert, Financial Components)
CEO
DMA
Health Strategies
9 Meriam Street, Suite 4
Lexington, MA
02420-5312
Office: (781) 863-8003
Lynda Gargan Ph.D. (Content Expert, Family)
Executive Director
National Federation of
Families for Children’s
Mental Health
12320 Parklawn Drive
Rockville, MD 20852
240-403-1901
Preethy George, Ph.D. (Content Expert, Children’s Behavioral Health Prevention Specialist)
Senior Study Director
Westat
1600 Research Boulevard
RB 4114
Rockville, MD 20850 -3129
(301) 738-3553
Craig Anne Heflinger, Ph.D. (Content Expert, SOC, prior CMHI Evaluation, Consultant)
Professor & Associate Dean
Department of Human and Organizational Development
Peabody College of Education and Human Development
Vanderbilt University
Mayborn Bldg., Room 206
130 Magnolia Circle
Nashville, TN 37203-5721
Office: (615) 322-8275
Wendy Holt, Ph.D. (Content Expert, Financial Components)
Principal
DMA
Health Strategies
9 Meriam Street
Lexington, MA 02420-5312
Office: (781) 863-8003
Craig Love, Ph.D. (Content Expert, Native American/Native Alaskans, Consultant)
Senior Study Director
Westat
1600 Research Boulevard
RB 3148
Rockville, MD 20850 -3129
(240) 314-2443
Nanmathi Manian, Ph.D.
Senior Study Director
Westat
1600 Research Boulevard
RB 3143
Rockville, MD 20850 -3129
301-294-2863
Brianne Masselli (Content Expert, Youth)
Director of Technical Assistance and Evaluation
Youth M.O.V.E National
Office (202) 808-3992 X104
Kurt Moore, Ph.D. (Report to Congress, Child and Family Outcomes, prior CMHI Evaluation)
Walter R. McDonald & Associates, Inc.
1626 Washington Street
Denver, CO 80203
Office: (916) 239-4020 ext. 409
Garrett Moran, Ph.D. (Project Director, Behavioral Health Systems Expert)
Vice President
Westat
1600 Research Boulevard
RB 4118
Rockville, MD 20850 -3129
Office: (301) 294-3821
Mary Anne Myers, Ph.D. (Qualitative Assessment Expert)
Associate Director
Westat
1600 Research Boulevard
RB 4105
Rockville, MD 20850 -3129
Office (240) 453-2673
Debra Rog, Ph.D. (Content Expert, Evaluation Design, Consultant)
Associate Director
Westat
1600 Research Boulevard
RW 3526
Rockville, MD 20850 -3129
Office: (301) 279-4594
Martha Stapleton (Survey and Questionnaire Design and testing Expert, Consultant)
Senior Study Director
Westat
1600 Research Boulevard
RB 4161
Rockville, MD 20850 -3129
Office: 301-251-4382
Beth A. Stroul, M.Ed. (Content Expert, SOC, prior CMHI Evaluation, Consultant)
Management & Training Innovations, Inc.
7417 Seneca Ridge Drive
McLean, VA 22102
(703) 448-7570
Jessica Taylor, Ph.D. (TRAC Data Collection and Management Expert)
Westat
1600 Research Boulevard
RB 4144
Rockville, MD 20850 -3129
Office: (240) 314-5852
The SAMHSA staff person responsible for receiving and approving contract deliverables is:
Kris (Kirstin) Painter, PhD, LCSW
Public Health Analyst
Center for Mental Health Services
Division of Service and System Improvement
Substance Abuse and Mental Health Services Administration
5600 Fishers Lane Room 14E89D
Rockville, MD 20857
Office: 240-276-1932
Fax: 240-276-1990
References
Barksdale, C. L., Ottley, P.G., Stephens, R., Gebreselassie, T., Fua, I., Azur, M., et al. (2012). System-level change in cultural and linguistic competence (CLC): How changes in CLC are related to service experience outcomes in system of care. American Journal of Community Psychology, 49(3-4), 483-493.
Bickman, L. & Heflinger, C. A. (1995). Seeking success by reducing implementation and evaluation failures. In L. Bickman & D. J. Rog (Eds.), Children's mental health services: Research, policy and innovation (pp.171-205). Newbury Park, CA: Sage.
Bird, H. R., Shaffer, D., Fisher, P., Gould, M. S., Stagehezza, B., Che, J. Y., et al. (1993). The Columbia impairment scale (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research, 3,167–176.
Brannan, A. M. (2003). Ensuring effective mental health treatment in real-world settings and the critical role of families. Journal of Child and Family Studies, 12(1), 1-10.
Brannan, A. M., Brashears, F., Gyamfi, P., & Manteuffel, B. (2012). Implementation and development of federally-funded systems of care over time. American Journal of Community Psychology, 49, 467–482.
Brannan, A.M., & Hazra, M. (2012). Final report of the evaluation of the commUNITY cares system of care initiative. Pine Belt Mental Health Resources, Hattiesburg, Mississippi.
Brannan, A. J., Heflinger, C. A. & Bickman, L. (1997). The Caregiver Strain Questionnaire: The impact of living with a child with serious emotional disturbance. Journal of Emotional and Behavioral Disorders, 5(4), 212-222.
Fixsen, D., Blase, K., Metz, A., & van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79(2), 213-230.
Manteuffel, B., Stephens R.L., Sondheimer D.L., Fisher S.K. (2008). Characteristics, service experiences, and outcomes of transition-ages youth in systems of care: Programmatic and policy implications. Journal of Behavioral Health Services Research, 35(4), 469-487.
Merikangas, K.R., He J.P., Brody D., Fisher P.W., Bourdon K., Koretz D.S. (2010). Prevalence and treatment of mental disorders among US children in the 2001-2004 NHANES. Pediatrics, 125(1):75-81.
Spybrook, J., & Raudenbush, S. W. (2009). An examination of the precision and technical accuracy of the first wave of group-randomized trials funded by the institute of education sciences. Educational Evaluation and Policy Analysis, 31(3), 298-318.
Stroul, B., Blau, G., & Friedman, R. (2010). Updating the system of care concept and philosophy. Washington, DC: Georgetown University Center for Child and Human Development, National Technical Assistance Center for Children’s Mental Health.
Stroul, B. A., & Friedman, R. M. (2011). Effective strategies for expanding the system of care approach. A report on the study of strategies for expanding systems of care. Atlanta, GA: ICF Macro.
Stroul, B. A., & Manteuffel, B. (2008). Sustaining systems of care Paul H Brookes Publishing, Baltimore, MD.
Evaluation Logic Model
Semi-Structured Key Partner Interviews
SOCESS
Network Analysis Survey
Financing Plan Survey/Interviews
Financial Mapping Interview Protocol
Financial Benchmarking Tool
Child and Family Level Data Tool
1 NOTE. Briefly, the power of a statistical test is generally defined as the probability of rejecting a false null hypothesis. In other words, power gives an indication of the probability that a statistical test will detect an effect of a given magnitude that, in fact, exists in the population. The power analysis does not indicate that a design will actually produce an effect of a given magnitude. The magnitude of an effect, as represented by the population parameter, exists independent of the component and is dependent on the relationship among the independent and the dependent variables in question. The probability of detecting an effect from the data, on the other hand, depends on several major factors in multi-level or repeated-measures frameworks, some of which include: (1) the level of significance used; (2) the size of the treatment effect in the population; (3) sample size; (4) the intraclass correlation(s), that is, the amount of individual variance accounted for by membership within a group (or nesting), or, similarly, the correlation among repeated measures; (5) the amount of measurement error.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Wendy Kissin |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |