NCS Supporting Statement_June 8 2015_1200pm

Supporting Statement for National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care: Evaluation of Awareness, Adoption, and Implementation

Date: June 8, 2015

OMB Cont.: 201412-0990-002 0990

Title: National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care: Evaluation of Awareness, Adoption, and Implementation

Program: HHS/OS/OMH

Contact: Rashida Dorsey, Director—OMH Division of Policy and Data, 240-453-2882

A. Justification

1. Circumstances Making the Collection of Information Necessary

The Office of Minority Health (OMH), Office of the Secretary (OS), Department of Health and Human Services (HHS) is requesting approval from OMB to collect information for an evaluation of the National Standards for Culturally and Linguistically Appropriate Services in Health and Health Care (the National CLAS Standards) within non-federal health and health care organizations in the United States. While much progress has been achieved in the provision of culturally and linguistically appropriate services, little is known about the process(es) through which health and health care organizations utilize the National CLAS Standards to implement these services. Despite their increased recognition as a fundamental tool for health and health care organizations to use in their efforts to become more culturally and linguistically competent, neither the original (released in 2001) nor the enhanced National CLAS Standards (released in 2013) have been systematically evaluated in terms of public awareness, organizational adoption and implementation, or impact upon health services outcomes. Given this, it is difficult for those who wish to support and use the National CLAS Standards in their efforts to become more culturally and linguistically appropriate, because of limited understanding of how the Standards are supposed to be implemented to achieve desired outcomes. It is also difficult to determine whether the enhanced National CLAS Standards actually achieve their expected outcomes (i.e., advancing health equity, improving quality, and reducing health care disparities). The goal of this evaluation is to generate detailed descriptions of the implementation of the National CLAS Standards, and to develop a foundational knowledge base required to formulate and explore additional questions.

This assessment will systematically examine and document the awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards among non-federal health and health care organizations. The information collected through this evaluation is necessary to assess how the Standards are currently being used by health and health care organizations, and is especially paramount given the lack of any such comprehensive study since the Standards’ original launch in 2001. Specifically, this evaluation is an attempt to describe the complex processes of awareness, knowledge acquisition, adoption, and implementation in health and health care organizations, using a small number of information-rich health and health care organizations as cases to identify: how the National CLAS Standards are implemented, the barriers and facilitators of organizational implementation of the National CLAS Standards, and intended outcomes of implementation.

In addition to the valuable knowledge gained from this evaluation, the results will inform the development of descriptive technical assistance materials, and preliminary support for the identification of promising practices, related to the implementation of the National CLAS Standards. The technical assistance materials, in particular, will serve as practical tools to guide health and health care organizations in their efforts to provide quality care to the culturally and linguistically diverse populations they serve through implementing the National CLAS Standards.

This evaluation supports the Office of Minority Health within the Office of the Secretary of the Department of Health and Human Services (HHS/OS/OMH), and is consistent with activities described in the HHS Action Plan to Reduce Racial and Ethnic Health Disparities, and the National Stakeholder Strategy for Achieving Health Equity.

2. Purpose and Use of Information Collection

The purpose of this evaluation is to systematically examine and document the process of awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards. Information will be collected directly from assessment participants through either a telephone or in-person interview, an online survey, or a web-based form. The information is needed from participants, who are knowledgeable of and/or currently utilize, the Standards, to provide a rich description of the complex processes of awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards in non-federal health and health care organizations.

Thirteen overarching evaluation questions are guiding this evaluation, each related to an aspect of awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards (See Appendix A). The information to be collected in this assessment will be collected from five categories of respondents. The respondents include: 1) National CLAS Standards stakeholders (individuals directly involved in the development of the original and/or enhanced National CLAS Standards), 2) CLAS Stakeholders (recognized experts on culturally and linguistically appropriate services (CLAS) and/or the National CLAS Standards), 3) leadership and senior executive members (e.g., CEOs, COOs, Director of Human Resources, designees, etc.) of up to 20 selected health and health care organizations that self-identify as a “CLAS adopter” and meet other inclusion criteria as outlined in Section B , 4) key staff in health and health care organizations that self-identify as a “CLAS adopter” and 5) Think Cultural Health registrants (users of the Think Cultural Health (TCH) website with knowledge and/or experience in using the National CLAS Standards in a health or health care setting). Individual-level information collected includes demographic information such as sex, race, ethnicity, current position/work title, and years in their position, and questions examining the respondent’s actual and perceived awareness, knowledge, adoption, and implementation of the National CLAS Standards among health and health care organizations. Organizational-level information collected will include descriptive information such as the type of health or health care organization, location of the organization, size of the organization, and information about the organization’s adoption and implementation of the National CLAS Standards. Please see the appendices for the evaluation questions, the full data collection instruments for each category, and a crosswalk of the evaluation questions, data collection instruments, and data analyses.

The information generated from this evaluation will have multiple applications. It will be used by OMH to develop and disseminate technical assistance on implementation of the National CLAS Standards for health and health care organizations. The information collected from this project will also be used to identify strengths and areas of improvement for policy surrounding the Standards themselves, and will inform the future promotion and dissemination of the Standards to health and health care organizations. Information from this evaluation will also be used to contribute to the body of literature in the field of culturally and linguistically appropriate services and about the National CLAS Standards, of which there is a paucity.

3. Use of Improved Information Technology and Burden Reduction

To obtain the information necessary for this assessment to meet its goals and objectives, while simultaneously minimizing the burden on participants, multiple data collection approaches will be used, consistent with a concurrent (convergent) mixed-methods design. Given the exploratory and process-oriented nature of the assessment and evaluation, key informant interviews lasting no more than 60 minutes will be conducted via telephone or in-person with National CLAS Standards and CLAS Stakeholders, and key informant interviews lasting no more than 90 minutes will be conducted via telephone or in person with Health and Health Care Leadership in up to 20 selected information-rich health and health care organizations. Collecting information in this time span will enable us to gather the rich, descriptive, and accurate type of information needed to describe appropriately the complex process of adoption and implementation of the National CLAS Standards. In addition, a survey, designed to take no more than 30 minutes to complete, will be administered to key staff within up to 20 selected health and health care organizations that identify themselves as “CLAS adopters” via a web-based survey platform. The online survey method was selected to minimize burden upon respondents. Finally, a web-based form, designed to take no more than 10 minutes to complete, will be available on the Think Cultural Health website soliciting voluntary feedback on National CLAS Standards’ implementation from registered users of the site.

4. Efforts to Identify Duplication and Use of Similar Information

A review of the literature currently shows little research that comprehensively examines the public awareness, organizational adoption, and implementation of the National CLAS Standards among health and health care organizations in the United States. Though there are some examples of studies describing one facet of these processes (e.g., implementation), they are limited in their scope of information provided and their generalizability. In 2004, for example, ORC Macro conducted a study on the implementation of the National CLAS Standards at the Alameda Alliance for Health, a managed care organization in Alameda, California. However, because this assessment and evaluation used a single-case study approach, it limited the generalizability and applicability of results to the diversity of types of health and health care organizations across the United States. The proposed assessment project attempts to expand both the number of types of health and health care organizations examined, and the processes related to implementation of the National CLAS Standards that are explored. Therefore, the information collected through this assessment and evaluation is not duplicative of other information collection.

Currently, there is no existing repository of this information that can be leveraged to substantially inform or conduct the assessment and evaluation.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

6. Consequences of Collecting the Information Less Frequent Collection

Information will be collected only once from all respondents; no data will be collected from a respondent periodically. Depending upon respondent category, each individual who agrees to participate will complete one telephone or in-person interview, a web-based survey, or a web-based form. If this data is not collected, OMH will have difficulty understanding and demonstrating how and to what extent the National CLAS Standards have been implemented by health and health care organizations. Furthermore, it will not be possible to answer pressing questions about whether the enhanced National CLAS Standards achieve their expected outcomes (i.e., advancing health equity, improving quality, and reducing health care disparities) without first understanding current organizational awareness, knowledge acquisition, adoption, and implementation of the Standards. There are no increases of burden for respondents. There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

Given the variety of health and health care organization typologies, and the diversity of organizations within each typology (e.g., geographic location, patient demographics, payor mix, etc.), the results of this evaluation are not intended to be generalized across the entire universe of health and health care organizations within the United States. The aim of this evaluation is to examine and document the awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards among a select number (up to 20) of information-rich non-federal health and health care organizations to describe and inform the development of technical assistance materials related to the implementation of the National CLAS Standards, both of which can be used by health and health care organizations. Beyond this aspect, there are no special circumstances that would result in data collection outside of this process.

8. Comments in Response to the Federal Register Notice/Outside Consultation

The following sentence will suffice: A 60-day Federal Register Notice was published in the Federal Register on September 26, 2014, vol. 79, No. 187; pp. 57930-31 (see attachment K). There were no public comments.

The information collection instruments were developed in 2014 in consultation with OMH and a 5-person Advisory Group consisting of content and evaluation experts. The Advisory Group members included:

• Jessica Briefer French

Senior Research Scientist, National Committee for Quality Assurance

202-955-1776

[email protected]

• Iris Garcia-Caban, Ph.D.
  Executive Office Health Human Services

Medicaid (MassHealth) Office of Providers & Plans
617-847-6528

[email protected]

• Robert Like, M.D.

Professor and Director, Center for Healthy Families and Cultural Diversity 
Department of Family Medicine and Community Health
Rutgers Robert Wood Johnson Medical School 

732-235-7662

[email protected]

• Francis Lu, M.D.

Luke & Grace Kim Professor in Cultural Psychiatry, Emeritus

University of California, Davis

916-708-5542

[email protected]

• Matthew Wynia, M.D.

Director, Center for Bioethics and Humanities

University of Colorado Health Sciences Center

312-464-7980

[email protected]

In summary, Advisory Group feedback included clarifications of interview prompts, explicitly differentiating between the two versions of the National CLAS Standards (2001 and 2013), adding and revising probes and wording, clarifying which leadership will be targeted for health and health care organization interviews, and considering global questions versus nuanced, standard-specific questions in light of interview length and burden.

The instruments were further refined based on the results of a pilot study.

9. Explanation of any Payment/Gift to Respondents

No payments or gifts are provided to respondents.

Health and health care organizations that participate in the assessment and evaluation will receive an aggregate summary of the anonymous responses from key staff who complete the survey, which may be useful to the organization as it plans and monitors activities related to the adoption and implementation of the National CLAS Standards.

10. Assurance of Confidentiality Provided to Respondents

There is no assurance of confidentiality provided to respondents; however, data will be kept private to the extent allowed by the law. This project has been reviewed by the George Mason University Institutional Review Board, and has been considered exempt per category #2 (“Some research involving educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures, or observation of public behavior.”)

11. Justification for Sensitive Questions

Per HHS data collection requirements, the sex, ethnicity, and race of respondents are requested.

12. Estimates of Annualized Hour and Cost Burden

Data will be collected only once from each participant. Based on estimates from conducting a pilot test with a sample of nine individuals total (three in each category) of the interview protocols and measures to be used in this evaluation, it is expected that it will take no more than 1 hour for respondents to complete the interviews, no more than 15 minutes to complete the online staff survey, and no more than 10 minutes to complete the National CLAS Standards Experience Form. On average, it took approximately 45 minutes (0.75 hours) to complete the National CLAS Standards Stakeholder Interview; 1 hour to complete the CLAS Stakeholder Interview; and 1 hour to complete the Health and Health Care Organization Leadership Interview. On average, the Health and Health Care Organization Staff Survey took 15 minutes (0.25 hours) to complete. On average, the National CLAS Standards Experience Form took approximately 10 minutes (0.17 hours) to complete. The Health and Health Care Organization Screener survey is estimated to take approximately 5 minutes (0.08 hours) to complete. The maximum respondents for the National CLAS Standards Stakeholder and CLAS Stakeholder interviews will be 30% of the total number of individuals within each Stakeholder group. The maximum respondents for the Health and Health Care Organization Leadership Interview will be seven (7) leadership from each of the 20 participant organizations (five organizations from each of the four typologies), generating a total of 140 respondents. The respondents for the Health and Health Care Organization Staff Survey is an estimated total staff size of 125 from each of the 20 participant organizations (five organizations from each of the four typologies), generating a total of 2,500 total respondents. The respondents for the screener survey are an estimated total of 50,000. The maximum respondents for the National CLAS Standards Experience Form are approximately 240,000 (total number of newsletter registrants). Data will be collected only once from each participant.

12A. Estimated Annualized Burden Hours

Form Name	Type of Respondent	No. of Respondent	No. Responses per Respondent	Average Burden per Response (hours)	Total Burden (hours)
National CLAS Standards Stakeholder Interview	Training and Development Specialists and Managers; Other Management Occupations	21	1	45/60	16
CLAS Stakeholder Interview	Training and Development Specialists and Managers; Other Management Occupations	21	1	1	21
Health and Healthcare Organization Leadership Interview	Health and Health Care Organization Executives and Managers	140	1	1	140
Health and Health Care Organization Staff Survey	Health and Health Care Providers, Managers, and Support Staff	2,500	1	15/60	625
Health and Healthcare Organization Screener Survey	Health and Health Care Organization Executives	50,000	1	5/60	4,167
National CLAS Standards Experience Form	Health Care Practitioners and Technical Occupations	240,000	1	10/60	40,000
TOTAL					44,969

12B.

This is not an annual collection. This is a one-time collection used to inform an assessment and evaluation. Though there are no annual burden costs to participants, a one-time cost can be calculated.

The assessment does not request salary information of participants as part of the data collection; therefore, figures were based on statistics provided by the Department of Labor, Bureau of Labor Statistics.

Using this resource, it was determined that the national average hourly wage for the types of participants in the National CLAS Standards Key Stakeholder interview is $41.78. The average participant is expected to complete the interview in 45 minutes (0.75 hours), thus making the cost of burden $41.78 x 0.75 hours = $31.34; this is a one-time total respondent cost burden per interview.

It was determined that the national average hourly wage for the types of participants in the CLAS Stakeholder interview is $41.78. The average respondent is expected to complete the interview in 60 minutes (1 hour), thus making the cost of burden $41.78 x 1 hour = $41.78; this is a one-time total respondent cost burden per interview.

It was determined that the national average hourly wage for the types of participants in the Health and Health Care Organization Leadership Interview is $61.53. The average staff respondent is expected to complete the interview in 60 minutes (1 hour), thus making the cost of burden $61.53 x 1 hour = $61.53; this is a one-time total respondent cost burden per interview.

It was determined that the national average hourly wage for the types of participants in the Health and Health Care Organization Staff Survey is $32.34. The average staff respondent is expected to complete the survey in 15 minutes (or 0.25 hours), thus making the cost of burden $32.34 x 0.25 hours = $8.09; this is a one-time total respondent cost burden per survey.

It was determined that the national average hourly wage for the respondents to the screener survey is $61.98. The average staff respondent is expected to complete the survey in 5 minutes (or 0.08 hours), thus making the cost of burden $61.98 x 0.08 hours = $4.96; this is a one-time total respondent cost burden per survey.

It was determined that the national average hourly wage for the types of participants in the National CLAS Standards Experience Form is $48.72. The average staff respondent is expected to complete the survey in 10 minutes (or 0.17 hours), thus making the cost of burden $48.72 x 0.17 hours = $8.28; this is a one-time total respondent cost burden per survey.

Type of Respondent	Total Burden (hours)	Hourly Wage Rate (Mean)	Total Respondent Costs
Training and Development Specialists and Managers; Other Management Occupations	16	$41.78	$668.48
Training and Development Specialists and Managers; Other Management Occupations	21	$41.78	$877.38
Health and Health Care Organization Executives and Managers	140	$59.59	$8342.60
Health and Health Care Providers, Managers, and Support Staff	625	$32.34	$20,212.50
Health and Health Care Organizations Executives	4,167	$71.98	$299,940.66
Health Care Practitioners and Technical Occupations	40,000	$48.72	$1,948,800
TOTAL			$2,278,841.62

13. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs

There is no burden of cost to registrants/users as the collection participation is voluntary and free to registrants/users. HHS OMH assumes the burden of cost pertaining to the maintenance and acquisition of necessary equipment (i.e., computer hardware and software) associated with the collection.

14. Annualized Cost to Federal Government

The total cost to the Federal Government for the implementation of this assessment project by the contractor, SRA International, will be approximately $139,622.00 per year for two years for a cumulative total of $279,244.00. This amount was arrived at by calculating the number of hours respective staff will spend working on this project. It is estimated that staff whose hourly rates (loaded) to the government range from $55 to $75 will use a total of 1294 hours per year. Also included in the total amount are consultant and travel costs.

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Interviews with National CLAS Standards stakeholders, CLAS stakeholders, and health and health care organization leadership will be recorded with a digital tape recorder and transcribed verbatim to enable analysis and interpretation. The data will be coded independently by members of the project staff using Nvivo software to identify structural and thematic codes and to facilitate review of any memos or other documentation collected from the interviews and site visits (if applicable). Data analysis will include immersion in the audiotapes and notes from the interviews; documenting awareness of nuance and meaning through coding for common themes and unique ideas between and across target audiences; and explaining and synthesizing opinions and experiences expressed by interview participants. The focus of analyses will be to develop robust descriptions of the phenomena of adoption and implementation in health and health care organizations, and to help assess and establish the transferability, credibility, dependability, and validity of the implementation process and the project’s findings. The qualitative analyses will also be used to contextualize the data collected through the health and health care organization staff surveys. These data sources will be aggregated to provide an increased, in-depth understanding of National CLAS Standards adoption and implementation among a small number of different types of health and health care organziations. The findings will be triangulated with other data available (e.g., site visit notes/memos, document review) for convergence validation.

Quantitative data will be analyzed using STATA software. Summary statistics (i.e., descriptive statistics such as percentages, means, medians, modes, and standard deviations) and missing data diagnostics will be computed for all variables. Outliers will be evaluated for elimination and missing data will be examined prior to any modeling. Frequencies will also be computed and, to the extent allowable, will be compared across various response categories or variables (i.e., comparative frequencies of responses to items by different types of health care organization leaders, health care organization demographics, etc.). Descriptive analysis of the organizational demographic characteristics will be examined.

After qualitative and quantitative data is analyzed separately, the data will then be merged using data comparison and data transformation techniques. Specifically, the qualitative and quantitative data will be compared and contrasted based on evaluation questions and overall evaluation objectives. Similarities and differences will be organized and displayed. If necessary, one type of data will be transformed into the other type of data, to facilitate additional comparative analysis and interpretation.

All analyses will be directed toward answering the stated evaluation questions, and will be reported as part of the evaluation findings.

The SRA-HDDP Evaluation Team will report on the progress of this evaluation effort in monthly HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee meetings, semiannual reports, and annual reports submitted to the HHS OMH. Upon HHS OMH review and approval, progress and results from this evaluation project will also be shared with key CLAS and National CLAS Standards stakeholders (e.g., the National CLAS Standards National Project Advisory Committee members), and posted on the HHS OMH’s Think Cultural Health website (thinkculturalhealth.hhs.gov). By posting the assessment and evaluation progress and results on the Think Cultural Health website, the progress and findings of this assessment and evaluation will also be publically available.

In coordination and conjunction with the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee, the SRA-HDDP Evaluation Team will also convene informational and briefing meetings to share and discuss the findings of the evaluation with HHS OMH leadership and staff, and will present progress and findings to the HHS OMH National CLAS Standards Implementation Team.

In consultation with and approval by the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee, the SRA-HDDP Evaluation Team will discuss and pursue appropriate vehicles to disseminate project findings, including publications, presentations at local and national conferences, webinars, and presentations to health and health care organizations and stakeholders. Contingent upon this, the SRA-HDDP Evaluation Team will work with the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee to develop and implement a dissemination plan for project findings, including a plan for submission of peer-reviewed publications. For the manuscript development process, SRA-HDDP Evaluation Team will compile the findings of the evaluation and submit at least two manuscripts for peer-review and publication to appropriate academic and/or professional journals in collaboration with the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee. Upon HHS OMH review and approval, findings will also be shared in a pre-determined medium and format with other federal, national, regional, state, tribal, and community stakeholders, including policy briefs and technical assistance documents.

See Appendix J for the project time schedule.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

2. Collection of Information Employing Statistical Methods

The information collection is a voluntary process for registrants/users registering on the TCH website to receive a newsletter and to complete training.

For this collection, item 17 on OMB 83-1 form is “No”. Therefore, section B of the supporting statement is not applicable.

Attachments to the Supporting Statement

1. Overarching Evaluation Questions

2. National CLAS Standards Evaluation Question/Instrumentation Crosswalk

3. National CLAS Standards Stakeholder Interview

4. CLAS Stakeholder Interview

5. Health & Health Care Organization Leadership Interview

6. Health & Health Care Organization Staff Survey

7. National CLAS Standards Experience Form

8. Health & Health Care Organization Screener

9. Assessment Informed Consent Form

10. Assessment and Evaluation Plan Timeline

10