Cmhs Mhcpe Ss_b 2.21.14

PARTICIPANT FEEDBACK FORMS FOR THE

MENTAL HEALTH CARE PROVIDER EDUCATION

IN HIV/AIDS (MHCPE) PROGRAM

SUPPORTING STATEMENT

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

CMHS funds three mental health professional associations that are required to train approximately 1,000 professionals each year (potentially a total of 10 education sites may be funded). Therefore, approximately 3,000 (10,000) participants will be trained. A review of the types of mental health providers served by previously funded education sites suggests that a wide range of participants can be expected. Types of trainees are likely to include social workers, psychologists, psychiatrists, nurses, clergy, counselors, non-psychiatric physicians, and other health and non-health care workers.

All training participants will be asked to provide feedback on the training sessions they attend. No sampling procedures will be employed. This approach is consistent with other types of program assessment for education programs and is consistent with the approach used in the earlier with the CMHS HIV/AIDS education program. The reasons for collecting feedback on the entire population include:

• The importance of fully assessing which types of mental health providers attend the training sessions, their demographic characteristics, the types of HIV-related services they provide and the types of HIV- and AIDS-affected clients they serve;

• The limited number of education sites to be funded;

• The diversity of sites in terms of geographic location and primary affiliation (e.g., university vs. community based);

• The differing training venues at which trainings are delivered;

• The differing training methods that are employed; and

• The variation in topics likely to be covered in the training sessions.

In order to maximize response rates, the participant feedback forms were designed to collect the minimum amount of information necessary for CMHS to address the assessment questions. Additional specific considerations for maximizing responses are discussed in Section B3. A limited review of currently operating education sites indicates that the response rates vary by training session size and venue, with smaller training sessions having higher response rates. Based on the experience of the earlier CMHS MHCPE Programs, the overall estimated response rate remains high with most sites, near or over 80%, with one that is lower, due to the specific context of the in-hospital grand-rounds training setting. The information collected will be used to solicit feedback for improving the training sessions and HIV/AIDS education program, as well as to get feedback on the usefulness of the education training to participants. During the training meeting, education site staff and other designees receive instructions from the CMHS Government Project Officer and the CMHS subcontractor on the administration of the participant feedback forms and the submission of forms for processing.

2. Information Collection Procedures

Feedback will be collected from all participants that attend training sessions conducted under the MHCPE Program. Participants will be asked to complete feedback forms based on the type of training session they attended. Table 2 in Section A summarizes the overall data collection strategy. The data collection strategy proposed for use will be the same as the strategy used in the current MHCPE Programs.

As illustrated in Table 2, education site staff will complete a Session Report Form that describes the training environment for all training sessions delivered. All trainees will be asked to complete a participant feedback form at the end of the training session.

For over 10 years of the MHCPE Program (beginning Sept. 1999), and the current cycle, the CMHS Office’s subcontractor collects and processes the feedback forms. The subcontractor creates electronic versions of the scanned data and distributes data-based reports to education sites on a quarterly basis.

3. Methods to Maximize Response Rates

A limited field test of procedures and instruments was conducted at the inception of the CMHS MHCPE II Program to assess the feasibility of administering that multi-site effort. One of the specific aims of the field test was to solicit information from education site staff, trainers, evaluators and training participants regarding methods for maximizing the response rates. One component of the field testing procedures involved administering the instruments to less than 10 training participants and then conducting a focus group to solicit comments regarding: (1) the likelihood of obtaining responses for selected items, and (2) methods of increasing the response rate of survey respondents. This process yielded valuable information that CMHS has implemented to maximize response rates and the usefulness of the information requested. To maximize response rates, CMHS, in the previously funded MHCPE Programs:

• Ensured that the questions on the multi-site feedback forms are the minimum needed to address the CMHS research questions;

• Ensured that the multi-site feedback forms were as user-friendly as possible and contained easy-to-read font, logical layout and straightforward language;

• Provided clear instructions for all sections of the surveys;

• Used culturally sensitive questions that are unlikely to be perceived as offensive or compromising to the respondents’ values and belief systems; and

• Used culturally sensitive questions that are unlikely to be perceived as offensive or compromising to the respondents’ values and belief systems.

The MHCPE Program has used the participant feedback forms for over 10 years. In 2012, the Program piloted the use of electronic feedback collection forms for trainings conducted online in addition to using pen and paper forms for in-person trainings. These forms were developed to maximize response rates.

4. Tests of Procedures

Prior to their use in the MHCPE Program, the initial assessment design and participant feedback forms were pilot-tested on a small sample of less than 10 individuals to ensure that the multi-site assessment requirements and procedures were consistent with activities conducted at education sites. The field-testing was designed to collect information on the overall evaluation design and draft feedback forms. Comments on the draft feedback forms included collecting information on the likelihood of obtaining specific responses, overall instrument layout, item flow, and administration times. Feedback on the overall design included collecting information on the:

• Overall feasibility of administering feedback forms at sessions of varying lengths;

• Feasibility of trainers and staff administering feedback forms;

• Anticipated challenges in submitting information to Coordinating Center;

• Identification of activities required to coordinate multi-site data collection activities at the local level;

• Recommendations for alleviating the data collection burden;

• Recommendations for ensuring that the feedback forms are gender, age and culturally sensitive;

• Recommendations for improving overall design for soliciting feedback from participants; and

• Utilization of the electronic form to ensure effectiveness.

As discussed in Section A2, several outdated and rarely-used response options for all participant response forms and the session reporting form were removed from the individual data collection tools. No new questions or response options have been added to any of the data collection forms.

Based on the feedback provided to CMHS by the MHCPE education sites, no further revisions are requested.

CMHS is proposing the continued use of the minimally revised post-session only participant feedback forms, in the current post-session feedback design, as approved by OMB in 2011. The current assessment design and the participant feedback forms are providing CMHS with invaluable information to inform quality improvement efforts. Further, the assessment data enables CMHS to monitor progress in meeting programmatic goals of educating providers of mental health services for HIV and AIDS-affected individuals and enhancing the nation’s ability to have an impact on the HIV/AIDS epidemic.

5. Statistical Consultants

The names and phone numbers of statistical contacts, individuals responsible for collecting and analyzing the data and responsible agency personnel are provided below, in Table 7.

Table 7: Data Collection Personnel, Analysts, Statistical Consultants and Responsible Agency Personnel

Education Site Directors
Organization	Contact	Title	Telephone
American Psychological Association	David DeVito, MPA	Project Director	(202) 216-7603
American Psychiatric Association	Diane Pennessi, M.D.	Project Director	(703) 907-8668
National Association of Social Workers	Evelyn Tomaszewski, ACSW	Project Director	(202) 336-8390

Statistical Consultants
Organization	Contact	Title	Telephone
Abt Associates	Liza Solomon, MHS, DrPH	Project Director	(301) 347-5785
Agency Personnel Responsible for Deliverables

Organization	Contact	Title	Telephone
CMHS	Ilze L. Ruditis, MSW	Government Project Officer, CMHS	(240) 276-1777

LIST OF ATTACHMENTS

Attachment A: Instructions for Administering Forms, Session Report Form and Multi-Site Feedback Forms

Instruction for Administering the Session Report Form

Session Report Form

Instructions for Administering the Participant Feedback Forms

Adherence Participant Feedback Form

Ethics Participant Feedback Form

Neuropsychiatric Participant Feedback Form

Participant Feedback Form

NOTE TO OMB REVIEWER

The participant feedback forms submitted for OMB approval have been used for the over 10 years in the CMHS MHCPE Programs. The only changes made to the forms (delineated within Section A2) have been the deletion of some outdated and rarely-used response options. The revised forms will be used for the duration of the current program, which is set to expire in September 2014, and for the future programs.

The participant feedback forms contain several sections that are identical across the different types of forms in order to enable data to be pooled and compared across sites and types of training sessions. Detailed explanation of the similarities and differences across the forms is provided below to facilitate the review.

• The instructions for completing all participant feedback forms are the same except for the estimated completion time, which varies according to the instrument’s length.

• For all of the Participant Feedback Forms (i.e., General, Adherence, Neuropsychiatric, Ethics) items on the first page (items 1 through 11) are the same.

• For the General, Adherence and Neuropsychiatric Participant Feedback Forms, items 12 through 19 are identical.

• The Session Report Form contains completely unique items. None of its items appear on the participant feedback forms.

7