0990-Eval of HIV Prev Prog for Young Women attending Minority Institutions final Sept 3 2008

Supporting Statement for Request for Clearance:

HIV PREVENTION PROGRAM FOR YOUNG WOMEN ATTENDING MINORITY INSTITUTIONS

Contact Information:

Adrienne M. Smith, PhD, MS, CHES

Public Health Advisor

Office on Women's Health

Office of Public Health Science

U.S. Department of Health and Human Services

200 Independence Avenue, SW, Room 733E

Washington, DC 20201

202-690-5884

202-690-7172 fax

[email protected]

July 1, 2008

SUPPORTING STATEMENT

HIV PREVENTION PROGRAM FOR YOUNG WOMEN ATTENDING

MINORITY INSTITUTIONS

A. JUSTIFICATION

A.1 Need and Legal Basis

In 1998, as the result of the HIV/AIDS state of emergency declared by African American community leaders and supported by the Congressional Black Caucus (CBC), Congress funded an initiative to address this crisis through increased funding and outreach. This initiative would eventually become known as the Minority AIDS Initiative (MAI). These funds targeted HIV/AIDS programs that directly benefit racial and ethnic minority communities. The MAI is not a part of the Ryan White Care Act authorizing legislation, but provides directed resources to some CARE Act programs, as it does to other Public Health Service HIV/AIDS programs.

The MAI takes a multi-faceted approach that focuses simultaneously on HIV prevention, care, treatment and research. Further, within these broad categories of funding, MAI funds direct services, technical assistance, training and capacity-building, and evaluation.¹ Legislation that gave birth to MAI came as a result of the Omnibus Consolidated and Emergency Supplemental Appropriations Act of 1999, PL 105-277, October 21, 1998 and was initially referred to “CBC” initiative. A copy of the legislation may be found at the following website: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=105_cong_public_laws&docid=f:publ277.105.pdf

The current data collection is being requested for a three-year evaluation of the HIV Prevention Program for Young Women Attending Minority Institutions. This is a new data collection requesting OMB approval. Due to the concern that rates of HIV infection among young women of color are increasing, the Department of Health and Human Services (HSS), Office on Women’s Health (OWH) is seeking to evaluate its funded HIV prevention programs at Minority Institutions in order to identify best practices and the most effective gender-centered approaches to HIV/AIDS prevention. Three categories of Minority Institutions will participate in this data collection: Historically Black Colleges and Universities (HBCUs); Hispanic Serving Institutions (HIS); and Tribal Colleges and Universities (TCU). (See Appendices A to A.3 for background information on OWH, its programs and women and HIV/AIDS).)

The primary goal of OWH’s Minority Institutions Program is to increase knowledge and reduce infection rates among young, college-aged, minority women. The program goals are to identify effective methods to educate and increase awareness for prevention of HIV/AIDS and STDs infection, develop a capacity for young, minority women to address the prevention education on campus, establish partnerships and student organizations to increase health education, risk-reduction, counseling, and HIV/STD testing, and to ensure that the health education is culturally and linguistically appropriate for young, minority women. In addition, the program aims to increase campus activities that serve women at risk of infection, increase knowledge of prevention, and improve HIV prevention education on campus.

OWH will use the evaluation findings in making programmatic and funding decisions about their HIV Prevention Program for Young Women Attending Minority Institutions. Evaluation findings will also be used to identify best practices and lessons learned that could be transferred to other organizations and communities attempting to replicate care for women. Furthermore, because this program was developed as part of the national effort to eliminate health disparities, evaluation findings will help OWH understand how the program is contributing to closing the health disparities gap and to improving care for underserved populations of women across the country. (See Appendix B for a summary of how the data collection (evaluation) meets OWH goals.)

A.2 Information Users

The purpose of this information collection is to gather data from institutional grantees participating in OWH’s Women and HIV/AIDS program related to program efficacy. Information obtained from this data collection will be used to assist in measuring the effectiveness of OWH efforts to reduce risk of acquiring HIV among women and to increase HIV prevention knowledge among women. All data collection forms and activities are designed within the parameters of a three-year evaluation of the HIV Prevention Program for Young Women attending Minority Institutions.

This evaluation will enhance OWH’s capacity to identify, support, and create effective HIV prevention programs for women. This data collection will also help to improve OWH’s knowledge of gender sensitive methods to reduce risk of contracting HIV and increase HIV prevention knowledge among college attending minority women. In addition to informing the development of current OWH HIV prevention programs, the evaluation results will also aid in the planning and development of future OWH and other public and private sector HIV prevention programs.

Failure to collect this information will have negative consequences on HIV prevention efforts among women and women of color. These data permit OWH to enhance its knowledge of effective program planning, development and delivery and to continue to work toward eliminating racial and ethnic disparity among women. The data will be

used to increase OWH’s knowledge base of gender-centered intervention models, provide guidance to funded programs and develop best practices for HIV prevention programs funded in the future.

Overall, the evaluation of OWH’s HIV Prevention Program for Young Women attending Minority Institutions will assess the effectiveness of this program in delivering gender - centered intervention models. Data will be collected by surveys and biannual interviews. There are four potential respondent types involved in the HIV prevention programs at minority institutions. They are college students (women), program directors, program staff, and peer educators. All schools do not have peer educators. This type of evaluation methodology assesses program effectiveness in a manner which is conclusive and efficient.

The DHHS-Office on Women's Health intends to use the evaluation results of the HIV/AIDS programs to address the PART deficiencies indicated by the Office of Management and Budget in 2004. The evaluation will address several of the objectives for program management, strategic planning and program results. Additionally, the evaluation results are critical to measuring the efficacy of the use of government funds.

A.3 Improved Information Technology

Program staff will be collecting data on an ongoing basis from college students and will be reporting information to OWH quarterly. To reduce respondent burden, OWH will create and distribute to respondents an Excel data management file. Respondents will easily be able to input required data into this system and email them to OWH and Global Evaluation and Applied Research Solutions (GEARS) Inc., the contractor engaged to conduct the evaluation. GEARS will use the data to compile quarterly analyses to generate its quarterly report to OWH.

The process evaluation assessments will be administered by GEARS staff via personal or telephone interview with the program directors, staff and peer educators of the funded grantees. (See Appendices C, D, and E.) The process evaluation questions require answers in a narrative format in order to obtain a comprehensive and coherent answer to process evaluation questions. Conducting either an in-person or telephone interview requires less of the respondents’ time than composing and typing narrative for an online process interview. Responses recorded by GEARS staff will be entered into a qualitative software program for data analysis and retrieval.

A.4 Duplication of Similar Information

No effort to collect similar data is being conducted within the agency. Additionally, no data collection efforts outside the agency have been made to collect this data. The respondents are participants in a new OWH program and the data are specific to the evaluation of this program.

A.5 Small Businesses or Other Small Entities

This data collection involves the collection of information from small businesses or other small entities. We have created the data collection methodology to include the minimal amount of information required to effectively evaluate the program. Additionally, as much as possible, we have attempted to collect data that respondents will currently need to maintain and locally evaluate their programs. Therefore, small business and other small entity respondents (minority institutions) will primarily submit information that they needed to collect for their own purposes. Grantee proposals were reviewed to ascertain what program participant data grantees planned to normally collect as part of their program implementation. This data, along with data needed for the proposed data collection, was integrated into a data collection instrument. This data collection instrument was designed, as much as possible, to request and utilize information that programs (i.e., grantees) collect as part of their required in-house evaluation activities and that imposes the minimal amount of burden as possible. The data collection requirements and survey questions have been held to the absolute minimum required for the intended use of the data. Each grantee has reviewed and approved the proposed data collection form regarding information on program participants.

Additionally, during the grantee orientation meeting and in a subsequent day long meeting, the evaluation team met with all grantees and discussed the least burdensome data collection mechanism and frequency of reporting. Most grantees supported submitting data electronically on a quarterly basis. The grantees supported development of forms that maintain all of the information they currently collect and submission of data electronically. Directors of the awarded grants also indicated the positive impact that the evaluation results will have on their program effectiveness and opportunities for future funding.

A.6 Less Frequent Collection

This is a one-time data collection effort with four respondent types program directors, program staff, and peer educators and program participant. Approval is sought for three years.

We are requesting that grantees report information quarterly. We will collect two kinds of information from grantee staff (i.e., program directors and program staff). The first type of information is information that they collect from students that will be shared with OWH and the second type of information is information collected from grantee staff regarding their program and activities. Information collected about students from grantee staff will be collected on a quarterly basis. This information is collected at this frequency in order to allow grantee staff time to verify and enter collected data. A less frequent data collection would increase the probability of errors.

Information collected about the program from grantee staff will be collected twice a year from program directors and program staff and once a year from peer educators. GEARS will collect program information from grantee staff. This frequency is requested in order to assess program changes during the course of the grant. A less frequent data collection increases the probability that grantee staff may not recall as thoroughly programmatic information critical to the evaluation.

If this information collection is not conducted, OWH’s ability to accurately measure and evaluate the impact of this program against its stated objectives will be negatively affected. Failure to include these data collection activities as part of the overall evaluation design will limit the validity of the results and negatively impact the health of women. There are no legal obstacles to reduce respondent burden.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5(d)(2)

The proposed evaluation fully complies with all guidelines of 5 CFR 1320.5 (d) (2). The information collection will not be conducted in a manner:

• Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• Requiring respondents to submit more than an original and two copies of any document;

• Requiring respondents to report more often quarterly;

• Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

• In connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of the study;

• Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or,

• Requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

The data collection notice for the Evaluation of the HIV Prevention Program in Women was published in the Federal Register, volume 73, number 124, page 36326 on June 26, 2008. A copy of the Federal Register notice is included as Appendix F. There were no comments received from the public regarding this data collection.

The DHHS/OWH Project Officer for this data collection is Adrienne Smith and the Project Officer for the HIV Prevention Program for Young Women attending Minority Institutions is Mary L. Bowers. Additionally, OWH engaged the consulting firm Global Evaluation & Applied Research Solutions (GEARS), Inc to assist in the development of the survey instruments and evaluation methodology for this evaluation. GEARS is experienced in managing and conducting evaluations and provided expertise on issues including the availability of data, frequency of collection, clarity of instructions, record keeping, confidentiality, disclosure of data, reporting format, and necessary data elements. Also, in 2007 GEARS completed the OWH evaluation of its Rural South, Incarcerated/Newly Released and Mentoring Partnership programs. This evaluation was approved by OMB.

A.9 Explanation of Any Payment or Gift to Respondents

There will be no payment, gift, or reimbursement to respondents for time spent.

A.10 Assurance of Confidentiality Provided to Respondents

The proposed evaluation does not collect identifying, personal data and all information collected from grantees on students is de-identified. Grantee program staff responsible for survey administrator will inform all potential respondents (students) of the purpose of the survey, how the information collected will be used, and that no personal identifiers will be associated with their responses. The Prevention Education Questionnaires (Pre-test, Post-test, Follow-up; See Appendices G, H and I respectively) administered to students by grantees will not collect student names or other identifiers that will allow survey responses to be linked to individual participants. Respondents will use a formula to develop their unique identifier (See Appendix J). Information will be kept private to the extent possible by law.

Evaluation activities occurring on the campus of each Minority Institution (grantee) will be conducted after approval has been obtained by an Institutional Review Board (IRB). Each institution will submit a protocol to their IRB and if a grantee has no IRB, GEARS will submit a comprehensive protocol to its IRB. No data will be collected without IRB approval. Respondents will sign a consent form approved by the Minority Institution’s IRB before taking the Prevention Education Questionnaires. This consent form will assure grantees and students that their services will not be adversely impacted by their decision not to participate in the survey.

The questionnaires will be administered to program participants (students) by grantee staff. Survey responses will be coded, entered into a database and electronically submitted to GEARS to use only for data analytic and evaluation purposes. In addition, GEARS will maintain the online Prevention Education Follow-up Questionnaire for College Women. Respondents to this online questionnaire will provide their unique identifier so that their responses can be linked to their pre and post-test. GEARS will use Survey Monkey as the online survey vehicle and will select the option that IP addresses of respondents not be saved. The evaluation team will ensure that no identifying information is shared with any entities outside of OWH to the extent allowed by law.

A.11 Justification of Sensitive Questions

This evaluation asks sensitive questions. These questions represent standard techniques used in public health practice in assessing the burden of HIV/AIDS on communities and populations and in assessing the public’s level of knowledge and risk behaviors. Moreover, all questionnaires used in the evaluation would have been reviewed by an Institutional Review Board to ensure that respondents’ rights are protected.

Sensitive questions are asked on the three versions of the Prevention Education Questionnaire for College Women, (i.e. the pre-test, post-test, and follow-up; see Appendices G, H and I). The sensitive questions asked on the Pre-test version of the questionnaire are also asked on the post-test and follow-up versions of the questionnaire. The types of questions asked are also asked by the grantees in their local evaluation. Grantees will also use the data from the sensitive questions asked in their local evaluations. The sensitive information has been standardized across all grantees for use in this evaluation. Grantees will advise their program participants that their participation and responses to questions are voluntary.

The three versions of the Prevention Education Questionnaire for College Women (PEQCW) collect information from grantees about their client’s social and demographic information, access to health care, sexual practices, sexual orientation, health status information, HIV/AIDS status, substance use, history of sexual assault victimization, pregnancy status and relationship questions. Much of this information might be considered as sensitive; however, with the exception of participants’ HIV status, program participants provide much of the information requested in the course of the program’s local evaluation data collection. We are collecting data that grantees have indicated is needed in the assessment of their programs. The GEARS evaluation team does not receive any personal identifying information about program participants from the grantees. Each program participant has a unique identifier only known to the participant, not even to the grantees. Participants are provided a formula to create this unique identifier so that it can be used during pre-test, post-test, and follow-up. (See Appendix I for the unique identifier.) Grantees participating in the demonstration programs will forward questionnaire information to the GEARS evaluators. Collectively, the sensitive information asked on the PEQCW provides a profile of the clients served by OWH funded grantees with respect to HIV status and on key indicators that have been empirically associated with HIV/AIDS status such as ethnicity/race, gender and health access. This profile can be linked in data analysis to evaluation outcomes, such as risk behavior practices, in order to provide a better understanding between the association between participant socio-demographic information, the intervention and participant outcomes. For HIV prevention program planning this information is critical.

In sum, the sensitive information requested provides an opportunity to examine the correlation between personal demographic, health, and behavioral factors and HIV/AIDS status. Including these factors among the evaluation variables is critical to determining the degree to which OWH programs are effective.

A.12 Estimates of Hour Burden Including Annualized Hourly Costs

This evaluation is a one-time effort conducted for three years with an estimated 910 annual burden hours. The evaluation will be targeted to approximately 748 respondents among four respondent types: program directors, program staff, and peer educators and students. Exhibit A.1 presents the hourly burden breakdown which was used to derive the total burden time. Exhibit A.2 presents the annualized hourly costs for respondents.

The burden to respondents who participate in the evaluation will be in terms of their time.

Exhibit A-1 Estimated Hourly Burden

Respondent Type	Activity	Number of Respondents	Responses/ Respondent	Time (hr)/ Response	Total Burden
Students (program participants)	HIV Prevention Questionnaire for College Women: Pre-test; Post-test; Follow-up)	660	3	20/60	660
Program Staff	Data capture (data entry into database) and Process Evaluation Interview	12	55	15/60	165
		12	2	45/60	18
Directors, Funded programs	Process Evaluation Interview: Program Directors	14	2	90/60	42
Peer Educators	Process Evaluation Interview: Peer Educators	50	1	30/60	25
Total		748			910

The annualized burden for this project is 910 hours.

Exhibit A-2 Estimated Cost Burden

Respondent Type	Total Burden Hours	Hourly Wage Rate	Total Burden
Students (Follow-up only)	660	$5.85	$3,861
Program Staff	165	$14.00	$2,310
Program Staff	18	$14.00	$ 252
Directors	42	$25.00	$1,050
Peer Educators	25	$8.00	$ 200
Total			$7,673

Participation in the Prevention Education Follow-up Questionnaire for College Women will occur outside of the program via a web-based survey. Respondents decide whether they participate in this data collection. The survey will take approximately 20 minutes.

There are no additional costs to grantees for participation in this evaluation. Funded grantees and program staff (directors, staff, and peer educators) will participate in a process evaluation interview. The interview for Directors will take 90 minutes, for program staff 45 minutes and for peer educators 30 minutes. In addition, program staff will capture data on student participants into the database. This activity takes 15 minutes per student.

A.13 Estimates of Total Annual Cost Burden to Respondents or Recordkeepers

There are no additional respondent costs associated with start-up or capital investments. Additionally, there are no operational, maintenance or equipments respondent costs associated with continued participation in the evaluation.

The total annual cost burden to respondents or recordkeepers is $7,673 as presented in Exhibit A.2.

A.14 Estimate of Annualized Cost to the Federal Government

The evaluation will be conducted for three years. The overall cost to implement the evaluation is associated with labor required to conduct the following activities: develop evaluation design and methodology; develop data collection forms; design and develop electronic data storage systems; manage data collection activities; develop quarterly reports; conduct and report site visits to funded contractors; develop the evaluation methodology and analysis plan; train evaluation staff; administer interviews; perform data entry; ensure accurate data maintained in data storage systems; and analyze and report evaluation results. Exhibit A-3 presents the cost breakdown by major budget category.

Exhibit A-3 Cost of the Proposed Study

Activity	Cost
Personnel Costs	$169,852
Other costs (facilities, travel, postage, copying supplies, conf. calls)	$ 24,473
Total	$194,325

Total annualized costs to conduct this evaluation are $194,325.

A.15 Explanation for Program Changes or Adjustments

There are no changes in burden. This is a new project.

A.16 Plans for Tabulation and Publication and Project Time Schedule

Exhibit A-4 Project Time Schedule

Activity	Time Period
Federal Register Notice and OMB Clearance	July-September 2008
Administrator Training and Sampling Plan Development	Early September 2008
On going data collection activities	September 2008– June 2010
Analysis	June 2010-September 2010 –
Reporting: Evaluation Report & Executive Summary	October 2010

Publication

Evaluation findings will be summarized in a comprehensive Evaluation Report and Executive Summary developed by GEARS for OWH. The findings from this evaluation will be shared with professionals working with HIV-infected women at regional and national conferences.

Analysis Plan

Both qualitative and quantitative data will be collected for this evaluation. Data analysis will be supervised by Deborah Brome, Ph.D., Project Evaluation and Data Manager, in consultation with Michael Milburn, Ph.D., Project Statistician. Data entry, file organization and data access and management will be supervised by Dr. Deborah Brome.

A. Qualitative Data Analysis. Qualitative data will consist of structured individually administered interviews. These data will be analyzed using the basic strategies and principles espoused from grounded theory and the interpretative process. From Grounded Theory, the data analytic strategy will focus on the systematic examination of data for the purposes of explicating the inter-relationships between concepts that assist with the testing of hypotheses. The interpretative process will provide a context for understanding the data gathered, especially as it pertains to culturally significant processes.

Each structured interview (process evaluation interviews) will be coded for themes relating to our project objectives. Coding organizes and identifies issues and themes of relevance in the text. Coded data may be descriptive, interpretive or identify patterns. Codes are used to provide labels or tags assigning meaning to the descriptive information provided by the interviews. GEARS will develop the coding scheme for the process interview protocols.

Following the initial coding staff will meet to review coding and present the analysis to the project team. Codes will be modified through a consensus among the team members. Inter-rater reliability of the coding scheme will be established among interview coders,

Following manual coding, data will be entered into a computer using Nvivo, the most recent version of the standard coding software package NUDist (Non-numerical Unstructured Data Indexing Searching and Theorizing). Nvivo is a program for data entry, coding, sorting and retrieval. The initial codebook will be entered into the program with any notes or memos by project staff taken in the initial review of the data. Subsequent to entering the interview data, an initial search for several single codes will be performed and adjustments made to the codebook and coding as necessary.

A final analytic step goes beyond classification of the data and explores whether or not linkages exist between/among particular categories. At the descriptive level, analysis involves seeing patterns. At the theoretical level, it involves thinking about why things happen. Nvivo will be used in this step.

In addition, structured interview data is amenable to analysis through nonparametric tests using SPSS. Where appropriate qualitative data will be coded and entered into an SPSS program for analysis. For example, through SPSS we can analyze number and types of prevention activities offered by grantees.

2. Quantitative Data Analysis. Quantitative data will consist of measures of prevention knowledge and attitudes and an appraisal of risk reduction behaviors.  Once the data have been entered and cleaned, statistics such as frequencies, percentages, means, and standard deviations, as well as reliability statistics will be calculated.  The basic research design utilizing quantitative data is a between-groups (e.g., type of minority institution) repeated measures design, with measurements taken at pre-test, post-test and follow-up for each cohort of participants.  There will be 12 minority institutions, (six institutions representing HBCUs, four representing Hispanic Serving Institutions, and two representing Tribal Colleges and Universities).

 

There are two primary dependent variables, HIV risk reduction and HIV knowledge.  The reliability of these measures will be assessed as with the utilization questionnaire, using factor analysis and reliability analysis.  Additionally, there are a group of questions that assess gender specific risk factors.  These questions have been generated from the curricula used by the minority institutions and in collaboration with the principal investigators of the minority institution grantees.  As with the risk reduction and HIV knowledge questions, factor analysis and reliability coefficients will be computed for these scales.  In addition, a structural equation model will be estimated that specifies and separately estimates both the amount of change over time in the measures and the reliability of the measures.  Ordinary test-retest analysis confounds reliability and change over time—we will employ the Wheaton model for the analysis of panel data to separate these different components.

 

 After reliability of our instruments has been determined, the primary analysis we plan is a mixed model design, estimated with the General Linear Model program in SPSS.  This model will include one within-subjects factor of time and one between-subjects factor of type of minority institutions (HBCUs, TCU, HIS) with differing numbers of institutions within each type of institution, and participants’ scores nested within their particular minority institution.  Background variables such as education will be entered as covariates. While there will be main effects between minority institutions over time, the time by minority institutions interaction will provide an assessment of difference between institutions.  Using the power analysis tables for F-tests on means in the analysis of variance (Cohen, 1977, p.325), we can make a judgment concerning the necessary sample size.  Hypothesizing a medium effect (f=.25), the time (2 df) by group (11 df) interaction has 11 degrees of freedom. So to obtain power=.80 at p=.05, 25 subjects per group are needed (Table 8.3.19, p. 325).  The proposed sample size thus ensures quite adequate statistical power.

 

Assessing differences between grantees (minority institutions) and contractors allows us to identify those that are particularly effective; process analysis will enable us to determine what aspects of their programs are particularly effective.  OWH can make use of important aspects of their programs as they consider future initiatives.

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

OMB expiration dates will be displayed on all materials.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement identified in item 19 “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

B. DESCRIPTION OF INFORMATION COLLECTION

B.1 Respondent Universe and Sampling Methods

There are four respondent types participating in this evaluation project. They are program directors, program staff, peer educators, and students (participants). No sampling methods are used for any respondent type. Instead a convenience sample will be used since data is being collected from all grantee staff and program participants. Moreover, program directors, program staff, and peer educators of funded minority institutions are predetermined based on awards made in 2007 by DHHS/OWH. Therefore, the entire universe of respondents for these groups consists of the program personnel at the minority institutions that received grants for this project.

The respondent universe for the students consists of those young women attending minority institutions who are recruited for and participate in the institution’s HIV prevention education program. It is expected that annually 660 young women will be recruited across the 12 participating minority institutions. It is expected that 90-100% of program participants (participating students) will participate in the pre and post-test data collection and 80% in the online follow-up data collection. This estimate is based on the experience of reported by the funded minority institutions on their success in using online surveys with students. The degree of success appears to be a function of student access to personal and student email accounts. Because of the sensitive nature of the information requested tracking of IP addresses will not be saved as students respond to the follow-up survey. In addition, GEARS, the contractor collecting follow-up data will not have access to identifying information on participants at any time during the data collection. Consequently, strategies that increase response rate that depend on knowing a respondent’s email and IP address and consistent contact will not be able to be used. Grantees will be able to send out “general” reminders to participants to complete their online follow-up survey.

B.2 Procedures for the Collection of Information

There are several data collection activities included in this project. Appendix J provides a description of the information collected, rationale and sources/references for each data collection used in this project. The following presents data collection procedures for the evaluation project:

1. Verify IRB approval from all institutions. GEARS will collect IRB approval letters from all institutions participating in the evaluation.

2. Obtain OMB clearance.

3. Finalize all forms with the OMB clearance number printed on forms.

4. The day OMB clearance is received, GEARS will send an email to each program to inform them that: 1) OMB clearance has been obtained and 2) that they will be receiving data collection forms and software via email express mail within the next three business days; and 3) their GEARS program coordinator will contact them to begin scheduling site visits.

5. Within two-three business days after receiving OMB clearance, GEARS will send each program copies of data collection forms and an electronic EXCEL spreadsheet for them to input data required for the evaluation.

6. Two weeks after receiving OMB clearance, begin training sessions for GEARS staff on details of the administration of the process evaluation interviews. This two-hour training will review the purpose of the process evaluation and how the interview fits within the evaluation framework. Each process interview question will be discussed and questions answered. This training will be led by this projects’ Evaluation and Data Manager, Dr. Deborah Brome, Vice President, GEARS. GEARS staff is trained in interviewing administration and have conducted interviewing as part of GEARS’ research and organizational development activities.

7. One month after OMB clearance begin conducting monthly conference call in order to monitor and answer questions about data collection procedures.

8. Approximately one to two months after OMB clearance, GEARS will attend the OWH Grantee Meeting and will train the grantee (minority institution) staff on data collection from students. Emphasis will be placed on the consenting process that has been approved by their IRB’s and utilizing the script developed for creating the unique identifier. (Appendix J)

9. Two months after OMB clearance is received, GEARS will schedule and conduct site visits to the funded minority institutions. Our goal is to complete all site visits within four months of obtaining OMB clearance. During this visit, GEARS staff will administer the initial intake form (process evaluation form with program directors, program staff and peer educators (Appendices C, D and E, respectively). These interviews will be conducted in person during the site visit. GEARS staff will also provide additional training on the Prevention Education Questionnaires for College Women with program staff if needed.

10. On a quarterly basis throughout the project period, respondents (program staff) will enter the data into Excel spreadsheet and transmit results to GEARS. Program staff will also use submitted data as part of their local evaluations. (Refer to Appendices G, H, and I.)

11. Within six months after clearance is obtained, follow-up emails will be sent to students three months post completion of the minority institution’s HIV prevention program. These emails will be sent to students by the program to email addresses provided by students. This email will ask students to participate in a follow-up survey and will provide an email address for them to access. Each minority institution will have a specific site. At this site students will sign in using a unique identifier and their IP addresses will not be saved. GEARS will download completed surveys on a monthly basis and will provide minority institutions with a copy of the data in EXCEL format.

Submission of Data

Data collected for this evaluation are designed as much as possible to fit within existing program data collection activities. GEARS Program Coordinators will make a site visit of all programs and conduct training on evaluation data collection forms and software after OMB clearance is obtained. MIS information as well as data markers required for the evaluation will be entered into a user friendly Excel database and electronically transmitted to GEARS on a quarterly basis.

Process evaluation data will be collected twice a year. These data will be collected by personal interview during the first part of the grant year and by telephone during the second half of the grant year by GEARS staff. GEARS staff will ask the respondent permission to audiotape the Process evaluation interview. Audio-taping is requested in order to ensure that the detailed information provided by the respondent in this interview is accurately captured in the evaluation data. Specifically, GEARS staff will check the accuracy of the recorded interview responses against the audio-taped interview. All audio-tapes will be kept in a locked filed in GEARS offices and will be destroyed at the conclusion of the evaluation project, when all data and data coding have been finalized.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

The follow-up, on-line survey is the only data collection that is not being collected as part of program activities. In order to increase response rates for the online survey, minority institution programs will email their participants three months post program completion and request their assistance in the evaluation. (See Appendix L for email.) They will provide a web link, indicate the ways in which the data collected will be used, and provide instructions on how to create their unique identifier. This email will only be sent twice, two weeks apart so that students do not feel harassed or pressured to participate in the data collection.

One hundred percent participation is expected from Minority Institutions in collecting process evaluation data. This data will be collected as part of the yearly site visit.

B.4 Test of Procedures or Methods to be Undertaken

The majority of the Instruments and items chosen for this evaluation have been selected from standardized instruments previously used by federal organizations, previously OMB approved evaluations, research groups, or academic institutions with either community-based populations or organizations and tested for their cultural and linguistic appropriateness.

Additionally, two sets of respondents (program directors and program staff) had opportunities to input on development of the instruments and submit copies of instruments they currently use in their local evaluations to guide development of instruments used in this project. Every effort was made to ensure that data collected for OWH was also data needed by grantees for their local evaluation. Items in the instruments have been reviewed by five minority college students to further assess culture and language appropriateness and to estimate length of time needed to complete instruments.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Program Development Contact

Adrienne M. Smith, PhD, MS, CHES

Public Health Advisor

U.S. Department of Health and Human Services

Office on Women's Health

202-690-5884

[email protected]

Data Collection/Analysis and Statistical Contact

Deborah Brome, Ph.D.

Vice President and Director, Evaluation & Applied Research

Global Evaluation & Applied Research Solutions (GEARS Inc.)

617-328-5141

[email protected]

Michael Milburn, Ph.D.

Professor of Psychology

University of Massachusetts, Boston

100 Morrissey Boulevard

Boston, Massachusetts

617-287-6386

[email protected]

 

http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=105_cong_public_laws&docid=f:publ277.105.pdf

1 Aragon, R. & Kates, J. Minority AIDS Initiative Policy Brief, Kaiser Foundation, June 2004

OMB Clearance Supporting Statement

Evaluation of OWH HIV Prevention Program at Minority Institutions Page 16