1. Respondent Universe and Sampling Methods
Potential respondent universe:
Key Persons Telephone Interviews—
10 Project Directors (PDs)
10 Project Coordinators
10 Local Evaluators
20 Coalition Chairs or Co-Chairs
Appx. 300 coalition members across all 10 grantee sites
Sampling: Statistical methods will not be used to select respondents. The universe is included for the Project Directors, Project Coordinators, Local Evaluators and Coalitional Chairs and Co-chairs.
Other respondent selection method: Purposive and convenience sampling will be used in consultation with the PDs to select 2 Coalition Members, 2 Community Leaders, and 50 Coalition Participants, and other Community Members. Each site will select members, leaders and participants who have participated in their coalition or intervention programs, or have been reached through education and outreach activities.
Data on the number of entities in the universe (corresponding sample):
Respondent Type |
Universe |
Sample |
Project Directors |
10 |
10 |
Project Coordinators |
10 |
10 |
Local Evaluators |
10 |
10 |
Coalition Chairs/Co-Chairs |
20 |
20 |
Coalition Members- Interviews |
300 |
20 |
Coalition Members—Online Surveys |
300 |
300 |
Community Leaders familiar with coalition-Interviews |
50 |
20 |
Community Leaders familiar with coalition—Online Surveys |
50 |
50 |
Coalition Participants who participated in coalition interventions for Online Surveys |
3,000 |
300 |
Community Members who participated in coalition activities for Online Surveys |
2,000 |
200 |
Expected response rates for the collection as a whole: 80%
Conducted previously? No previous collection.
Statistical justification for sample sizes. A sample size of 50 for the Coalition Participants and Community Members (combined total) is sought to meet industry standards with respect to n=30 as an adequate sample size to achieve in order to apply basic statistical tests. With an oversampling by 20 members, we expect to achieve the minimum 30 respondents who agree to complete the online survey. Identifying in each site an additional four respondents (two each in the coalition members and community leader categories) will increase the same size of “key persons” for the purposes of having adequate numbers in the various categories of respondent to compare online survey data and quantitative data in the Interview protocol (by respondent type).
Explanation for universe: The samples based on the universe are respondents for the Key Persons Interviews. There are only 10 grantee sites, and one PD, project coordinator, and local evaluator will be included from each site—thus, yielding 10 in each of these categories. For the Chair/Co-chair respondent type, there might be one or two in each site, but we used the upper limit of two to arrive at the universe of 20. All of these respondents hold key roles on the coalition funded by the OWH CHC grant, and their input is needed to document the various perspectives, triangulate data from these various perspectives (data sources), and reflect the community-based, participatory nature of the initiative as well as the evaluation.
2. Procedures for the Collection of Information
Describe the procedures for the collection of information including:
• Statistical methodology for stratification and sample selection: Not Applicable (N/A)
• Estimation procedure—N/A
• Degree of accuracy needed for the purpose described in the justification—N/A
• Unusual problems requiring specialized sampling procedures—N/A
• Any use of periodic (less frequent than annual) data collection cycles to reduce
Burden: We have included two collections of information from the key persons to reduce burden in capturing information about this five-year initiative. The first collection will focus on respondents’ perspectives on the period prior to the CHC initiative and implementation in their community through September 30, 2016 (their fourth year of the grant). The second collect will occur at the end of the third quarter of year five—their last year of the grant, to capture information on the impact of the coalition’s efforts in their community and plans for sustaining any aspects of the coalition beyond OWH’s grant funding. Project directors will also provide additional information about the sustainability planning for the coalition in the last month of the grant.
Who will collect the
information: OWH’s contractor for the national evaluation, The
MayaTech Corporation
How it will be done:
Key Persons Interviews—Form 1: Grantee staff and consultants were notified of the expectation to participate in the national evaluation as part of their application and continuation applications. These respondents will be notified in advance through email communication from MayaTech that the national evaluation is underway and MayaTech will use a Web-based scheduler to identify a date and time convenient to the respondents. Other respondents will be notified through email from MayaTech that they have been nominated by the grantee’s PD to participate in the Key Persons Interviews and Coalition Member and Community Leader Surveys. The text for the e-mail transmittals used to contact the Key Persons is attached. Prior to the interview, participants sign a written informed consent form (Attachment B in the supporting document), including permission to audio-record (if permission to audio-record is not given or withdrawn at the time of the interview, no recording is made).
Key Persons Surveys (Coalition Members and Community Leaders—Form 2). MayaTech will distribute the online survey directly to these respondents. The survey includes a preamble explaining their selection, the purpose of the survey, and informed consent/participant rights. The preamble includes an item prior to the survey that explicitly asks participants to indicate they have read and understood their rights and are 18 years of age or older.
Coalition Participants and Community Members Survey—Form 3. These participants are selected by their grantee PD and their email addresses are not known to the national evaluation contractor. The survey links are sent directly to the participant, and the survey includes a preamble similar to that for Form 2. The surveys, however, are submitted to the online survey vendor site maintained by MayaTech, so the sites will not be aware of the participants’ responses. MayaTech will download the Excel files generated by the survey vendor with no identifying information (email addresses) and use these to generate the SPSS files for analysis.
Wilder Collaboration Factors Inventory—Form 4 (secondary data). Project Directors will submit this form with their end-of-year report. The form is based on individual distributions to coalition members who will record their individual ratings for each item; and then these ratings will be averaged for each item by the PD and evaluator and submitted to OWH. MayaTech will receive these data (one form from each site) with the averages for use in secondary data analyses (summary across sites and comparisons between sites).
Grantee Status Report on Cost-effectiveness—Form 5 (secondary data). Project directors will complete this form in collaboration with their local evaluator/health economist. Responses are sent to OWH. MayaTech will receive these forms (one for each site) and produce a summary report.
Grantee Report on Sustainability Planning—Form 6 (secondary data). The PDs and local evaluators will each complete this form and then submit the average rating for each item to OWH. MayaTech will receive these data for use in secondary analyses to produce averages for each item and subscale across the initiative.
Quality control procedures: Trained interviewers will conduct the Key Persons Interviews via telephone using the Discussion Guide (Form 1); and trained note takers will assist by operating the digital recording equipment, with the respondents’ permission (included on the written Informed Consent Form-Attachment B in the OMB supporting document). The note takers will compare their notes to the digital recording to produce a written transcript for review and approval by the interviewer. During coding of transcripts, two coders will initially be trained using ten text samples to 80% reliability to extract themes based on operational definitions for each primary and secondary code. Where discrepancies exist, the coders will discuss the reasons for their codes and come to consensus. If consensus cannot be achieved between the coders, the interviewer or MayaTech’s project director or deputy project director will resolve the conflict. All digital recordings and transcripts will have a unique alphanumeric identifier to ensure that the transcript, recording, and resulting data file of codes match. The deputy project director at MayaTech will provide oversight for the QC procedures. An online survey vendor will produce the summary descriptives for each quantitative item and a compilation of qualitative responses for each open-ended item. MayaTech’s research assistants/associates will transfer data to word-processed tables that will be checked by the deputy project director against the online survey reports. Any tables generated from the SPSS files that rely on these data will also be doubly checked for consistency in reporting.
Respondents’ re-interview or re-contact for validation: No re-interview or re-contact planned. N/A
Statistical power—N/A
3. Methods to Maximize Response Rates and Deal with Nonresponse
Methods to maximize response rates. The OWH project officer for the grans also serves as the project officer (Contracting Officer’s Representative) for the evaluation contract. She will assist by sending emails to the PDs (whom have already been introduced to the national evaluator at grantee meetings, on grantee conference calls, etc.; and provided input into the national evaluation design and measurement strategies to reduce burden). PDs and Project Coordinator burden estimates include time to send emails to the other respondents or to assist with recruitment and distribution of online surveys. Three attempts will be made to non-respondents before a replacement respondent (e.g., another coalition member or community leader) is contacted.
Deal with issues of nonresponse. MayaTech will send reminders after one week to key persons who have not responded to initial requests for participation or are no-shows for a scheduled telephone interview. PDs will assist with reminding coalition members and community leaders to complete surveys; and Project Coordinators will assist with auto-generated email reminders to coalition participants and community members for that survey. All of the grantees’ staff time has been included in the burden estimate.
4. Tests of Procedures or Methods to be Undertaken
Pilot test to estimate burden. The burden for the key persons’ interviews was estimated from one administration with a mock participant with a background similar to a project director level individual in this initiative. Some participants will have more knowledge and familiarity with the initiative; and, therefore, we estimated two hours for these key persons, but only 1 hour for less knowledgeable or familiar respondents. The estimate for the project directors for the key persons’ interview includes one additional hour to assist OWH and the contractor with identifying other key persons and encouraging participation. The estimate for the key persons’ online survey was estimated with the same person used to test the time for the interviews. The estimate for the participant and other community members’ online survey was estimated from published literature from which some survey items were derived and further estimated from two undergraduate students’ completion of the draft survey using a word-processed version of the form.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
Design:
Agency Personnel:
Stephanie Alexander, MS
Health Scientist Administrator
Contracting Officer Representative (responsible for receiving and approving contract deliverables)
Office on Women’s Health
U.S. Department of Health and Human Services
200 Independence Avenue, S.W., Room 712E, Washington, DC 20201
E-mail: [email protected]
Main: 202-690-7650 | Direct: 202-401-9546
Adrienne Smith, PhD, MS, CHES
Office on Women’s Health
U.S. Department of Health and Human Services
200 Independence Avenue, S.W., Room 712E, Washington, DC 20201
Keiva Nelson
Office on Women’s Health
U.S. Department of Health and Human Services
200 Independence Avenue, S.W., Room 712E, Washington, DC 20201
Contractor for National Evaluation:
The MayaTech Corporation
8401 Colesville Road – Suite 430
Personnel:
Suzanne M. Randolph, Ph.D. (Project Director)
Shelly Kowalczyk, MSPH, CHES
(Deputy Project Director)
Veronica Thomas, PhD (Sr. Research
Scientist)
Patrick Richard, PhD (Health Economist)
Robert Goodman, PhD (Consultant; coalition evaluation specialist)
Gender Experts: Elaine Walker, Ph.D., Miguelina Leon, MSW, The Iris Group
Data Collection:
Agency Personnel (submit secondary data from progress reports and end-of-year reports to MayaTech):
Office on Women’s Health
Stephanie Alexander, MS
Keiva Nelson
Contractor for National Evaluation:
The MayaTech Corporation
8401 Colesville Road – Suite 430
Personnel:
Suzanne M. Randolph, Ph.D. (Project Director)
Shelly Kowalczyk, MSPH, CHES
(Deputy Project Director)
Veronica Thomas, PhD (Sr. Research
Scientist)
Patrick Richard, PhD (Health Economist)
Data Analysis:
Contractor for National Evaluation:
The MayaTech Corporation
8401 Colesville Road – Suite 430
Personnel:
Suzanne M. Randolph, Ph.D. (Project Director)
Shelly Kowalczyk, MSPH, CHES (Deputy Project Director)
Veronica Thomas, PhD (Sr. Research Scientist)
Patrick Richard, PhD (Health Economist)
Agency personnel responsible for receiving and approving contract deliverables:
Stephanie Alexander, MS
Health Scientist Administrator
Contracting Officer Representative (responsible for receiving and approving contract deliverables)
Office on Women’s Health
U.S. Department of Health and Human Services
200 Independence Avenue, S.W., Room 712E, Washington, DC 20201
E-mail: [email protected]
Main: 202-690-7650 | Direct: 202-401-9546
| File Type | application/msword |
| Author | DHHS |
| Last Modified By | Windows User |
| File Modified | 2015-11-05 |
| File Created | 2015-11-05 |