“Multidisciplinary Health Care Models for Women Sustainability Assessment”
OMB Clearance Application
Office on Women’s Health
September 10, 2009
Submitted by:
Office on Women’s Health
Department of Health and Human Services
200 Independence Avenue, S.W. 537-H
Washington, DC 20201
Supporting Statement
Background
The purpose of this collection is to identify and measure the key components of sustainability of Federally funded programs. The assessment will examine this issue using the Department of Health and Human Services (DHHS), Office on Women’s Health (OWH)’s Multidisciplinary Health Models for Women (MHMW) effort as a case study.
The MHMW effort was funded from 1996 to 2007 and comprised of five programs designed to provide comprehensive, multidisciplinary, and integrated health services for women. The five programs are described in Attachment A.
A. Justification
Circumstances Making the Collection of Information Necessary
OWH is seeking review and approval by OMB on this new information collection, Multidisciplinary Health Care Models for Women Sustainability Assessment.
This assessment is designed to contribute to a comprehensive understanding of the key drivers of Federal program sustainability. Specifically, the assessment is designed to answer the following research questions:
How has Federal program sustainability been defined and conceptualized?
What are the characteristics of programs that have sustained themselves after Federal funding expired? What factors are associated with sustainability?
How is program sustainability determined?
What types of guidelines for sustaining a Federal program could be included in future Federal grant/contract announcements?
As part of the assessment, the contractor has developed a conceptual framework that draws upon major aspects of previous frameworks to hypothesize a causal pathway leading to sustainability. The research design is intended to test how well the MHMW Sustainability Framework presented in Attachment D measures implementation of the core five-component model and captures each stage of the sustainability process.
This data collection is authorized under Section 301 of the Public Health Services Act (42 U.S.C. 241).
Purpose and Use of Information Collection
This assessment will have extensive practical utility for OWH and other Federal agencies. The information collected and analyzed through this assessment will be used by OWH to develop evidence-based practices to sustain successful and effective agency programs and initiatives. It also will be shared with other DHHS agencies to inform their efforts to sustain promising and effective initiatives. The assessment will provide empirical information on what factors enabled programs to continue to implement the goals of the MHMW program after funding ended. These findings will be translated into recommendations for OWH and other agencies as to how they might best provide support to funded entities to maximize the chances that program goals will continue to be achieved after Federal funding has ended. These recommendations may result in guidance on program sustainability that can be included in future Federal grant and contract announcements.
OWH has hired a contractor to conduct an assessment of Federal program sustainability. The assessment is designed to collect data from all 48 MHMW program sites through four methods: (1) abstraction of administrative data from grant and contract documents to establish a baseline understanding of each site, (2) a Web-based survey of staff, (3) key informant interviews with staff, and (4) in-person visits to a 12 sites to collect in-depth information from additional types of staff, community partners, and clients. OWH is seeking approval from the Office of Management and Budget (OMB) for two of these four data collection methods:
Web-based survey (Attachment B) Key program staff at each site will be asked to answer questions on the extent to which the five core components were implemented, have been sustained, and the factors that may have inhibited sustainability. The primarily categorical responses will be used to group sites based on the level of implementation and sustainability each has attained. The web-based survey is a primarily close-ended survey designed to collect data on the key dependent and a number of key independent variables in a uniform manner across sites.
Telephone interview (Attachment C). The contractor will conduct telephone interviews with core program staff, and/or other types of staff as necessary, to collect more descriptive information on the factors that influenced implementation and sustainability, how well sites were able to integrate the five core components, the resulting outcomes of sites’ efforts, and respondents’ recommendations on how Federal agencies could better promote program sustainability. This information will be coded and used in the analysis of sustainability and to inform site visit selection.
Use of Improved Information Technology and Burden Reduction
All MHMW program sites were charged with integrating a multidisciplinary model consisting of five core components:
Clinical Care, comprehensive health service delivery that include gender and age appropriate preventive services and allied health professionals as members of the comprehensive care team;
Research, the development or integration of research activities on women’s health;
Public Education and Outreach, activities that emphasize gender-specific and age-appropriate prevention and/or reduction of illness or injuries;
Training, training on women’s health issues and services for laypersons, allied health professionals, and professional health care providers;
Leadership Development, efforts to increase opportunities for developing and exercising leadership skills for women as health care consumers and providers.
Some sites were encouraged to address two additional components:
Sustained Technical Assistance, sharing valuable lessons learned from the MHMW model with other organizations, such as effective strategies for uniting partners, securing resources, coordinating activities, and recruiting and managing volunteers.
Replication of the MHMW model, reproducing all components of the MHMW model at another organization. Sites were encouraged to start the replication process with the most developed component or the component that addresses the organization’s greatest needs and then phase in the other components.
This assessment will assess the:
Extent to which organizations previously funded under OWH’s MHMW effort have sustained or enhanced the core and additional components of the program approximately 20 months after OWH funding ended, and
Factors that have facilitated or inhibited sustainability at MHMW program sites.
OMB clearance is not being sought for the data abstraction as it will not involve direct contact with individuals at sites. Clearance is also not being sought for the site visit protocols as they will be tailored to each individual site.
Efforts to Identify Duplication and Use of Similar Information
Prior to the submission of this application, the contractor completed a review of the literature on how sustainability has been defined and measured, funders’ expectations for planning for sustainability across health and human service initiatives, and types of strategies that have been proposed to help program implementers sustain their programs beyond initial funding. This review concluded that only a limited number of empirical studies with varying degrees of rigor have been conducted on sustainability. It also found that no sustainability studies have been conducted of the MHMW effort or similar women’s health initiatives funded by OWH or any other entity.
Although the development of the conceptual framework guiding the assessment and the set of data collection instruments have been informed by existing literature, direct data collection and analysis are necessary to test the model and gather information to develop concrete recommendations for strategies that will facilitate sustainability.
Impact on Small Businesses or Other Small Entities
No small businesses will be involved in this study.
Consequences of Collecting the Information Less Frequent Collection
This assessment requires one-time data collection for both the Web-based survey and the telephone interviews. Without collecting these data, OWH will not have access to a comprehensive assessment of the current implementation and sustainability of the MHMW effort, as well as the key factors that drive Federal program sustainability. This information will lead to the development of a policy statement on the importance of planning for sustainability and guidance on how grantees and contractors can improve the likelihood of long-term program survival. The Federal government will benefit from this policy statement that may inform efforts to promote program sustainability among a wide array of health and human service initiatives.
There are no legal obstacles to reducing the burden of collection.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This request complies with the information collection guidelines of 5 CFR 1320.5. There are no special circumstances.
Comments in Response to the Federal Register Notice/Outside Consultation
A 60-day Federal Register Notice was published in the Federal Register on July 10, 2009, vol 74, No. 131, pp. 33257-33258. There were no public comments.
The instruments were directly shared with respondents from 2 MHMW sites as part of an instrument pre-testing process. Their comments and suggestions were incorporated into revised instruments. These changes consisted of clarifying some of the wording on both survey and interview questions. For example, one pretest respondent requested that the survey clearly specify what was meant by the number of clients served in question 3 of the survey. As a result of the pretest there were also more specific probes added to the interview protocol in Attachment C.
Contractor staff consulted on data collection include:
Namratha Swamy, PhD
Director of the Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(P) 202-828-5100
(F) 202-728-9469
Renee Schwalberg, M.P.H.
Co Lead Women, Children, and Adolescents Subject Area
Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(202) 772-1410
Christopher Botsko, MA
Deputy Director
Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(P) 202-828-5100
(F) 202-728-9469
Amy Brown, MPH
Senior Policy Associate
Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(P) 202-828-5100
(F) 202-728-9469
The materials were also reviewed by an expert advisory group consisting of academics, a consultant, and government staff who have studied or addressed issues of sustainability in government programs. Advisory group members included:
Jay Mancini, PhD, Virginia Tech
Lydia Marek, PhD, Virginia Tech
Pierre Pluye, MD, PhD, McGill University
Susan Hailman, M.Ed., Campaign Consultation, Inc.
Karen Hench, R.N., M.S., HRSA
Lisa King, MA, HRSA
Laura Kavanagh, MPP, HRSA
The expert advisory group made a small number of recommendations regarding changes to the survey and interview protocols. The original survey did not include questions asking respondents what factors contributed to their success at sustainability. The expert advisory group was concerned that this was an important issue that should be covered in the survey and that it gave the survey a negative “tone”. An earlier draft of the survey had included questions on this topic but they had been deleted because of concern over respondent burden. However, the pretest indicated the survey took less time than the initial burden estimates so based on this finding and the recommendation of the advisory group a series of questions were added about what factors contributed to sustainability among those sites that were able to sustain particular aspects of the components to a great extent or a large extent. The group also recommended that a question be added asking what year planning for sustainability began and this was also added. For the telephone interview the group recommended the use of more specific probes and these were added.
Explanation of any Payment/Gift to Respondents
No payment or gift is being offered to respondents.
Assurance of Confidentiality Provided to Respondents
Individuals and organizations contacted will be assured that their replies will be kept private to the extent allowed by law, under 42 U.S.C. 1306, and 20 CFR 401 and 4225 U.S.C.552a (Privacy Act of 1974). For the Web-based survey, the contractor will collect the contact information (i.e., phone number, email address, and mailing address) of the key individual at each site who is overseeing participation in the survey in compliance with all aspects of the Privacy Act. The contractor will use this contact information to send reminder notices and, if needed, to offer technical support to help complete the survey. On the Web-based survey instrument, only the organization’s name will be recorded; individual respondents’ names and contact information will not be recorded. For the telephone interviews, the contractor will collect names and contact information for each person participating in the interviews, also in compliance with all aspects of the Privacy Act. The contractor will use the contact information to arrange the telephone interviews and to collect follow-up information after the interviews are completed, if necessary. Social Security numbers will not be collected for any individuals participating in either the Web-based survey or telephone interviews.
Prior to the start of data collection, respondents in both data collection methods will be presented a written copy (Web-based survey) or read aloud (telephone interviews) the following privacy assurance statement regarding the handling and use of their reported information:
Your name and organization will not be attached to specific comments that you share today. Your response may be included with those of other respondents in aggregate form in reports or journal articles.
In addition, participants’ names will not be included in any information viewed by OWH or any other HHS officials.
Methods will also be taken to protect study data. Resulting data from the survey and interviews will not identify any person; rather, results will be presented for each MHMW grantee, and across all grantees and contractors. Data from the suveys and interviews will be stored in a password protected database. Only authorized contractor staff working on the assessment will have access to the database. The briefs and reports produced for the assessment will not identify any specific individuals. All potentially identifying information will be destroyed at the study’s conclusion.
Justification for Sensitive Questions
The interviews will not include any questions of a sensitive or personal nature. The questions are designed to solicit information solely regarding particular aspects of each site’s MHMW program and efforts to promote Federal program sustainability. Respondents will be asked to provide factual data and opinions from the perspective of their organizations and their role as a staff member or community partner.
Estimates of Annualized Burden Hours (Total Hours & Wages)
12A. Estimated Annualized Burden Hours
In Exhibit 1, we provide estimates of the collection burden on participants from each category of respondent. Data collection activities include: (1) a Web-based survey and (2) telephone interviews. Draft protocols may be found in Attachments B and C. The survey will take approximately 35 minutes to complete. This estimate is based on an assumption that respondents will complete about three and a half survey questions per minute. This is a conservative estimate. Most respondents will not see every question because their responses to some questions will lead them to skip some follow-up questions. The pretest indicated respondents were able to complete the survey in less time than originally estimated, but since additional questions have been added it was determined that the original burden as published in the Federal Registry Notice is valid. We also expect that prior to beginning the survey, the respondents may wish to review their final report to help them recall details from the funding period and we estimate that this will take approximately one hour, which is incorporated into the burden estimate.
The telephone interview will take approximately 60 minutes to complete, and the estimate assumes that respondents will spend approximately 30 minutes reviewing their MHMW experience prior to the interview. This estimate is based on the contractor’s experience with many similar instruments involving comparable levels of detail and the length of time required to complete the pretest interviews. This is a reduction in the estimate of burden from the initial Federal Registry notice. This is based on the pretest of the instrument. The pretest interviews averaged 45 minutes, but a few additional probes have been added leading to the estimate of 60 minutes for the revised instrument. The review time estimate assumes respondents will spend less time reviewing past experience because of time spent reviewing for the survey. Respondents are not being asked to gather any additional information or data. Estimated review time assumes that respondents may read their final reports or other program documents prior to the survey or interview.
Exhibit 1. Estimated Burden Hours
Type of Respondent |
Form |
# of Respondents |
No. Responses per Respondent |
Average Burden Per Response (Hours) |
Total Burden Hours |
Key Site Staff |
Web-based survey (Attachment B) |
48 |
1 |
95/60 |
76 |
Site Staff and Community Partners |
Telephone Interview (Attachment C) |
48 |
1 |
90/60 |
72 |
TOTAL |
96 |
--- |
--- |
||
12B. Annualized Cost to Respondents
In Exhibit 2, we present the estimated burden cost for the Web-based survey and telephone interviews. The total annualized cost to the respondents is $5,702.12. This cost estimate was calculated based on the total respondent hour burdens noted in Exhibit 1. The wage rate is the median wage rate for medical and health service managers according to the Bureau of Labor Statistics Occupational Wage Statistics.
Exhibit 2. Estimated Burden Cost
Type of Respondent |
Total Burden Hours |
Average Hourly Wage Rate |
Total Hour Cost |
Site Staff and Community Partners (Web-based survey) |
76 |
$38.581 |
$2,924.36 |
Site Staff and Community Partners (Telephone interview) |
72 |
$38.581 |
$2,777.76 |
TOTAL |
160 |
--- |
$5,702.12
|
1 Based on median hourly wage for medical and health service managers, “May 2008 National Occupational and Wage Estimates”, United States, U.S. Department of Labor, Bureau of Labor Statistics.
Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs
Data collection for this study will not result in any additional capital, start-up, maintenance, or purchase costs to respondents or record keepers. Therefore, there is no burden to respondents other than that discussed in the previous section.
Annualized Cost to Federal Government
The majority of costs for conducting this assessment are included in the contract between OWH and the contractor which totals $359,232 over the course of 2 years. Of that amount, approximately $113,444.30 will be used to cover the costs of the data collection and analysis, which includes $62,970.26 for direct labor and fringe and the remainder for overhead and fees. This is an annualized cost of $56,722.15 for contracted data collection. In addition, a portion of the costs are for personnel costs of several Federal employees involved in the oversight and analysis of information collection, amounting to an annualized cost of $2,180 for Federal labor. The total annualized cost for the assessment is therefore the sum of the annual contracted data collection cost ($56,722.15) and the annual Federal labor cost ($2,180), or a total of $58,902.15.
Explanation for Program Changes or Adjustments
This is a new collection of data.
Plans for Tabulation and Publication and Project Time Schedule
The results of this data collection will be tabulated and summarized in a final report that will be submitted to OWH. OWH plans to post this final report on its Web site. Additionally, the previously completed literature review and the resulting final report will be used to prepare five manuscripts for publication in peer-reviewed journals.
The remainder of this section describes the analytic techniques that will be employed. Information will be collected over a seven month period following OMB approval. Exhibit 3 provides a schedule of data collection, analysis, and reporting following OMB approval.
Exhibit 3. Timetable for Data Collection, Analysis, and Publication
Activity |
Estimated Start Date |
Estimated End Date |
Web-based survey |
1 month following OMB approval |
3 months following OMB approval |
Tabulation of Web-based survey results |
2 months following OMB approval |
4 months following OMB approval |
Telephone interviews |
4 months following OMB approval |
7 months following OMB approval |
Site visits |
8 months following OMB approval |
10 months following OMB approval |
In-depth data analysis |
7 months following OMB approval |
11 months following OMB approval |
Development of Final Report |
9 months following OMB approval |
16 months following OMB approval |
The assessment is designed to address the following research questions:
How has Federal program sustainability been defined and conceptualized?
What are the characteristics of programs that have sustained themselves after Federal funding expired? What factors are associated with sustainability?
How is program sustainability determined?
What types of guidelines for sustaining a Federal program could be included in future Federal grant and contract announcements?
The previously completed literature review addressed the first research question. Data from the abstraction of administrative data, Web-based survey, telephone interviews, and site visits will be analyzed to answer the remaining research questions. In addition, the analysis will test how well the MHMW Sustainability Framework captures the sustainability process. Subsequently, the findings will be used to revise the framework, which can then be used to make recommendations to promote the sustainability of Federally-funded programs.
Data will be analyzed using the following techniques:
Descriptive Analysis. Descriptive statistics will be applied to the Web-based survey data to describe the characteristics of the 48 programs and the extent to which components were implemented and have since been sustained.
QCA. QCA will be used to analyze Web-based survey and telephone key-informant interview responses for factors contributing to sustainability. QCA is a method for comparing information on sets of cases and is uniquely suited for this type of study, because it is based on the premise that specific combinations of factors contribute to particular types of outcomes. Also, it is well-suited to studies that include a relatively small number of cases and use both quantitative and qualitative data. The purpose of the QCA analysis is to determine whether particular configurations of factors are associated with sustainability in general as well as the sustainability of particular components.
Logistical Regression. Logistical regression will be used to measure the relative effect of specific variables on the probability that MHMW had been sustained at particular sites, controlling for other variables.
Content Analysis. The telephone interview responses will be coded to identify common themes and recommendations across sites. These data will be used to inform the interpretation of findings from the other analyses and to analyze the recommendations that are offered in the interviews.
The SAS statistical program will be used to organize and conduct analysis of quantitative data, and the NVivo software program will be used to code the qualitative data to identify themes.
Display of OMB Expiration Date is Inappropriate
OWH does not seek approval to eliminate the expiration date from the data collection instruments. All data collection materials will display the OMB expiration details.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification statement.
1. Respondent Universe and Sampling Methods
Primary data collection for which OMB approval is being sought includes the following activities:
A Web-based Survey. This study is surveying the universe of 48 MHMW grantees and contractors. Surveys will be completed primarily by core staff members (such as Center Directors, Program Directors, and Program Coordinators), who will likely have the most comprehensive information about various aspects of MHMW-related activities. However, in some cases, one or more core staff members may no longer work at a site. In such cases the contractor will try to locate these core staff so they can complete the portions of the survey related to MHMW implementation and sustainability planning that took place during their tenures. If they cannot be located or are unavailable to complete the survey then, the contractor staff will attempt to locate the next most knowledgeable individuals to complete the survey. These may include other staff members and community partners that were directly involved with MHMW-related activities.
Telephone Interviews. The contractor will conduct telephone interviews with all 48 MHMW grantees and contractors. Similar to the case for the Web-based survey, it is anticipated that core staff members will be the most appropriate staff to answer questions about MHMW-related activities during the interview. In instances where core staff are no longer employed at a site, the contractor will attempt to locate them and enlist their participation in the interviews. If these former core staff members cannot be located or if current staff members feel they are not knowledgeable about all aspects of their MHMW program, the contractor will work closely with them to identify other staff or community partners that may have greater knowledge of specific aspects of the MHMW program.
2. Procedures for the Collection of Information
These strategies are part of a mixed-method research design that addresses the complex nature of the sustainability process and that will ensure all of the major components of the MHMW Sustainability Framework are adequately measured. The research design will therefore require multiple contacts with each site and, in many cases, multiple contacts with the same individuals at each site. These multiple contacts are necessary to capture different types of information and inform subsequent stages of the assessment. In the first phase of the study, the contractor will abstract as much data as possible from grantee’s/contractors’ administrative documents to establish a baseline understanding of each program. In the following phases, the contractor will begin collecting primary data across all sites by administering a Web-based survey, conducting telephone interviews with key informants, and conducting site visits. During the collection of this data, the contractor will take a number of steps minimize burden on respondents.
Web-based Survey
The purpose of the survey is to gather data on the nature and status of implementation of the MHMW core model components, the extent to which these components have been maintained since the end of the funding period, and efforts to evaluate the effectiveness of core model implementation. The draft survey instrument may be found in Attachment A. The survey will consist of primarily close-ended questions with categorical response options. The survey is designed to collect data on the key dependent and a number of key independent variables in a uniform manner across sites. Categorical data will subsequently be used to group sites based on the level of core model implementation and sustainability they have attained. These groups will also be used during analysis to explore differences and similarities across the groups that may have contributed to the level of implementation and sustainability achieved and to select a diverse sample of 12 sites to visit.
To limit cost to the government and minimize burden on respondents, surveys will be conducted via a secured Web-site. Recruitment for the survey will occur through an initial notification letter from OWH and follow-up contacts via Email and telephone from the contractor. Respondents will be given a link to the Web-based survey and detailed instructions on how to enter and submit their responses. The Contractor will also offer technical assistance to respondents to help successfully complete the survey if needed. Once a survey has been submitted, the contractor’s survey manager will review the survey file to confirm that it is complete and, if necessary, follow-up with the site to fill in any missing information. Once surveys from sites have been submitted and approved, the survey manager will provide a copy of the SQL data file from the survey Website to the data analysts. The analysis team will then create a fully labeled data file in SAS to analyze the quantitative and categorical responses.
Telephone Interviews
The purpose of the telephone interviews is to gather more comprehensive information across all sites about MHMW program planning, factors affecting sustainability, program outcomes, and recommendations on how to help Federal grantees and contractors sustain their programs. The interview guide is located in Attachment C. The semi-structured interview guide consists of open-ended questions and series of probes to elicit more detailed responses.
To limit cost to the government and minimize burden on respondents, the interviews will be conducted over the telephone. Recruitment for the survey will occur through an initial notification letter from OWH and follow-up contacts via telephone and Email from the contractor. The contacts will be asked to identify appropriate interviewees and several possible dates and times to schedule the interviews; the contractor will make every effort to accommodate interviewees’ schedules. Prior to the scheduled interview date, interviewers will review the site profiles that were previously developed as part of the abstraction of administrative data and a summary of the site’s responses to the Web-based survey. This site-specific information will be used to familiarize interviewers with their assigned sites’ activities and to pre-populate one section of the protocol with previously reported data.
The telephone interviews will be audio recorded pending informed consent of the interviewees. During the interview the interviewer will also take detailed typed or written notes using a standard template. Following each interview, interviewers will be required to finalize and submit their notes within 24–48 hours. Interviewers will be able to refer to audio recordings of the interviews to ensure that they have captured complete responses in their notes. Data from the interviews will be compiled in the NVivo qualitative software program for analysis across sites.
Site Visits
Although OMB clearance is not being sought for the site visit protocols, it is important to note that efforts have also been made to minimize burden during this phase of data collection as well. First, site visits will only be a conducted with a sample of 12 of the 48 sites. Next, the protocol development process will first involve a review of the initial program document abstractions, Web-based surveys, and the telephone interviews to identify any data that are missing or information that needs to be clarified. The site visit protocols will then be tailored to fill these gaps while avoiding duplication of information. During the site visits, data will be collected from a wider range of respondents than earlier data collection methods which primarily target core staff members; the site visits will involve discussions with support staff, community partners, and clients as well.
Exhibit 4 provides an estimated timeline of data collection activities.
Exhibit 4: Timeline of Data Collection Activities
Activity |
Estimated Timeline |
Web-based survey |
1-3 months following OMB approval |
Telephone interviews |
4 to 7 months following OMB approval |
3. Methods to Maximize Response Rates and Deal with Nonresponse
Gaining buy-in and cooperation from MHMW program sites are key requirements for this assessment. We believe that most sites will participate in the study and support OWH’s effort to improve understanding of key factors that drive sustainability and to develop guidelines to promote greater Federal program sustainability. It is important to note that the MHMW program guidance encouraged sites to plan for sustainability throughout the funding period and a number of sites were required to develop and submit a formal sustainability plan to OWH. However, it is very likely that some sites may be disinclined to participate in the data collection activities because they no longer have fully functioning MWHW programs, lost their Federal designation as a recognized MHMW program, or do not feel obligated to do so since funding from OWH has ended. The targeted response rate for both the Web-based survey and the telephone interviews is therefore 85–90 percent.
Every effort will be made to meet or exceed the targeted response rate for data collection. A notification letter from the OWH Project Officer will be sent to sites to introduce the study, explain its importance to gaining a better understanding of Federal program sustainability, and encourage sites to participate. Contractor staff will make a series of follow-up contacts with each site by telephone and Email to reinforce the importance of participation, answer questions, and offer any support needed.
In addition, the contractor will use the following proven methods to maximize participation and cooperation in the study:
Paper copies of the survey instrument will be distributed to sites that prefer not to utilize the survey Web-site
Contractor staff will provide technical support if respondents encounter difficulties completing the Web-based survey
Throughout the telephone interview recruitment process, staff will work with sites to accommodate the schedules of all interviewees. Multiple staff will be available in each of the five interview teams (one for each of the five MHMW programs: CoE, CCOE, AFC, R8Demo, and RFCC), which will increase the likelihood that at least one contractor team member will be available at times specified by sites. Also, interviewers will make themselves available in both the early morning and evening to accommodate sites in different time zones.
Telephone interviewees will be provided with the discussion questions in advance to allow them sufficient time to prepare.
All sites and respondents will be given a realistic appraisal of the amount of contributions in time, information, and human resources that will be expected.
In previous projects involving similar data collection activities, these steps produced collegial relationships between the research team and participants and contributed to high response rates.
4. Tests of Procedures or Methods to be Undertaken
Prior to the launch of the Web-based survey, a pilot test will be conducted to ensure that the data collection process proceeds as intended. A link to a beta version of the survey will be sent to 3-4 grantees and contractors that agree to pilot-test it. This process will ensure that questions elicit interpretable and intended responses and address any logistical issues with completion of the survey on the Website and data processing. In addition, the contractor will conduct pilot-test telephone interviews with the same or a different set of 3-4 grantees and contractors that are willing to participate. Similarly, this process will ensure that questions are easy to understand and produce intended responses.
The survey and interview instruments will be revised based on information gleaned from the pilot tests and submitted to OMB for review, along with a refined estimate of the response burden.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
The following individuals contributed to the questionnaires and study design and will be involved in the interpretation and analysis of findings:
Christopher Botsko, MA
Senior Associate
Co-Lead Women, Children, and Adolescents Subject Area
Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(P) 202-828-5100
(F) 202-728-9469
Amy Brown, MPH
Senior Policy Associate
Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(P) 202-828-5100
(F) 202-728-9469
Namratha Swamy, PhD
Director of the Policy, Planning, and Evaluation Practice Area
Altarum Institute
1200 18th Street NW, Suite 700
Washington, DC 20036
(P) 202-828-5100
(F) 202-728-9469
The government project officer for this study is:
Anna L. Kindermann, JD
Public Health Analyst
DHHS Office on Women's Health
5600 Fishers Lane
Parklawn Room 16A-55
Rockville, Maryland 20857
(P) 301-443-1383
(F) 301-443-1384
Attachments for Supporting Statement, Parts A and B
Attachment A: Overview of the Multidisciplinary Health Models for Women (MHMW) Initiative
Attachment B: Web-based Survey Instrument
Attachment C: Telephone Interview Discussion Guide
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | “Defining Sustainability of Federal Programs Based on the Experiences of the |
| Author | Amy Brown |
| File Modified | 0000-00-00 |
| File Created | 2021-02-03 |