SSA - Title X Implementation Study

The Title X Service Grants Program, the only federal program solely dedicated to family planning and reproductive health, has a mandate to provide a broad range of acceptable and effective family planning methods and services. Supported services include individualized counseling, pregnancy testing, and helping couples achieve their desired family size. Administered by the Office of Population Affairs (OPA) in the U.S. Department of Health and Human Services (HHS), Title X prioritizes services for people with low incomes who might otherwise lack access to family planning care. In 2019, Title X clinics provided services to more than 3.9 million clients through a national network of 3,954 service providers.¹ About two-thirds of these clients had family incomes below the federal poverty level, and one in four lacked health insurance.

With this new Information Collection Request (ICR), OPA seeks Office of Management and Budget (OMB) approval over three years for data collection activities necessary to conduct an implementation study of Title X grantees. The study will document (1) how grantees ensure access to equitable, affordable, client-centered, high quality family planning services; (2) the steps Title X grantees take to give clients from diverse communities equitable access to affordable, high quality, client-centered health services; (3) any pivots and/or accommodations they made in recent years to keep providing care, including during the COVID-19 pandemic; and (4) how they assess their impact.

A.1. Circumstances Making the Collection of Information Necessary

The Title X Service Grants Program was created in 1970 and is authorized under the Public Health Service Act (42 United States Code [USC] 300). Section 1001 of Title X (“Project Grants and Contracts for Family Planning Services”) enables OPA to award grants to assist in the establishment and operation of voluntary family planning projects that provide a broad range of acceptable and effective family planning methods and related preventive health services. Services include natural family planning methods, infertility treatment, and services for adolescents; effective contraceptive methods; breast and cervical cancer screening and prevention services that correspond with nationally recognized standards of care; education, counseling, testing and referral on preventing STIs and HIV; abstinence counseling for adolescents; and other preventive health services. In addition, Section 1004 of Title X (“Research”) enables OPA to award grants and contracts for research in the biomedical, contraceptive development, behavioral, and program implementation fields that is related to family planning and population. Projects under this Section conduct data analysis and related research and evaluation on issues of interest to the family planning field, as well as research into specific topic areas related to improving service delivery. The proposed Title X Implementation Study meets these requirements by supporting research in the program implementation field and specific topic areas related to improving service delivery. Attachment A to this statement contains a copy of the authorizing Title X program regulations that necessitate the collection of the information.

A.2. Purpose and Use of the Information Collection

The Title X Implementation Study has four main objectives. The first is to describe how Title X grantees ensure access to equitable, affordable, client-centered quality family planning services. The study team will document and describe the service delivery strategies that have been put in place to ensure grantees’ services are in alignment with the program’s pillars of access, quality, and equity. This will include documenting (1) the structure of clinical encounters in different settings and (2) how the types of Title X services vary by sub-recipient and service delivery site.

The second study objective is to describe the steps Title X grantees take to provide clients from diverse communities with equitable access to affordable, high quality, client-centered health services. For the Title X program to achieve its desired outcomes, grantees must be able to provide diverse populations with access to high quality family planning services. To understand the program’s effectiveness in achieving this goal, the study team will explore the various strategies grantees, sub-recipients and service sites have used to ensure they reach and serve a range of populations, including adolescents, LGBTQ+, Indigenous communities, persons with limited English proficiency, and those living in rural and remote settings.

The third study objective is to describe pivots and accommodations made by Title X grantees and sub-recipients to provide care in recent years, including during the COVID-19 pandemic. The study team will summarize the pandemic’s impact on grantees, sub-recipients, and service delivery sites. The study will focus on understanding which pivots, accommodations, and innovations in service delivery introduced during the pandemic can inform future efforts to expand equitable access to quality family planning care by placing the client at the center.

The fourth study objective is to describe grantees’ self-assessments of impact. The study team will document grantees’ processes for monitoring and evaluating their effectiveness in meeting each client’s desired family planning goals. This will include gaining a better understanding of how grantees use performance measures data, findings from site visits and any correction action plans, and other assessments to evaluate the effectiveness of their work and their capacity to monitor and ensure federal quality standards.

To carry out these objectives, the study team will rely on the following five proposed data sources: (1) a web-based survey of the 2022 cohort of Title X grantees (Instrument 1); (2) grantee telephone interviews (Instrument 2); (3) in-person or virtual listening visits with clinic administrators (Instrument 3), service providers (Instrument 4), and community outreach or partner staff (Instrument 5) at a subset of Title X sub-recipients and service delivery sites; (4) an intercept survey of up to 30 clients at up to 10 of the sites selected for listening visits (Instrument 6); and (5) telephone interviews with subject matter experts (Instrument 7). These proposed data sources are summarized in Table A.2. Attachment B shows how the proposed data sources map to each of the four study objectives.

OPA will use the information to contribute to research in the fields of program implementation and service delivery improvement, as specified under Section 1004 of Title X. The information is not intended to be used as the basis for making grant awards or monitoring or evaluating the performance of current Title X grantees.

Table A.2. Overview of data sources and associated data collection instruments

Data source	Instrument(s)	Respondent and purpose of collection
Web-based survey of the 2022 cohort of Title X grantees	1. Grantee web survey	Respondents: Title X project directors Purpose: Collect information on grantee service provision, project administration, billing and payment practices, strategies for reaching diverse communities, recent pivots and accommodations made to service provision, and self-assessments of impact.
Grantee telephone interviews	2. Grantee interview topic guide	Respondents: One or two designated grantee staff members (project directors, medical directors, nurse leads, or other medical professionals) Purpose: Collect information on staff responsibilities and roles, clarify and probe for details on responses to the grantee survey, and identify potential candidates for in-person or virtual listening visits.
In-person or virtual listening visits	3. Listening visit topic guide for clinic administrators 4. Listening visit topic guide for clinical service providers 5. Listening visit topic guide for community outreach or partner staff	Respondents: Clinic administrators, clinic service providers, and community outreach or partner staff at a subset of Title X sub-recipients or service delivery sites. Content: Collect information on the implementation of family planning and related preventive health services, outreach and community engagement, and training and technical assistance.
Web-based client survey	6. Client survey	Respondents: Title X clients who speak English or Spanish Purpose: Collect information on clients’ perspectives on access to and quality of care.
Telephone interviews with subject matter experts	7. Subject matter expert topic guide	Respondents: Title X program subject matter experts Purpose: Collect information on the evolution of the Title X program and current approaches to service delivery in the family planning field.

A.3. Use of Information Technology to Reduce Burden

To reduce respondent burden and make data processing and reporting faster and more efficient, OPA is using technology to collect and process data. The contractor will program and administer the grantee survey and client survey (Instruments 1 & 6) with Confirmit, a state-of-the-art survey software platform the contractor uses to program and launch surveys. The surveys will be web based, and program directors will receive an email with a unique URL to access the survey. The Confirmit software platform has built-in mobile or tablet formatting to ensure the display adjusts for device screen size. If needed, respondents can pause and restart the survey, and their responses will be saved after each session. Confirmit includes tailored skip patterns and text fills when appropriate. These features allow respondents to move through the questions easily and automatically skip questions that do not apply to them, thus minimizing burden.

A.4. Efforts to Identify Duplication and Use of Similar Information

The information collection requirements for the Title X Implementation Study have been carefully reviewed to avoid duplication with existing and ongoing studies. This will be the first external implementation study of the Title X Service Grants program.

A.5. Impact on Small Businesses

For in-person or virtual listening visits, data will be collected from staff at Title-X-funded sub-recipients and service sites. Some of these organizations might be small entities. To reduce burden on the staff of these entities, the study team will schedule data collection activities at times that are convenient for them, and questions in interviews will be held to the minimum required for the intended use of the data.

A.6. Consequences of Not Collecting the Information/Collecting Less Frequently

OPA intends to collect information only once, so it has planned no repetition of effort. Not collecting the information at all would substantially limit OPA’s ability to carry out the research activities specified under Section 1004 of Title X.

A.7. Special Circumstances

There are no special circumstances for the proposed data collection.

A.8. Federal Register Notice and Consultation Outside the Agency

A 60-day Federal Register Notice was published in the Federal Register on May 19, 2022, vol. 87, No. 97; pp. 30507 (see Attachment I). There were no public comments.

A 30-day Federal Register Notice is included with this submission.

OPA consulted with staff from Mathematica (contractor for the Title X Implementation Study), JSI Research & Training Institute (contractor for OPA’s Reproductive Health National Training Center), three Title X service grantees (Missouri Family Health Council, Inc., Louisiana Department of Health, and Virginia Department of Health), Power to Decide, and Planned Parenthood Federation of America in developing the data collection instruments.

A.9. Payments to Respondents

No respondent payments are proposed for the grantee survey (Instrument 1); grantee telephone interviews (Instrument 2); in-person or virtual listening visits with clinic administrators (Instrument 3), service providers (Instrument 4), and community outreach or partner staff (Instrument 5); or telephone interviews with subject matter experts (Instrument 7).

For the intercept survey of Title X clients (Instrument 6), OPA proposes giving a $10 gift card to participants who complete the survey. The offer of the gift card may encourage a more diverse group of people to participate, not just those who are the most engaged or motivated. In general, gifts and rewards have been found to be important for encouraging participation in federal surveys, especially for more reluctant responders²^,³^,⁴and those with low incomes.⁵ Increasing response rates for populations that are otherwise not motivated to participate improves equity in research, addresses and mitigates nonresponse bias, and provides fair compensation to participants for their willingness to accept the burdens related to participating in research,⁶ including time and inconvenience. In the case of Title X clients, the Family Planning Annual Report reveals 88 percent of family planning clients at Title X clinics had a family income at or below 250 percent of the federal poverty line.⁷ In interviews with grantee administrators during the study design (n = 2), grantee leaders pointed out that offering incentives will encourage participation by clinical staff who will be handing out survey information to clients. Clinic staff encouragement to clients should boost the response rate.

A.10. Assurance of Confidentiality

The contractor will report data collected through the Title X Implementation Study in two ways, and assurances of privacy will reflect the different requirements of those two ways.

First, the contractor will create a summary profile for each Title X grantee containing a factual description of the grantee’s project design—for example, the intended service population and the number of service delivery sites. The contractor will report this information for each organization receiving Title X funding. Therefore, although the information will not be attributed to a specific respondent, it will be attributed to a specific grantee. Respondents will be made aware of how this information will be reported.

Second, the contractor will prepare summary reports on themes emerging from responses, such as the strategies used by Title X grantees to ensure access to equitable, affordable, client-centered, high quality family planning services. These responses will not contain confidential information. The contractor will notify respondents it will not attribute their responses to these questions to them or to their grantees.

For telephone interviews, the contractor will ask respondents’ permission to record the interview. Following each interview, the interviewer will review their notes and refer to the recording as needed to fill in unclear or missing details. At the start of the client survey, respondents will be informed of all planned uses of data and that their participation is voluntary. The institutional review board, Health Media Labs, has approved the data collection protocol, instruments, and consent forms. The contractor will submit amended materials to the institutional review board as needed upon OMB approval of the study.

All electronic data will be transmitted and stored according to the level of security appropriate for the sensitivity and identifiability of the data. The grantee web-based survey will be password protected, with a unique user name and passcode for each respondent. The web-based survey of Title X clients will include a unique URL that ensures the survey can only be completed one time. Clients responding to the survey will not be asked for their names or for personally identifiable information other than their email address. Email addresses will be stored in secure files, separate from survey data. Email addresses will only be used for delivery of gift cards. Responses to all data collection instruments will be stored by the study contractor (Mathematica) on secure network servers, with access limited to project staff on a need-to-know basis.

A.11. Justification for Sensitive Questions

For the web-based client survey, participants will be informed that their identities will be kept private, and they do not have to answer questions that make them uncomfortable. Table A.11 lists the sensitive topics on the survey, along with the justification for including each topic.

Table A.11. Summary of sensitive topics to be included on the client survey, and justification for including them

Topic	Justification
Race and ethnicity	The study will collect demographic information—including race and ethnicity—from clients to help assess nonresponse bias.
Gender identity	The study will collect demographic information—including gender identity—from clients to help assess nonresponse bias. With the equity focus across both the Title X program and evaluation, these questions will also help the study team understand more about the unique experiences of clients with diverse gender identities.
Reason for clinic visit	The study will collect the reason for the clinic visit to understand which Title X services were utilized and to allow the study team to see if there are differences in experience or satisfaction based on the type of service received.

A.12. Estimates of the Burden of Data Collection

OPA is requesting three years of clearance for the Title X Implementation Study. The estimated annual reporting burden for study participants is in Table A.12.1.

Grantee web survey. Grantee web surveys will be administered one time to all project directors of grantees funded in 2022 (n = 89). Based on experience with similar questionnaires, grantees should take about 1 hour to complete the survey, on average, for a total burden of 89 hours. The annual data collection burden for the grantee web survey is estimated to be 30 hours (89 hours/3 years).

Grantee telephone interviews. The contractor will complete telephone interviews with one or two designated staff members (in a group setting) for grantees funded in 2022, for a total of 178 interviews (89 grantees*2 staff members per grantee). Each interview is expected to take 90 minutes (90/60 hours), yielding a total burden of 267 hours (178 responses*90/60) and an annual burden of 89 hours (267/3 years).

The contractor will conduct in-person and/or virtual listening visits with staff from a subset of up to 40 sub-recipients and service delivery sites:

Listening visits, clinic administrator interviews. The contractor will complete listening visit interviews with one or two clinic administrators per site, for a total of 80 interviews (40 sites*2 clinic administrators), 27 annually. Each interview is expected to last 45 minutes (45/60 hours), for a total burden of 60 hours (80 responses*45/60 hours) and an annual burden of 20 hours.
Listening visits, clinical service providers. The contractor will complete listening visit interviews with an average of 4 clinical service providers per site, for a total of 160 interviews (40 sites*4 clinical service provides), 53 annually. Each interview is expected to last 1 hour, for a total burden of 160 hours and an annual burden of 53 hours.
Listening visit, community outreach or partner staff interviews. The contractor will complete listening visits with one or two community outreach or partner staff per site selected, for a total of 80 interviews (40 sites*2 community partner and outreach staff), 27 annually. Each interview is expected to last 45 minutes (45/60 hours), for a total burden of 60 hours (80 responses * 45/60 hours) and an annual burden of 20 hours.
Client web survey. The contractor will complete up to 30 client surveys at up to 10 of the sites selected for listening visits, for a total of 300 clients (10 sites*30 clients). Each survey is expected to take 10 minutes to complete (10/60 hours) for a total burden of 50 hours and an annual burden of 17 hours.
Subject matter experts. The contractor will complete interviews with up to 25 subject matter experts. Each interview is expected to take 1 hour to complete, for a total burden of 25 hours. The annual burden for these interviews is 8 hours.

Table A.12.1. Calculations of annual burden hours

Instrument	Type of respondent	Annual number of respondents	Number of responses per respondent	Average burden hours per response	Annual burden hours
1. Grantee web survey	Project director	30	1	1	30
2. Grantee interview topic guide	Grantee staff	59	1	90/60	89
3. Listening visit topic guide for clinic administrators	Clinic administrator	27	1	45/60	20
4. Listening visit topic guide for clinical service providers	Clinician, R.N., L.P.N., M.A., health educator, public health nurse	53	1	1	53
5. Listening visit topic guide for community outreach and partner staff	Clinic community outreach and partner staff	27	1	45/60	20
6. Client survey	Title X client	100	1	10/60	17
7. Subject matter expert topic guide	Title X subject matter expert	8	1	1	8
Estimated annual burden: Total		304	7		237

Total annual cost

The average hourly wage for Title X project directors and other grantee respondents is estimated to be $57.61, based on the average hourly wage for medical and health services managers as determined by the U.S. Bureau of Labor Statistics Occupational Employment and Wage Statistics for 2021.⁸ The total annual cost of including Title X program directors and other grantee respondents is estimated to be $6,855.59 ($57.61*119 annual hours). The average hourly wage for clinic administrators is estimated to be $57.61, based on the average hourly wage for medical and health services managers. The total annual cost of including clinic administrators is estimated to be $1,152.20 ($57.61*20 annual hours). For clinical service providers, which includes doctors, nurses, medical assistants, and public health nurses, the average hourly wage is estimated to be $43.80, based on the average hourly wage of health care practitioners and technical occupations. The total annual cost of including clinical service providers is estimated at $2,321.40 ($43.80*53). For community partners and outreach staff, the average hourly wage is estimated to be $25.94, based on the average hourly wage of community and social service occupations. The total annual cost of including community partner staff is estimated at $518.80 ($25.94*20). The average hourly wage for the Title X subject matter experts is estimated to be $43.66, based on the average hourly wage of social scientists and related workers. The total cost for including the subject matter experts is estimated to be $349.28 ($43.66*8). The average hourly wage for Title X clients is estimated to be $7.25, based on the federal minimum wage. The total annual cost for Title X clients is $123.25 ($7.25*17 annual hours).

The estimated total annual cost burden is $11,320.52 (Table A.12.2).

Table A.12.2. Annualized cost to respondents

Type of respondent	Total burden hours	Hourly wage	Total respondent costs
Title X project director	119	$57.61	$6,855.59
Clinic administrator	20	$57.61	$1,152.20
Clinical service provider	53	$43.80	$2,321.40
Community partner and outreach staff	20	$25.94	$518.80
Title X client	17	$7.25	$123.25
Title X subject matter expert	8	$43.66	$349.28
Total	237		$11,320.52

A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

These information collection activities do not place any capital cost or cost of maintaining requirements on respondents.

A.14. Annualized Cost to Federal Government

Data collection and analysis will be carried out by the contractor for the Title X Implementation Study (Mathematica). OPA staff will not be involved in either data collection or data analysis; thus, there are no agency labor or resources involved in conducting this study. The total and annualized costs to the federal government are in Table A.14.

Table A.14. Annualized cost to federal government by cost category

Cost category	Estimated cost
Instrument development and OMB clearance	$41,549
Data collection	$249,238
Analysis and reporting	$111,867
Total annualized costs over the request period	$402,654

A.15. Explanation for Program Changes or Adjustments

This is a new data collection.

A16. Plans for Tabulation and Publication and Project Time Schedule

1. Analysis plan

For the quantitative data, the contractor will calculate mean values across grantees for continuous variables and percentages for categorical and dichotomous variables. The contractor will use qualitative coding software such as NVivo to organize and code the qualitative data from open-ended web survey questions and qualitative interviews. The contractor will also look for opportunities to construct quantitative variables from qualitative open-ended responses.

2. Time schedule and publications

Table A.16 shows the tentative timeline for data collection and reporting activities. The grantee web survey will be fielded in fall 2022, pending OMB approval. Grantee phone interviews will start on a rolling basis, two to three weeks after a grantee completes their web survey. In-person and virtual listening visits will begin in early winter 2023 after initial preliminary analysis of the grantee web survey and interview data.

Data from the analysis of the grantee web survey and interview data will also inform the development of detailed grantee profiles, which will begin in January 2023. A final report and case studies based on all implementation data will be completed in fall 2023 or winter 2024.

Table A.16 Schedule for Title X Implementation Study

Activity	Timing^a
Data collection
Grantee web survey	October–December 2022
Grantee phone interviews	December 2022–January 2023
In-person or virtual listening visits and client survey	January–June 2023
Subject matter expert interviews	July–September 2023
Reporting
Grantee profiles	January–September 2023
Report	October 2023
Case studies	October 2023–January 2024

^aSubject to timing of OMB approval.

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

All instruments will show the OMB Control Number and expiration date.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 Fowler, C.I., J. Gable, B. Lasater, and K. Asman. “Family Planning Annual Report: 2019 National Summary.” Washington, DC: Office of Population Affairs, Office of the Assistant Secretary for Health, Department of Health and Human Services, 2020.

2 Berry, S.H., J. Pevar, and M. Zander-Cotugno. “Use of Incentives in Surveys Supported by Federal Grants.” Rand Corporation, March 2008. Available at http://www.copafs.org/seminars/use_of_incentives_in_surveys.aspx.

3 Singer, E., and C. Ye. “The Use and Effects of Incentives in Surveys.” Annals of the American Association of Political and Social Science, vol. 645, 2013, pp. 112–141. Available at http://journals.sagepub.com/doi/pdf/10.1177/0002716212458082.

4 Singer, E., and R.A. Kulka. “Paying Respondents for Survey Participation.” In Studies of Welfare Populations: Data Collection and Research Issues.105-28. Washington DC: National Academies Press, 2002. Available at https://aspe.hhs.gov/system/files/pdf/174381/04.pdf.

5 McGonagle, K.A., and V.A. Freedman. “The Effects of a Delayed Incentive on Response Rates, Response Mode, Data Quality, and Sample Bias in a Nationally Representative Mixed Mode Study.” Field Methods, vol. 29, no. 3, August 2017, pp. 221–237.

6 Gelinas, L., E. Largent, G. Cohen, S. Kornetsky, B. Bierer, and H. Fernandez Lynch. “A Framework for Ethical Payment to Research Participants.” New England Journal of Medicine, vol. 378, February 2018, pp. 766–771.

7 Fowler, C. I., J. Gable, B. Lasater, and K. Asman. “Family Planning Annual Report: 2019 National Summary.” Washington, DC: Office of Population Affairs, Office of the Assistant Secretary for Health, Department of Health and Human Services. 2020.

8 U.S. Bureau of Labor Statistics. May 2021 National Occupational Employment and Wage Estimates (bls.gov).

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Mathematica Report Template
Author	Sharon Clark
File Modified	0000-00-00
File Created	2022-08-11