Supporting Statement B HCPS Revised TC PB 2 3-20

Supporting Statement B

Health Center Patient Survey: National Study

OMB Control No. 0915-0368

Reinstatement

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The respondents for the survey are patients who receive services from Section 330-funded Health Center awardees. These respondents are clustered within health centers in each awardee. The awardee, health center, and patient universes are described in greater detail in the subsections that follow. Please see Attachments 7 and 8, the Statistical Design Plan and the Survey Methodology and Selection Specifications for further detail.

Awardee Universe

The awardee sample will be selected from eligible awardees. Health Resources and Services Administration (HRSA) will use the 2018 Uniform Data System (UDS) awardee-level data file to identify eligible awardees. All awardees within the UDS and funded by Section 330 will be eligible except the following:

• Awardees located in U.S. territories or possessions (i.e., those in Puerto Rico, the Virgin Islands, and the Pacific Basin);

• Awardees funded by Section 330 that have been operating less than 1 year;

• Awardees funded through the Community Health Center (CHC) funding program that only operated school-based sites;

• Awardees that received Migrant Health Center (MHC) funding program only and that served clients only through a voucher program; and

• Any awardee that is no longer a Section 330-funded awardee.

Site Universe

Many awardees operate multiple health center sites. The sites eligible for the survey within selected awardees are required to participate in at least one of the four specific funding programs: CHC, Health Care for the Homeless (HCH), MHC, and Public Housing Primary Care (PHPC). All sites within selected awardees are eligible unless:

• The site has been operating under the awardee for less than 1 year.

• The site is a school-based health center.

• The site is a specialized clinic, excepting clinics providing OB/GYN service.

• The site provides services only through a migrant and seasonal farmworker voucher-screening program.

• The site serves fewer than 100 patients.

Patient Universe

The HCPS patient sample is a nationally representative sample through three stages of probability sampling, namely: 1) selecting awardees at the first stage, 2) selecting health center sites within awardees at the second stage, and 3) selecting patients from health center sites at the third stage. We apply some inclusion/exclusion rules of awardees and health center sites when constructing sampling frames for eligible awardees and health center sites. 

The patients eligible for the survey must satisfy the following eligibility requirements:

• They must have received face-to-face services at the site from a clinical staff member who exercises independent judgment in the provision of service at least once in the 12 months prior to the current visit.

• If they are under the age of 18, they must be accompanied by an adult.

• They must speak any one of the languages offered in the survey (English, Spanish, Chinese, Vietnamese, and Tagalog).

For data collection cost consideration, we also exclude health center sites with low patient volume. In addition, to ensure that the HCPS patient sample represents the target population in the United States, we will adjust the sample weights for all responding patients to the patient counts for the UDS

2. Procedures for the Collection of Information

Sample

As noted, the Section 330-funded awardees operate one or more sites. The sites are clustered within awardees, and the patients are clustered within the sites within the awardee. HRSA will employ a three-stage sample design in which the awardees are selected as the primary sampling units (PSUs), sites are selected within participating awardees, and patients are selected within selected sites. HRSA’s goal is to recruit 210 awardees and complete approximately 9,000 interviews, among them 5,100 for the CHC funding program, 1,480 for the MHC funding program, 1,660 for the HCH funding program, and 760 for the PHPC funding program. In addition, to meet HRSA’s research interests in race/ethnicity groups, HRSA will oversample patients of American Indian/Alaska Native (AIAN), Native Hawaiian/Pacific Islanders (NHPI), and Asian race groups. HRSA will also oversample veterans and patients aged 65 and older.

The target sample sizes in three design domains, namely funding program, race/ethnicity and age group, are shown in Table 1.

Table 1. Target Sample Sizes for the 2019 Health Center Patient Survey

Funding Program	Target Sample Size	Race / Ethnicity	Target Sample Size	Age Group	Target Sample Size
CHC	5,100	Hispanic	3,170	0–17	2,130
MHC	1,480	Non-Hispanic White	2,250	18–64	5,770
HCH	1,660	Non-Hispanic Black	1,920	65+	1,100
PHPC	760	Non-Hispanic AIAN	670
		Non-Hispanic Asian	650
		Non-Hispanic NHPI	200
		Non-Hispanic Others	140

Awardee Sample

The awardee sampling frame is constructed from all the eligible awardees (the awardee universe) in the HRSA’s 2018 UDS data file. The target is to recruit 210 awardees to the survey, HRSA will select a total of 280 awardees (assuming a 75% awardee recruitment rate) through a stratified probability proportional to size (PPS) for participation from seven strata with a prespecified sampling rate for each stratum using PROC SURVEYSELECT in SAS 9.4. Independent site and patient samples will be selected for each funding program if the awardee receives multiple Section 330 funds. The seven strata will be formed as shown in Attachment 9. They are defined as follows:

(1) First-Stage Strata

Four mutually exclusive first-stage strata are used to ensure that the selected awardees are representative to the four funding programs.

• Stratum 1: Awardees received PHPC funding solely or in combination with other programs.

• Stratum 2: Awardees received MHC funding solely or in combination with other programs.

• Stratum 3: Awardees received HCH funding solely or in combination with other programs.

• Stratum 4: Awardees received CHC funding solely.

(2) Second-Stage Strata

Stratum 1 (PHPC Stratum) was further divided into three second-level strata according to the proportion of PHPC patients in an awardee, and awardee size (large and small). AIAN, Asian, and NHPI patients are not evenly distributed among all awardees. They tend to be clustered in a few awardees. To achieve target sample sizes in three race/ethnicity groups, awardees with concentrated patients in those three race/ethnicity categories must be obtained and selected at the first-stage selection. Awardees with more than 20% of patients in one of the three race/ethnicity categories are considered patient-concentrated awardees. In the 2018 UDS, Stratum 4 (CHC funding solely) had over 88% of such awardees, and very few such awardees were from Strata 1, 2, and 3. Therefore, to effectively select awardees with concentrated patients in three race/ethnicity categories, Stratum 4 is further divided into two second-level strata according to whether an awardee has concentrated patients (over 20%) in any one of the three oversampling race/ethnicity categories.

Site Sample

Once the awardees are recruited, our recruiters will work with the awardee’s administration staff to identify eligible sites using the eligibility criteria discussed in Health Center Site Universe. Specifically, HRSA will collect the following information from each participating awardee on their sites (using the Awardee Recruitment Guidelines):

• Number of eligible sites serving each patient type (i.e., migrant and seasonal farmworkers, homeless, public housing, and general patients);

• Address and contact information for each eligible site;

• Number of patients served in each eligible site, overall and by type of patient (CHC, MHC, HCH, and PHPC);

• Sites with patients concentrated in one of the three race/ethnicity categories (AIAN, Asian, or NHPI);

• Sites with high concentration of patients 65 and older; and

• Sites with higher concentrations of veteran patients.

To achieve the target sample sizes of AIAN, Asian, and NHPI patients, HRSA will not only oversample awardees with higher concentrations of patients in these three race groups at the first stage of selection but will also identify sites with higher concentration of patients in at least one of the three targeted race/ethnicity categories. Sites with patients with higher concentrations of patients 65 and older or veteran patients will also be identified. These sites will be selected with higher probabilities than sites with lower concentrations of these patients.

HRSA will select a maximum of three sites for each funding program within an awardee.

When sites within an awardee have low patient volume for a funding program, HRSA may allow selecting more than three sites so that it is easier to meet the patient interview quota for that awardee. If three or fewer sites are serving a patient population type (i.e., migrant/seasonal farmworkers, homeless, public housing, and general community), all sites will be included in the sample unless the distance between sites is too great for one interviewer to cover all of them.

For awardees with more than three sites for a specific funding program, HRSA will use a PPS sampling method to select three or more sites out of all the sites that are within a reasonable distance for one interviewer to cover. The number of patients for the sites of a specific funding program will serve as the size measure in the PPS sampling. The PPS selection is independent for each funding program in the awardees with multiple funded programs. HRSA will select three sites using the PPS sample using PROC SURVEYSELECT in SAS 9.4.

Materials for awardee and site recruitment, training site staff, and data collection are included in Attachment 10. These include introductory letters and/or materials for the awardees and sites.

Patient Sample

Because some of the target populations for this study are fairly mobile, a random sample of patients will be chosen for interview as they enter the site and register with the receptionist for services. The survey patients will be selected using onsite recruitment procedures. To protect the patient’s rights, project staff (non-health center staff) will not be allowed to approach any of the health center’s patients or obtain any identifying information about a patient unless the receptionist at the health center has provided information about the study and the patient has expressed interest.

Patient Sample Allocation. HRSA will evenly allocate the targeted number of completed interviews for each funding program to all the awardees serving a special population. For example, if HRSA recruits 57 awardees that serve farmworkers (MHC), then 26 (1,480/57

is rounded to 26) interviews will be completed for each awardee. We will use two different methods to allocate the number of patient interviews for each site between awardees with

three or fewer sites in a funding program, and awardees with more than three sites in a funding program. For awardees with three or fewer sites, HRSA will select all sites and allocate the number of patient interviews within that awardee proportionally to the patient size of the sites. For awardees with more than three sites that are selected through PPS, HRSA will equally divide the number of selected patients among three or more selected sites.

Patient Screening and Referrals. We will design a screening form that the receptionist can use to screen and refer patients when a patient enters the site and registers for service. Patients are considered eligible if they received services at least once in the past 12 months prior to the current visit. Patients under 18 must be accompanied by parent or guardian. The receptionist will ask eligible patients questions about patients’ race/ethnicity, age and veteran status to determine whether they belong to the oversampling groups as shown in Table 2.

Table 2. Oversampling and Non-oversampling Patient Group

Patient Subgroup	Patient Aged 64 and Younger	Patients Aged 65 and Older
Race/Ethnicity
AIAN	Yesa	Yes
Asian	Yes	Yes
NHPI	Yes	Yes
Races Other Than AIAN, ASIAN, and NHPI	Nob	Yes
Veteran Status
Veteran	Yes	Yes
Non-Veteran	No	Yes

^aYes – oversampling; ^bNo – non-oversampling

During the time when an interviewer is at a site, the receptionist will be asked to use the Patient Arrival and Referral Tracking form (Attachment 11) to screen the patient’s eligibility when a patient enters a site and registers for service. The receptionist will keep track of the patient’s age, race/ethnicity, veteran status, the number of patients who enter the site, the number of patients who are eligible, and number of patients referred while the field interviewer (FI) is at the site to conduct data collection for each patient group. The receptionist will either use tally marks to count patients as they enter or complete a table based on the sign-in sheet or appointment list before the FI leaves the site.

FIs will send the patient tracking forms to RTI for data entry, and counts will be used to calculate the analysis weights for the study. If sites have more than one receptionist, all receptionists will track the number of patients who visit and are eligible. If the site has a high proportion of patients in any of the oversampled subgroups all receptionists will be asked to refer patients to the FI. If the site does not have patients in the oversampled subgroups, only one receptionist will refer patients to the FI.

Patient Referral. If a patient belongs to a group that will not be oversampled, the receptionist will refer the first eligible patient registered after the FI has informed the receptionist that he/she is ready for the next interview. The receptionist will read a brief recruitment script to the patient (or his or her parent or guardian for selected children) and give him or her a study brochure that will provide patients with information about the study, how long the interview takes, who to contact for more information, and inform him/her of the $25 incentive (or noncash equivalent) they will receive for their participation. If a patient belongs to one of the oversampling groups, the receptionist will always refer the patient to the FI. If the FI is working on an interview, or is unavailable, the receptionist will hand the selected patient a study material and instruct him/her to wait in a designated area. When the FI is available and ready, he/she will look for a patient holding the study material.

Patient Interviewing. When the patients approach the FI, the FI will ask some initial screening question from the Participant Screening Form (Attachment 12) to determine eligibility and population type (i.e., homeless, migrant/seasonal farmworker, public housing, or general community). After the FI explains the study, interested patients will be taken to a private location at the health center and administered the appropriate informed consent procedures and survey.

The FI will use an electronic consent form, which will involve computer-assisted personal interview (CAPI) instructions that inform the FI to hand the respondent a copy of the consent procedures. The FI will read the entire consent form from their laptop and ask the respondent if he/she has any questions. Once questions are answered, the FI will ask for verbal consent from the respondent to participate in the study and document the consent in the CAPI.

The FI will employ one of the following informed consent procedures depending on (1) the age of the respondent and (2) if the respondent is 13 to 17, whether he or she is accompanied by an adult. The respondent’s answers to the questions included on the Participant Screening Form will inform which procedures the FI will follow. The different types of consent/assent forms depending upon the age of the respondent are as follows:

• Self-consent for adult respondents aged 18 and older - The FI will present the participant with a copy of the Adult Survey Participation Consent Form and read it aloud. Afterward, the participant will be invited to ask any questions about the study. Respondents who agree to participate will then be asked to provide verbal consent.

• Parental/guardian consent for child respondents aged 12 and younger (proxy interviews) - The FI will present the participant with a copy of the Parent/Guardian Participation in Proxy Interview for Accompanied Children Consent Form and read it aloud. Afterward, the participant will be invited to ask any questions about the study. Respondents who agree to participate will be asked to provide verbal consent.

• Parental/guardian consent and adolescent assent for respondents aged 13 to 17 who are accompanied by a parent/guardian - The FI will present the parent/guardian with a copy of the Parent/Guardian Permission Form for Adolescent Participation and read it aloud with the adolescent present. Verbal consent will be obtained from parent/guardian. The adolescent will be asked to leave the room so the FI can administer three of the modules in the survey to the parent: health insurance, household income, and a portion of the demographic section. After the parent completes these modules, the FI will ask the parent/guardian to go to a waiting area. The adolescent will then be asked to join the FI in the room. Adolescents will feel more comfortable answering sensitive questions on sexual orientation, gender identity, HPV, mental health, and drug use if the parent is not in the room with them. The FI will obtain adolescent assent using the Minor Participant Assent Form. If the adolescent agrees to participate, he/she will be asked to provide verbal assent.

The order of the interviews (parent first, adolescent second) may be reversed depending on participant availability. The adolescent can be interviewed first but only after parental consent is obtained.

All participants will be provided with a copy of the consent form to take with them after completion of the interview. After the administration of the informed consent, the interview will begin.

If at any time the privacy of the interview setting is compromised, the FI will pause the interview until privacy can be reestablished, rescheduling as necessary. After the interview is completed the respondent will be given a $25 incentive payment (or noncash equivalent) and asked to initial a receipt.

Materials for patient recruitment (informed consent forms and other procedural forms) are included in Attachment 13. All materials that are shown or read to respondents will be available in English, Spanish, Chinese (Mandarin and Cantonese), Vietnamese, and Tagalog.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Response rates for the study will be a function of success in two basic activities: (1) identifying eligible patients and (2) obtaining those patients’ cooperation and time to complete the interview. We will train site staff to use a patient recruiting script, train all FIs on critical cooperation-gaining techniques, regularly debrief with data collection staff, conduct in-person interviews, allow interviewers to schedule interviews at a later time if necessary, make multiple attempts to reach and reschedule respondents who miss appointments, and offer a $25-value remuneration to all participants. Given those plans (which are discussed further below), a 90% cooperation rate is anticipated among referred patients.

Training Site Staff

Site staff will determine whether patients are new or have had services previously, which would deem them potentially eligible for selection. Prior to speaking with potential respondents and using the patient tracking form, site staff will have adequate training to assist in obtaining high levels of patient cooperation. These trainings will last approximately 1.5 hours and will be conducted via telephone with key health center staff and administrative staff at each site immediately before data collection begins. It is anticipated that on average two project staff members per site will attend.

Prior to the training, self-study materials will be distributed for site staff to review. The materials will describe the study and instruct staff on how to create and modify an anonymous roster of patients. In addition, the self-study materials will serve as a reference guide during data collection with mock scenarios demonstrating how to handle various situations.

The training will include study purposes and procedures. Particular attention will be paid to the role of site staff in patient recruitment and as recipients of referrals for mandatory reporting issues. In addition, training will emphasize the appropriate use of project-provided recruitment materials and the importance of privacy. During data collection, field staff will stay in close contact with site staff to ensure that project protocols are followed and assist with any data collection concerns that may arise.

Training Field Interviewing Staff

HRSA will provide a comprehensive multiday training to the field interviewing staff. They will be trained on the study purpose and procedures, interview administration, and the protection of human subjects. Part of the interviewer training will address in detail specific techniques for gaining cooperation and averting and converting a respondent refusal. Reasons for refusals and barriers to participation will be continually evaluated in light of the experience gained during the data collection process.

Regular Debriefings with Data Collection Staff

Project staff will regularly meet with field supervisors and FIs to discuss issues related to data collection operations. Methods to enhance response rates will be a standard agenda item at these meetings.

Onsite Data Collection

When surveying a hard-to-reach population, such as health center patients, we find that interviewer-administered modes yield higher response rates than self-administered modes.¹ Additionally, allowing the option to interview respondents in person at the site right after selection will assist in maximizing cooperation and response rates. Interviews will, therefore, be conducted in-person via CAPI.

If a respondent is interested in participating but unable to complete the interview at the site at that time, the interviewer will have the option of scheduling an appointment at a later time. Future appointments will take place either back at the site, in a conveniently located library with a private room, or at the respondent’s home (if applicable). Multiple attempts to reach respondents who do not appear for scheduled interviews will be a standard protocol.

Offering Remuneration

Providing respondent remuneration during the interviewing phase of data collection increases the likelihood that sample members will participate, particularly for the low-income populations.² Respondents will be provided with remuneration valued at $25 for their participation.

A nonmonetary benefit in the form of data analysis and summarization will be provided to participating awardees with awardee-specific data on patient satisfaction, behavior, and other characteristics in table format. All data analyses will be utilizing de-identified data. Comparisons to results from all participating awardees may also be provided. These data will provide the organization’s management and board with useful information on patient characteristics, barriers to care, and awardee performance. In addition, a $25-value patient recruiter remuneration will be offered to each participating site.

Bilingual Approach

The questionnaire and other respondent materials will be translated into Spanish, Chinese, Tagalog, and Vietnamese. The bilingual approach allows respondents with limited English skills to fully understand their participation. Further, this approach increases the likelihood that those respondents will complete interviews because they will be given the choice of using the language with which they feel more comfortable. A bilingual-certified interviewer will be available for awardees that have a substantial non-English speaking population. Bilingual interviewers will be available to complete interviews in Spanish, Chinese (Mandarin or Cantonese), Tagalog and Vietnamese, or English, depending on respondent preference.

4. Tests of Procedures or Methods to be Undertaken

The procedures and methods to be undertaken will be tested in various ways. Cognitive interviewing will be used to test and finalize the ease of use of the survey instrument. In addition, all procedures and methods are based on those successfully implemented during the previously executed Health Center Patient Surveys (HCPSs). Additional details are supplied below.

Cognitive Interviewing

During the survey questionnaire development phase, two rounds of cognitive interviews will be conducted to finalize a questionnaire that is comprehensible in different languages, can be administered within approximately 60 minutes, and generates accurate data.

Methods from Past Studies

The data collection procedures and materials of this survey were built upon the following surveys:

• Health Care for the Homeless User/Visit Survey (HCH) conducted in 2003

• 1995 Community Health Center User/Visit Survey

• 2002 Community Health Center (2002 CHC)

• National Health Service Corps Site User/Visit Survey

• 2009 Primary Health Care Patient Surveys

• 2014 Health Center Patient Survey.

All of these surveys achieved high response rates and were found to be easily administered correctly by site and field staff.

The HCPS is the only nationally-representative survey of its type that focuses on the health care of populations seeking care at health centers. HRSA annually collects organizational but not patient-level data from HRSA-supported health centers for program evaluation purposes. The information collected through the HCPS informs HRSA on how well health centers provide access to primary and preventative health care from patients’ perspectives. These data are unique and cannot be obtained elsewhere. A majority of the questions in the questionnaire (about 80%) come from the 2014 HCPS. HRSA is adding new topic areas and adapting questions from existing surveys to allow for comparability between questionnaires.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals will be consulted on statistical aspects or involved in data collection and analysis for both the main survey.

Judith VanAlstyne
Contracting Officer’s Representative
Health Resources and Services Administration
(301) 443-6819
[email protected]

Joshua Bolton
Statistician
Health Resources and Services Administration
(301) 443-0547
[email protected]

Kathleen Considine
Project Director
RTI International
(919) 541-6612
[email protected]

Azucena D. Hammill
Assistant Project Director/Data Collection Task Leader
RTI International
(919) 541-7231
[email protected]

Tim Flanigan
Instrumentation Task Leader
RTI International
(919) 485-7743
[email protected]

Patrick Chen
Sampling Task Leader
RTI International
(919) 541-6309
[email protected]

Cynthia Bland
Analysis Task Leader
RTI International
(919) 541-6154
[email protected]

Mai Nguyen
Programming Task Leader
RTI International
(919) 541-8757
[email protected]

1 Dillman, D. A. (2011). Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method (4th ed). New York, NY: John Wiley & Sons.

2 Dodd, T. (1998) “Incentive Payments on Social Surveys: A Summary of Recent Research”. Survey Methodology Bulletin, 43: 23-27

8