Supporting Stmt B_NHSC Site Ret Assess Questionnaire_070913

Health Resources and Services Administration

National Health Service Corps
Site Retention Assessment Questionnaire

SUPPORTING STATEMENT B

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Information to identify and contact all NHSC-designated sites was obtained from the BCRS’s BMISS (BCRS Management Information System Solution) data file in late October, 2012. Analyses of these file data revealed that there are 14,005 sites with email addresses for their designated “point of contact (POC),” i.e. the person designated by the site to interact with the NHSC. While multiple points of contact may be listed as individuals sites, only sites with one point of contact were selected to report on, as to minimize response burden. This left a universe of 6,057 sites across all site types and geographic regions with unique points of contact to be surveyed.

This universe of sites will be surveyed without use of sampling. Surveying the entire universe of eligible sites will minimize the time and costs required of the survey process. With an expected participation rate of 20% to 30% after three emailed invitations, surveying the 6,057 site universe will yield between 1,211and 1,817 completed responses. This respondent group size will provide a suitable baseline for the NHSC to be able to begin gathering data on its sites’ knowledge, personnel practices and development needs around clinician retention. The goal of this assessment is to collect baseline data for NHSC staff with an overall sense of retention management and the needs within its sites and not to generate precise point estimates of any specific parameter assessed through the questionnaire. These findings are not intended for reporting to outside agencies nor for publishing.

Analyses will principally be carried out for all respondents as a group. Nevertheless, it will be useful to present some of the knowledge, current practice and site needs outcomes for the most common types of practice sites that are of particular importance to the BCRS. Specifically, subanalyses will be run for Federally Qualified Health Centers, Rural Health Clinics and Indian Health Service/Tribal Clinic/Urban Indian Health Clinic sites. Again, the goal is not to produce point estimates of parameters for each type of site nor to assess for statistically significant differences between types of sites, but rather to provide data separately for each group to help BCRS identify if there may be substantial differences in retention needs across types of sites. The expected respondent group sizes will permit these qualitative comparisons of these subgroups (Table).

Table

	Survey Sample (universe)		20% Participation Rate	30% Participation Rate
	Frequency	Percent	Frequency	Frequency
Correctional Facility	161	2.66	32	48
Federally Qualified Health Center (FQHC) Look-Alike	105	1.73	21	32
Federally Qualified Health Centers (FQHC)	2063	34.08	413	619
Hospital Affiliated Primary Care Outpatient Clinic	567	9.37	113	170
Indian Health Service, Tribal Clinic, and Urban Indian Health Clinic (ITCU)	354	5.85	71	106
Other Health Facility	1488	24.58	298	446
Private Practice (Solo/Group)	805	13.3	161	242
Rural Health Clinic (RHC)	511	8.44	102	153
Total surveyed and expected to respond	6054	100%	1,211	1,817

2. Procedures for the Collection of Information

1. The survey process will be managed by faculty and staff of the Cecil G. Sheps Center for Health Services Research at the University of North Carolina at Chapel Hill. The questionnaire was created on and will be fielded through the Sheps Integrated Research System (“SIRS”), a secure, enterprise database and programming framework specifically designed for the health research projects of the Sheps Center. SIRS provides reusable modules for data collection, management, and tracking for a diverse set of studies and registries. The SIRS framework is completely web based and open source utilizing MySQL, Apache, JavaScript and PHP. Project data are physically located in a Tier II Data Center providing backup power sources, climate control, fire protection, and 24 x 7 surveillance. This same SIRS system and survey approach were used by the same Sheps team and project leader (Dr. Donald Pathman) for a 2011 survey of then-current NHSC clinicians (1,724 respondents, 54% response rate) and for a 2012 survey of current NHSC clinicians (1,726 respondents and again a 54% response rate).

2. The BCRS will send a pre-survey announcement to NHSC sites. The BCRS will also announce the survey in the NHSC’s newsletter or in another regular mailing to NHSC sites.

3. An email invitation will be individually addressed to each site’s designated point of contact. Invitations will arrive addressed from the University of North Carolina at Chapel Hill, and Dr. Donald Pathman will be the signatory. Invitation letters will describe the purpose of the survey, that participation is voluntary, and that all information provided will be seen and analyzed only by investigators at UNC-CH and then shared with the BCRS/NHSC staff only in aggregate, preserving anonymity for participants.

4. Response rates will be regularly monitored and reported by Sheps staff, at least weekly. Two follow-up invitations will be sent at approximately weekly intervals to non-respondents.

5. During the survey process Sheps staff will review survey data provided by respondents to be certain that respondents are completing all items and providing seemingly appropriate responses. Calls from subjects with questions, concerns or problems with the survey will be logged, and issues affecting significant numbers of subjects will be addressed whenever possible.
   
   
   

3. Methods to Maximize Response Rates and Deal with Nonresponse

A variety of accepted approaches will be used to maximize response rates:

1. invitation letters will be individualized, brief, reference that the study is being conducted for the NHSC, promise anonymity, and appeal to subject’s interest in helping the NHSC better serve their practice;

2. invitations will be sent out on Tuesdays or Wednesdays to avoid clinics’ very busy Mondays and still give clinic staff (POC’s) several days to complete questionnaires before the weekend;

3. subjects can immediately and readily access questionnaires through a URL provided on the invitation letter;

4. the questionnaire is relatively brief, does not require subjects to gather information, encourages approximate response figures when needed and provides “I do not know” response options;

5. two follow-up mailings are planned after the initial mailing to stimulate response;

6. survey responses and submitted data will be monitored throughout the survey window to identify and rectify any problems that may be affecting subjects’ willingness or ability to respond.

Ultimately, the survey sample and processes are designed so that a relatively modest response rate will still yield information that will allow the NHSC to understand its sites’ principal retention approaches and needs.

4. Tests of Procedures or Methods to be Undertaken

The survey data system, approaches and staff that will carry out this survey of NHSC sites regularly conduct email and other types of surveys, including two email/on-line surveys of NHSC clinicians in 2011 and 2012, both with high response rates. The system, approaches and staff are well tested.

Survey invitations will initially be sent to 300 sites so that any issues in the survey system can be identified through calls back from subjects to survey staff and by qualitatively reviewing completed survey data. Any issues will be addressed by project staff before invitations are sent several days later to the points of contact of the remaining 5,700 sites.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Given that this survey targets the universe of eligible NHSC sites, will not use sampling, and will involve only descriptive statistics and no inferential statistics in analyses, no statistician is used. Data will be collected and analyzed by the staff and faculty of UNC’s Sheps Center for Health Services Research, led by Dr. Donald Pathman. Dr. Pathman is Professor and Research Director of UNC’s Department of Family Medicine, Director of the Program on Primary Care within the Cecil Sheps Center, and Director of UNC’s NRSA Primary Care Research Fellowship. He has led and participated in numerous health services research projects addressing clinicians and practices in underserved communities over the past 25 years.