Inpatient Sampling

Inpatient survey

methodology brief: sample design

Introduction

VA provides comprehensive health benefits including all the necessary inpatient hospital care to promote, preserve, and restore Veterans’ health. The VA medical center (VAMC) offers a full spectrum of treatment services, such as critical medical care, surgery, mental health care, dialysis, acute care, in addition to other VA specialized treatment services. Upon completion of necessary inpatient care, Veterans may be discharged and return home or transferred to non-VA clinics if they require further medical services.

The Veteran Experience Office (VEO) has been commissioned to measure the satisfaction of Veteran inpatients regarding their interaction with physicians and medical staff as they are hospitalized as well as their experience with discharge arrangements and their transition as they exit the VAMC. The survey also seeks Veteran input on the quality of the care and treatment they received from the VA facilities. VEO proposes to conduct a brief transactional survey on discharged Veterans who utilized the VA inpatient care and medical services within 72 hours. A subset of Veterans will be randomly selected to participate in the survey via an invitation email. A link will be enclosed so the survey may be completed using an online interface.

Table 1. Measurement Goals and Survey Mode

Preferred Mode of Data Collection	Recruitment Method	Time After Transaction	Recruitment Period	Data Collection Frequency
Online Survey	Email Recruitment	Within 72 Hours After Discharge	14 Days (Reminder After 7 Days)	Semi-weekly

Target Population and Sample Size Determination

The target population consists of all Veterans who are discharged from any VAMC inpatient ward within the past week with a length of hospital stay (LOS) of 30 days or less. Long-term hospitalizations are excluded to reduce measurement error. Moreover, the recruitment process omits Veterans admitted for substance abuse or mental health issues, due to the privacy concerns.

Table 2 depicts the approximate monthly inpatient discharge population, email population, number of monthly contacts, as well as the sample size determination with Confidence Level (CL) and Margin of Error (MOE). The sample size listed indicates the number of responses from Veterans required to attain the stated level of precision. The target sample size is determined to be 500 Veteran respondents per week or 2,000 respondents per month. Due to non-response (VEO Veteran surveys generally attain 20% response rate), it is required to initiate contact with about 2,500 (10,000 every month) discharged inpatients per week to attain that target. The sample size is determined to be high enough to obtain at least 15 responses from every VAMC over a month’s time to ensure adequate representation of all VAMC. This sampling strategy prevents excessive repeated contacts, but the number of respondents will still result in survey estimates higher than 5% MOE at the 95% CL.

Table 2B provides the sample targets from the weekly and monthly reporting period, presuming a return rate of 20%.

Table 2A. Target Population Figures

Approximate Monthly Population	Approximate Monthly Contact	Approximate Monthly Email Population	Precision at 5% MOE 95% CL
50,000	10,000	15,000	385

Table 2B. Proposed Sample Targets by Time Period

Weekly Target	Monthly Target
500	2,000

Weighting Adjustments

Many survey practitioners recommend the use of sample weighting to improve inference on the population. Under this process, the respondent sample is artificially made to more closely resemble the true population, with respect to age. It is reported earlier that email population comprises 30% of the overall veteran population and 20% veterans usually respond to the survey, weighting will be used to adjust for non-coverage (for the non-email population) and non-response (occurring when certain subpopulations are less prone to participate). The weighting variable will be the respondent’s Age, Gender, District, and VAMC.

Reporting and Quality Control

The sample sizes reported above adhere to monthly reporting. To ensure the prevention of errors and inconsistencies in the data and the analysis, quality control procedures will be instituted in several steps of the survey process. Records will undergo a cleaning during the population file creation. The quality control steps are as follows.

1. Records will be reviewed for missing sampling and weighting variable data. When records with missing data are discovered, they will be either excluded from the population file or put into separate strata upon discussion with subject matter experts.

2. Any duplicate records will be removed from the population file.

3. Invalid emails will be removed.

The survey sample loading and administration processes will have quality control measures built into them.

1. The survey load process will be rigorously tested prior to the induction of the Inpatient Survey to ensure that customers are not inadvertently dropped or sent multiple emails.

2. The email delivery process is monitored to ensure that bounce-back records will not hold up the email delivery process.

The weighting and data management quality control checks are as follows:

1. The sum of the weighted respondents will be compared to the overall population count to confirm that the records are being properly weighted. When the sum does not match the population count, weighting classes will be collapsed to correct this issue.

2. The unequal weighting effect will be used to identify potential issues in the weighting process. Large unequal weighting effects indicate a problem with the weighting classes, such as a record receiving a large weight to compensate for nonresponse or coverage bias.

Quarantine Rules

VEO seeks to limit contact with Veterans as much as possible, and only as necessary to achieve measurement goals. These rules are enacted to prevent excessive recruitment attempts upon survey participants. VEO also monitors veteran participation within other surveys, to ensure veterans do not experience survey fatigue. All VEO surveys offer options for respondents to opt out, and ensure they are no longer contacted for a specific survey.

Table 3. Proposed Quarantine Protocol

Quarantine Rule	Description	Elapsed Time
Repeated Sampling for Inpatient Survey	Number of days between completing online survey, and receiving another Inpatient survey.	2 Months or 60 Days
Other VEO Surveys	Veterans engaged that have recently completed other VEO surveys will not be selected for 30 days.	1 Month or 30 Days
Anonymous	Callers explicitly wishing to remain anonymous will not be contacted.	N/A
Opt Outs	Persons indicating their wish to opt out of either phone or online survey will no longer be contacted.	N/A

2