Supporting Statement B_Dental Care Patient Satis Survey (SHEP)_2900-0764_updated Oct 2021

SUPPORTING STATEMENT B

SURVEY OF HEALTHCARE EXPERIENCES OF PATIENTS (SHEP)

Dental CARE Patient Satisfaction Survey
VA Form 10-10070

OMB Control Number 2900-0764

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each strata. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

Potential respondent universe receiving dental care: Over 500,000 patients/year**

Expected response rate: 30-35%

(based upon experience of VA Office of Quality and Performance and prior DPSS years)

Sampling Methodology: A random selection of Social Security numbers of all veterans receiving dental care at VA facilities will receive surveys. This information is derived from the Dental Encounter System which is a subset of the Patient Care Encounter System (PCE).

** from the DES/CDW

2. Describe the procedures for the collection of information, including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed

• Unusual problems requiring specialized sampling procedures

• Any use of less frequent than annual data collection to reduce burden

Procedures for Data Collection:

• Statistical methodology for stratification and sample selection

• The sampling plan calls for dividing the population for Admin Parent into three dental class groupings (IV, I, IIA, IIC), (II), and (III, V, VI). These serve as strata for stratified random selection.  For each month, the sample target is to select up to 34 Veterans per cluster. The goal is to yield 20 to 30 completes per quarter for each stratum.

• Estimation procedure

• The resulting data are weighted to account for the sample design and population sizes.  Each of the survey measures are recoded from its original scales typically into a top two-box variable.  A 0 to 10-scale is recoded into 9 and 10 has a value of 1 and 0 to 8 has a value of 0.  A 4-point Always, Usually, Sometimes and Never scale question is recoded (top two box) into 1 for “Always” or “Usually” and 0 for “Sometimes” or “Never”.  If the variable X are the recoded values of a survey question, the estimate of the proportion falling into the recoded top-box uses , where the weights, , account for the sample design and non-response adjustments, are zero or one recoded values from the survey measure, and the sample proportion is the estimate of the population proportion. Standard errors are estimated using the variances of proportions with an adjustment for the effective sample size, where

• Degree of accuracy needed

• Monthly estimates should achieve a maximum reportable confidence interval ±3% for the nation.

• Unusual problems requiring specialized sampling procedures

• During certain COVID months with extreme sample depletion, dental class sample splits were adjusted from the typical 22/5/7 split to 30/5/7 in order to boost up the sample n for the monthly outgo (and resulting completes), where more Veterans from the primary dental class group 1 were oversampled compared to typical months.

• Any use of less frequent than annual data collection to reduce burden

• Sampled up to 34 Veterans monthly in an attempt to achieve 30+ completes per quarter.

Background Information for Data Collection:

Letters are sent to selected veterans soliciting their response to a variety of questions concerning their care at VA facilities. Veterans are asked to return the survey to the Office of Dentistry contracted data collection vendor. The sample size has been determined to be of a sufficient size to give statistically reliable data for each VISN and VA facility. The monthly data collection provides for sufficiently timely identification of problem areas and undertaking of steps toward remediation. This allows VA to provide quality medical services to veterans.

VHA’s Office of Quality and Performance (OQP) conducted a study of the Dillman Total Design Methodology (TDM) is used (Reference: Dillman, D. A. Mail and telephone surveys: The total design method. New York: Wiley, 1978), considered to be the most effective mailed survey data collection protocol in the industry, and validated Dillman’s methods in the VHA population.

The survey questionnaire is translated into Puerto Rican Spanish for those veterans seeking services at or through the VA Medical Center San Juan, Puerto Rico, and its affiliated facilities. The questionnaire is provided in both English and Spanish, allowing the veteran to choose the version that they are most comfortable responding with.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Survey questionnaire is mailed to veterans via first class mail; a postage-paid return envelope is included. A modified Dillman Total Design Methodology (TDM) is used. The TDM process incorporates several strategies designed to minimize the burdens of survey participation and establish trust. Procedurally the TDM involves three carefully spaced mailings.

The timing sequence of the mailings in each study is as follows:

Week 1 Pre-notification letter mails

Week 2 First survey with cover letter mails

Week 3 Thank you / reminder postcard mails

Week 5 Close of data collection

Sociodemographic information for the survey sample is obtained from central files in the Corporate Data Warehouse (CDW). Individual identifiers are subsequently stripped from the database to maintain respondent confidentiality.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

This survey is comprised of questions taken from two separate well-researched surveys; slight alterations were made to reflect care in the dental setting, e.g., “dentist” in place of “doctor”, or “dental hygienist” in place of “nurse”.  Approximately 27% of the 34 questions were taken from past and present versions of the VA Nation-wide Customer Satisfaction Survey: Survey of Health Experience of Patients (SHEP).  The SHEP survey has OMB approval under clearance number 2900-0712.  Twenty-nine percent of the questions were taken from the DCAHPS survey developed by AHRQ to assess patient satisfaction with dental care provided by TRICARE.  The majority, 44% of the questions, were common to both SHEP and DCAHPS surveys; not surprising as both surveys were developed as part of the AHRQ family of consumer assessment of healthcare surveys.  

Please see Table 1 for a synopsis of the survey question sources.

Source of Survey Questions – UPDATED 4/19/2021

Source of Survey Questions TOTAL 34 Questions	Question #’s
VA SHEP (9/34= 27%)	3,19,20,21,22,23,24,25,26
DCAHPS (10/34=29%)	1,2,5,11,12,13,15,18,27,28
VA SHEP and DCAHPS (15/34=44%)	4, 5,6, 7, 8, 9, 10, 14,16,29,30,31,32,33,34

Table 1. Source of questions for VA Dental Care Patient Satisfaction Survey

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Contractor:

Alan Roshwalb

Senior Vice President and Senior Statistician

US Public Affairs

Ipsos Public Affairs

2020 K Street NW, Suite 410

Washington, DC 20006

Mobile:  703-459-0023

[email protected]

Page 1