2900-0712 Supporting Statement-Part B

SUPPORTING STATEMENT FOR REQUEST FOR MODIFICATION OF -- 2900-0712

NATION-WIDE CUSTOMER SATISFACTION SURVEYS

B. Collections of Information Employing Statistical Methods

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each stratum. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

Veteran patient population 7,000,000

Potential respondent universe: 900,000 patients/year

Expected response rate: Average of 43% (n=387,000)

Sampling methodology: Random selection based on social security numbers and specific visit type at or coordinated through a VA facility

2. Describe the procedures for the collection of information, including:

a. Statistical methodology for stratification and sample selection

b. Estimation procedure

c. Degree of accuracy needed

d. Unusual problems requiring specialized sampling procedures

e. Any use of less frequent than monthly data collection to reduce burden

Inpatient eligibility and sample sizes follow the HCAHPS (Hospital-Consumer Assessment of Healthcare Providers and Systems) Quality Assurance Guidelines (QAG, currently version 8.0) in order to allow VHA to publically report scores on Hospital Compare. Samples are drawn monthly for each hospital in the VHA, with a total monthly sample size of about 15,000 Vveteran patients.

Eligibility and sample size for the Patient Centered Medical Home (PCMH) survey (outpatient) follows the Technical Specifications for Administering the PCMH survey published by the National Committee for Quality Assurance (NCQA) in order to allow VHA to publically report PCMH scores. Patients selected for the current month’s Inpatient sample, and responders to any SHEP survey in the last 12 months are also excluded. Samples are drawn monthly for each site of primary care in the VHA, with a total monthly sample size of about 55,000 veteran patients.

Veteran patients having an encounter with a VHA healthcare provider in the current month are eligible for the Outpatient survey. Patients selected for the current month’s Inpatient or PCMH samples, and responders to any SHEP survey in the last 12 months are also excluded. Samples are drawn monthly for each site of ambulatory care in the VHA, with a total monthly sample size of about 5,000 Vveteran patients.

VHA’s Office of Analytics and Business Intelligence (OABI) is currently obtaining a contract vehicle to administer the other SHEP surveys submitted in the amendment to Justification Supporting Statement-Part A. The surveys include CAHPS Nursing Home, CAHPS In-Center Hemodialysis, CAHPS Home Health, and the Mental Health Veteran Survey. In order to publically report and/or fairly compare VHA data with the private sector, the CAHPS protocols for sample selection and data and data collection will be strictly followed.

Once Vveteran patients are selected, materials are mailed soliciting their response to a variety of questions concerning their care at VA facilities. Veterans are asked to return the surveys to the Office of Quality and Performance contracted data collection vendor.

All survey questionnaires are translated into Puerto Rican Spanish for those Vveterans seeking services at or through the VA Medical Center San Juan, Puerto Rico and its affiliated facilities. Questionnaires are provided in both English and Spanish, allowing the Vveteran to choose the version that they are most comfortable responding with.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Outpatient questionnaires are mailed to veterans via first class mail; a postage-paid return envelope is included. A modified Dillman Total Design Methodology (TDM) is used. The TDM process incorporates several strategies designed to minimize the burdens of survey participation and establish trust. Procedurally the TDM involves five carefully spaced mailings:

Week 1 Pre-notification letter mails

Week 2 First survey with cover letter mails

Week 3 Thank you / reminder postcard mails

Week 5 Close of data collection

The Inpatient sampling and administration protocols will be compliant with CAHPS® HOSPITAL SURVEY (HCAHPS) QUALITY ASSURANCE GUIDELINES V8.0 or later. The mailing sequence shall be a multiple step process consisting of a first survey and cover letter, followed in two weeks by a second survey and cover letter to non responders.

The timing sequence of the mailings in each study is as follows:

Week 1 First survey with cover letter mails

Week 4 Second survey with cover letter

Week 7 Close of data collection

VHA has found the first mailing adequate to achieve very good (45% to 50% range) response rates among veterans. Inpatient questionnaires follow a strict HCAHPS mailing protocol where the patient receives a survey no later than 42 days after discharge, then receives a second survey if the first is not returned within 21 days after the first mailing. This mailing method has shown to provide adequate response rates for the 4,000+ hospitals participating in the HCAHPS survey.

Socio-demographic information for the survey sample is obtained from central files in the VHA databases. Individual identifiers are subsequently stripped from the database to maintain respondent confidentiality.

VHA also wishes to administer web based versions of the surveys in order to bolster response rates from the younger Vveterans, which have a much higher non-response rate. Preliminary contact may be initiated via email, secure text messaging, or other electronic means as a mechanism to increase survey participation. In addition, this mode of administration should significantly reduce cost. Web-based surveys will not replace the mail-out mail back survey mode currently used, but as a tool to bolster diminishing response rates. Every effort will be made to asses and adjust for mode effects attributed to mode of survey administration.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

The National Committee for Quality Assurance (NCQA) field tested a new version of the Consumer Assessment of Healthcare Providers and Services (CAHPS) Clinician & Group Survey specifically designed to evaluate Patient-Centered Medical Homes (C&G-PCMH). The constructs measured tested quite favorably and proved to be both valid and reliable.

A complete psychometric workup of the new C&G-PCMH survey will be conducted to confirm validity and reliability in the VHA population. Since the current inpatient and outpatient SHEP instruments are deeply rooted in the CAHPS family of survey instruments, and since they preformed quite well in our psychometric evaluations, we should expect no different of this new instrument.

In addition, the C&G-PCMH survey offers a web-based data collection version of the survey. The Office of Information & Analytics – Business Intelligence (OABI) will work with the CAHPS Consortium to evaluate this mode of data collection.

Occasionally, OABI would like to examine methods to bolster response rates (e.g. send email to patients in the sample informing them they will receive a survey in the mail), or to explore periodic modular collection of data (questions sent to a portion of the sample regarding Information Technology, Cultural Competence, etc.) in order to better inform the agency about emerging issues.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Jim Schaefer, MPH

Director of Surveys

Office of Information & Analytics – Business Intelligence

919-474-3914

James Flaherty

Administrative Officer, Survey Section

Performance Analysis Center for Excellence

919-474-3917

Page 1