Award Fee Provider Satisfaction_OMB Supporting_Statement_Part_B_Revised_4_29 14

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

If the collection of information employs statistical methods, it should be indicated in Item 17 of OMB Form 83-I, and the following information should be provided in this Supporting Statement:

1.  Description of the Activity

Analysis of the results does not require statistical methods other than descriptive statistics. The sample frame for the TAFPS is constructed from a database of all TRICARE network participating providers with at least 1 claim in the past six months. From each CONUS/ OCONUS region, the frame encompasses a simple random sample of unique network providers with claims processing and associated customer support services to claims submitted by and on behalf of MHS beneficiaries who are eligible for both Medicare and TRICARE benefits. The data sources are collected from electronic transactions or claims of any authorized network provider who has submitted a claim for the six month reference period. On a monthly basis, the government provides a list of de-duped randomly selected regional network providers from the TRICARE Encounter Data (TED) records. The Government extracts last name, first name, ID #, and office phone number for each provider.

The sample design is a random sample of all network providers. The random samples are constructed such that there is sufficient amount in each sample to yield 1,224 completed surveys per year.

Survey samples are provided monthly to the survey operations contractor to use to phone network providers. The survey operations vendor makes at least three attempts to reach each potential respondent in order to complete a survey. Subjects are only called during normal business hours. Calls are not made during weekend or evening hours.

At the end of survey fielding, raw survey results are tabulated as components of the award fee determination. The survey is not meant to cover all aspects of contractor performance; rather it is the simplest way to obtain a sample. The random samples are constructed such that there is sufficient amount in each sample to yield 1,224 completed surveys per year.

2.  Procedures for the Collection of Information

For each region, a random sample is drawn with name and contact information. The telephone contact information is then sent to the survey operations contractor for survey fielding operations.

For the TAFPS, the sample frame consists of civilian network providers who have submitted a claim for purchased health care in a CONUS/OCONUS TRICARE region. Monthly, the government provides a list of de-duped provider phone numbers from TRICARE Encounter Data (TED) records to the survey operations contractor. Using telephone surveys, the survey operations contractor monthly contacts network providers who have submitted a claim within the previous 6 months.

The Government assumes the contact information included on claims submissions are business addresses. The survey operations contractor administers the telephone survey. The vendor uses standard telephone survey research methodology in administering the telephone questionnaires to include documentation of interviewer training, valid retrievable call records, and a log of interview sessions. Telephone protocols include a minimum of three call attempts to each case for which a telephone number can be identified. A computerized telephone matching service (if needed) and Directory Assistance are used to track current telephone numbers. To optimize the chances of locating respondents and enlisting cooperation, calls are made at different times of the day, on different days of the week, but calls are made only during normal business hours. Calls are not made during weekend or evening hours.

The sample for this telephone-administered survey is randomly drawn from a list of TRICARE Network providers as described by the TMA and identified by each of the TRICARE MCSCs. “Satisfaction" with provider services is then coded on complete surveys as responses that include a survey response scale of 1-6 where “1” indicates “completely dissatisfied “and “6” indicates “completely satisfied”. “Percent satisfied" is calculated as the simple ratio of responses to all completed survey items for that MCSC.

The results of the survey are aggregated, and are not associated with any individual person. The dialog for telephone survey includes the federal requirements for disclosure. The respondent database is only retained by the Government. Only aggregated information about demographics of the entire database is available to the contracting officer and vendor.

3.  Maximization of Response Rates, Non-response, and Reliability

Response and retention rates are of utmost importance. Much effort has been focused on investigating effects of survey length, frequency and periodicity of conducting the survey on survey response rates and non-response. To optimize the chances of locating respondents and enlisting cooperation, calls are placed at different times of the day, on different days of the week, with day time and evening call attempts occurring midweek and on weekends.

Historically, the average cooperation rate for this survey is 72%, using a cooperation rate calculation as outlined by the American Association for Public Opinion Research. A cooperation rate “indicates the extent to which contacted individuals cooperate with a request to participate in a survey. Generally, the cooperation rate is the ratio of all cases interviewed out of all eligible units ever contacted, whereas a response rate is the ratio of all cases interviewed out of all eligible sample units in the study, not just those contacted." (SAGE Encyclopedia of Survey Research Methods, 2008)

Essentially, our cooperation rate excludes all the phone numbers/people who were NOT able to be reached (bad phone numbers, fax numbers, answering machines, etc.). It includes only people whom the interviewers actually were able to reach. For this reason, cooperation rates will always be higher than response rates, but they can be considered to be helpful in the sense that they measure the population's cooperation with the survey effort, which is often a measurement of interest.

This cooperation rate is calculated using the following formula:

Using the aforementioned formula, the table below displays the cooperation rates for each CONUS and OCONUS region per month. At the bottom of the table, these monthly rates are averaged for each region to determine overall region averages, and these averages are averaged once more to arrive at the overall cooperation rate of 72%.

Cooperation Rates by Region (last six months)

		OCONUS			CONUS
		Europe	Pacific	Latin America	North	South	West
Month 1	Completes	17	17	17	17	17	17
	Incompletes	0	0	0	3	2	0
	Refusals	2	4	0	9	7	6
	Average	89%	81%	100%	69%	73%	74%
Month 2	Completes	17	17	17	17	17	17
	Incompletes	0	0	0	0	1	0
	Refusals	6	8	8	9	15	2
	Average	74%	68%	68%	65%	55%	89%
Month 3	Completes	17	17	17	17	17	17
	Incompletes	0	0	0	1	4	3
	Refusals	10	5	2	8	8	5
	Average	63%	77%	89%	69%	72%	80%
Month 4	Completes	17	17	17	17	17	17
	Incompletes	0	0	0	4	0	1
	Refusals	7	16	8	12	2	8
	Average	71%	52%	68%	64%	89%	69%
Month 5	Completes	17	17	17	17	17	17
	Incompletes	0	0	0	3	4	2
	Refusals	3	8	2	14	11	5
	Average	85%	68%	89%	59%	66%	79%
Month 6	Completes	17	17	17	17	17	17
	Incompletes	0	0	0	2	2	1
	Refusals	14	8	9	12	9	8
	Average	55%	68%	65%	61%	68%	69%
Region Averages		73%	69%	80%	65%	70%	77%
Overall Average		72%

To reach the required 1,224 completed surveys per year, a sample of approximately 11,500 is prepared and provided by the Government. Survey researchers make approximately 9,100 calls to complete 1,224 surveys annually (this number represents the number of phone calls made, not phone numbers used; that is, some phone numbers are called more than once if the phone calls go unanswered).

4.  Tests of Procedures

The data collection instrument was pre-tested prior to the full launch of the survey to refine the instrument and improve the respondents’ ability to provide the requested information

5.  Statistical Consultation and Information Analysis

a. Provide names and telephone number of individual(s) consulted on statistical aspects of the design.

Dr. Paul Rosenfeld

Survey Design, Analysis, & Operations Branch Chief

Human Resources Strategic Assessment Program (HRSAP)

Defense Manpower Data Center (DMDC)

4800 Mark Center Drive, Suite 04E25

Alexandria, VA 22350-4000

[email protected]

Kimberley Marshall, PhD

Senior Health Care Research Analyst

Defense Health Agency

Defense Health Cost Assessment and Program Evaluation (DHCAPE)

[email protected]

Patrick Koeppl, PhD

Deloitte Consulting, LLP

[email protected]

b.  Provide name and organization of person(s) who will actually collect and analyze the collected information.

Patrick Koeppl, PhD

Deloitte Consulting, LLP

[email protected]

5