2900-0744 Call_Center_Justification_A Public Comment

Department of Veterans Affairs, Veterans Benefits Administration (VBA) Call Center Satisfaction Survey

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The mission of the Veterans Benefits Administration (VBA) National Call Centers is to provide information and assistance to clients (e.g., Servicemembers, Veterans, beneficiaries, etc.) regarding the various programs, services, and benefits offered by VBA.

As part of VBA’s continuing commitment to improve the overall quality of service, VBA will benefit from obtaining direct feedback from its clients regarding their recent call center experience. Specifically, the clients’ feedback will provide VBA three key benefits: 1) identify what is most important to its clients in determining their satisfaction with their call center experience; 2) determine what to do to improve the call center experience and 3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to its clients.

VBA and the contractor will survey clients who have contacted one of seven National Call Centers, the Education Call Center in Muskogee and the National Pension Call Center in Philadelphia. Interviewing began in January 2010 and will continue through FY15.

This sampling plan enables VBA to track and compare call center performance overall, as well as at each site. This will further enable VBA to gauge the effectiveness of its call centers in delivering information and assistance to its clients, as well as assess the clients’ overall level of satisfaction with the call center experience. In addition, the data will be used by VBA to make improvements to the call center operational processes and service delivery, which in turn, will enable VBA to serve its clients in the most efficient and effective way possible

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

Survey data is cascaded throughout VBA via an online reporting tool that is updated on a daily basis with survey returns collected on the previous day, and monthly information sessions. The monthly sessions include a review of call center performance in aggregate, as well as individual scorecards for each center. This transparent reporting model allows Leadership and call centers to make actionable changes that will positively effect call satisfaction. The information collected from the planned surveys during FY10-FY12 enabled VBA to understand, quantify and compare call center satisfaction levels among its clients. The results offered VBA critical inputs to formulate operational changes in the call center environment and ensured that VBA’s clients are effectively served.

Based on the recommendations of the call center research program thus far, VBA has identified 31 service enhancements and have implemented 25 of those service enhancements to effect process improvements within the call centers (i.e., Virtual Hold and Scheduled Call Back technologies were implemented to establish greater convenience for clients, public contact representatives (PCRs) Level II technical assessments, revised PCRs telephone call scripts, launched a SharePoint discussion board for the National Call Centers and training coordinators, established mandatory consistent 2 hour weekly development sessions across NCCs, ECC, and NPCC, hired and trained 100 PCRs to address increased call volume and wait times, Lexus Nexus upgrade to allow for remote proofing for eBenefits, etc.).

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

Information Technology is not feasible at this time. J.D. Power and Associates will conduct CATI (Computer Assisted Telephone Interview) survey among Veterans who have recently contacted VBA’s National Call Centers, the National Pension Call Center, and the National Education Call Center. Calls will be made to respondents no sooner than 24 hours after the respondent speaks with a representative at the call center. The electronic submission of survey responses is not feasible at this time. Respondents will be randomly selected to participate from each of the six sample files provided by VBA throughout each month until a minimum of 55 interviews per call center are completed from each sample file. This will produce a total of 3,000 completes per month for a total of 36,000 interviews completed annually.

Telephone calls will be made between 11 a.m. and 4 p.m. on Saturdays (no telephone calls on Sunday) and between 2 p.m. and 7 p.m. on weekdays in the potential respondent’s local time zone. Telephone calls will not be made on designated holidays. A maximum of seven call attempts will be made to each potential respondent. Interviews will be conducted by live interviewers who will inform potential respondents who they are and why they are calling at the beginning of each call.

In conjunction with the contractor, VBA staff will develop, administer, and analyze the survey data. Also, the contractor in collaboration with VBA, will select the telephone survey provider(s) that will maintain effective security and privacy procedures (e.g., data encryption).

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

VBA is currently not collecting any other call center satisfaction data from clients, so there is no duplication across other efforts conducted within the VBA. Additionally, review of available data suggests that there are no outside sources of data which the VBA could use to obtain a representative sample of the clients feedback on call client satisfaction.

To prevent duplication within the survey (i.e. calling the same client more than once) the VBA and the contractor will de-dupe the call list so that a client is only contacted for their most recent call center experience regardless of calls made to a National Call Center for two or more different reasons. Callers using the same telephone number will not be contacted more than one time in a six month time period.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If VBA is unable to proceed with the collection of the data, it will not have the benefit of continuing to receive client feedback on what is important to them or how best to improve their service and to best serve the needs of our clients. The collection of the data will enable VBA to track and document improvements or declines in call center client satisfaction and service delivery over time.

The design and administration of the Call Center Satisfaction Survey incorporates significant measures to minimize burden on respondents (see section 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.

8. Part A: If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

The 60-day Federal Register (FR) notice was published on September 13, 2012, page 56710. There were two comments received:

Comment 1 from Chairman John Runyan: Improving access to VBA points of contact for private practitioners, decreasing blocked call and incorrect information given by VA and entering information sent to VBA in a correct and timely manner.

Comment 2 from Mr. Paul Sullivan: Adequacy of call center representative training, use of a single, consolidated computer system, and increasing accountability for the call center representatives for accurate answers.

VBA’s response to both comments: We take our responsibilities in serving Veterans and their dependents very seriously. Accordingly, we are developing and implementing process improvements throughout VBA by leveraging feedback from Servicemembers, Veterans, and beneficiaries about their recent contacts with our National Call Centers. We believe that the specific, detailed information obtained through the Call Center Satisfaction Survey will greatly enhance our efforts to improve customer service.

8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances, which preclude consultation every three years with representatives of those from whom information is to be obtained.

VBA’s contractor for this survey is J.D. Power and Associates. VBA is consulting with J.D. Power and Associates regarding their Call Center Certification Program, which benchmarks call

center performance across a number of industries within the private sector. The J.D. Power and Associates Call Center Certification program is an event-driven approach to measuring satisfaction. That is, for each call center that is benchmarked, they survey clients who have interacted with that center. Recency within the J.D. Power and Associates methodology is defined as between 48 hours up to 1-week post call. J.D. Power and Associates chose this time frame after conducting empirical studies aimed at determining which of two methods (immediate or delayed surveying) yields the most valid/reliable results. Those studies indicated that immediate transfers overestimate satisfaction scores and yielded less reliable and less valid results than those obtained from the slight delayed surveying approach.

The ability to leverage J.D. Power and Associates methodology and benchmark data offers VBA a set of well-established and accepted industry performance benchmarks across numerous industries from which to compare and contrast various reporting metrics.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

An assurance of strict confidentiality is made in the introduction respondents receive with the live telephone interviewer. Respondents are assured that answers given will be kept private to the extent of the law and will be used for research purposes only. Respondents will also be given the opportunity to opt-out of completing the survey during the introduction. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument does not contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDDENT GROUP
Population surveyed	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden hours
Veterans who have contacted a call center	36,000	1	.10	3,600
				3,600

TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Estimate hourly wage	Average burden per response (hr.)	Total cost to all respondents
Veterans who have contacted a call center	36,000	16.57	.10	59,652
	36,000			59,652

The Department of Labor estimates the median hourly wage at $16.57 in 2011. ¹

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There is no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent (average of 6 minutes per respondent).

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is estimated at $776,532.18. Table 3 below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to a contractor and will be included in the contractor’s total cost.

TABLE 3: Estimated Cost to the Federal Government
Cost Item	Hours	Cost
VA-Labor	600	26,268
Contractor Costs		750,264.18
TOTAL	235	$776,532.18

The VA Labor cost was estimated using a composite average salary and benefits figure of $43.78 per hour.² The amount paid to the contractor for the survey effort includes as its major components the survey of veterans who have contacted a call center for total cost of $750,264.18.³ These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for administration, administration of the instrument, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reason for any changes reported in Items 13 or 14 above.

There is an increase in total burden overall due to an increase in the survey population size.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Scan Survey and Monitor Response Rates

During the survey field-period, the survey will be scanned as they are received to ensure the currency of the name and telephone number data file. During the scanning process, VBA’s contractor will inspect and remove duplicate surveys in case any individuals inadvertently received a second telephone call after completing a telephone survey.

The contractor will host a password-protected Web site that will provide response rates from the Web-based surveys for the call center client satisfaction surveys. Throughout the period that the survey is being fielded, data will be updated at a minimum on a weekly basis. VBA staff will be provided with passwords to access the site at any time. VBA will review the response rates on a biweekly basis and generate ideas to increase the response rates.

Clean and analyze survey data.

Each month when the interviews have been completed, a raw ASCII data file will be is produced, and the process of creating SPSS data files will begin. A SAS syntax program will be run to convert the ASCII data into separate SPSS data files representing month-to-date, fiscal year-to-date, and quarter-to-date information for each call center and the national data. Each call center’s “raw” SPSS data file will be saved into its own sub-directory, and a copy of the original ASCII data file will be archived separately as a quality control measure. The contractor will analyze the SPSS data files, including conducting frequencies, cross-tabulations, and quadrant analyses. The analyses will be geared toward providing VBA staff with user-oriented results.

• Create monthly sample disposition reports

Monthly disposition reports, which include total calls made, the number and percent of refusals, terminates, number of completes, incidence, response, and cooperation rate reports on an electronic dashboard.

• Create monthly data matrix

The contractor shall provide VBA with monthly data matrices via a secure FTP site that is accessible to appropriate VBA staff and management. Matrices shall provide cumulative summaries of all data month-to-date and year-to-date, on a secure FTP site, hosted by the contractor. The reports shall provide data at the National and call center levels for month-to-date, quarter-to-date, and fiscal year-to-date aggregations.

• Conduct biannual formal briefings

The contractor shall prepare semi-annual overviews of data collected for VBA leadership. This overview will be delivered in-person to VBA leadership and outline data trends and recommendations based on the data collected. The contractor shall provide VBA with 25 color copies of the presentation as handouts for leadership at the meeting.

16B: PROJECT TIMELINE

The major activities for the VBA Call Center satisfaction survey project are structured by task, and are outlined below.

• Task 1: Conduct kickoff meeting and develop the project management plan (PMP)

• Task 2: Contractor shall participate in one face-to-face meeting with VBA staff

to review survey materials, sampling plans, and ongoing study details

• Task 3: Written Call Center Methodology Plan, Survey Samples and Sampling Plan

• Task 4: Create monthly sample disposition reports

• Task 5: Toll-free telephone assistance line, internet-based help desk

• Task 6: Data upload to VOV Reporting Site

• Task 7: Create monthly data matrix

• Task 8: Conduct biannual formal briefings

TIMELINe – call center satisfaction SURVEYS

Field Surveys	Continuous FY13-15
Analyze Data	Continuous FY13-15
Post Data to Reporting Site	Continuous FY13-15
Finalize Reports	Biannually FY13-15

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

We are not seeking such approval, as this will be a telephone survey with no vehicle for respondents to complete without answering via the telephone.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

1 The median hourly wage of all occupations in the United States is estimated by the Department of Labor at $16.57. Source: May 2011 National Occupational Employment and Wage Estimates, United States. Accessed July, 2012, from http://www.bls.gov/oes/current/oes_nat.htm#00-0000.

2 FY 2009 Budget Estimates

3 This total is based on a cost estimate for the proposed contracted work.

- 9 -