Call Center Justification A

Department of Veterans Affairs, Veterans Benefits Administration (VBA) Call Center Satisfaction Survey

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The mission of the Veterans Benefits Administration (VBA) National Call Centers is to provide information and assistance to veterans and active duty personnel regarding the programs, services, and benefits offered by VBA.

As part of the VBA’s continuing commitment to improve the overall service quality, the VBA will benefit from obtaining direct feedback from veterans regarding their recent call center experience. Specifically, veteran feedback will provide the VBA three key benefits: 1) identify what is most important to veterans in determining their satisfaction with their call center experience; 2) determine what to do to improve the call center experience and 3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to veterans and active duty personnel.

VBA and contractor would survey veterans who have contacted one of eight National Call Centers or the National Pension Call Center in Philadelphia. Interviewing will begin January 2010.

This sampling plan enables the VBA to track and compare performance overall as well as at each site. Doing so will enable the VBA to gauge the effectiveness of VBA Call Centers in delivering information and assistance to veterans, as well as assessing veterans’ overall level of satisfaction with the call center experience. In addition, the data will be used by the VBA to make improvements to the call center operational processes and service delivery, which in turn, will enable us to serve veterans and active duty personnel in the most efficient and effective way possible

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

The information collected during the planned FY09 surveys will enable the Veterans Benefits Administration (VBA) to understand, quantify and compare call center satisfaction levels among veteran and active duty personnel. Since currently there are no other means of evaluating satisfaction with the veterans’ experience with VBA Call Centers, these results offer the VBA critical inputs that can be used to formulate operational changes in the call center environment to ensure that veterans and active duty personnel are effectively served.

These data will be used by three primary constituents: 1) Field Directors; 2); Directors of each benefit program (Compensation & Pension, Insurance, Vocational Rehabilitation & Employment; Loan Guaranty; and Education); and 3) VBA Senior Leadership, to make operational improvements.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The proposed survey methodology involves two survey techniques: a live phone interviewer and an IVR-hosted (Interactive Voice Recording - automated) technology. Specifically, a live-phone interviewer will introduce the purpose of the survey and answer any questions veterans have before beginning the survey. Once a veteran agrees to take the survey, they will automatically be connected to the IVR-hosted survey. We are using the combined techniques in order to maximize response rates, reduce the effectives of interviewer bias (i.e. scores for interviewer assisted surveys are notably higher than in non-assisted methods) and effectively manage fielding costs. Furthermore, if a respondent indicates the inability to complete the survey via IVR, the survey will be conducted by a live interviewer.

In conjunction with a contractor, VBA staff will develop, administer, and analyze the survey. Also, the contractor in collaboration with the VBA, will select the phone and IVR provider(s) that have strong capabilities in programming and hosting IVR-surveys, and one that maintains effective security and privacy procedures (e.g., data encryption) when designing and programming the IVR-surveys

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

As noted previously, the VBA is currently not collecting any call center satisfaction data from veterans so there is no duplication across other efforts conducted within the VBA. Additionally, review of available data suggests that there are no outside sources of data which the VBA could use to obtain a representative sample of veterans feedback on call customer satisfaction.

Finally, to prevent duplication within the survey (i.e. calling the same veteran more than once) the VBA and contractor will de-dupe the call list so that a veteran is only contacted for their most recent call center experience regardless of calls made to a National Call Center for two or more different reasons.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If VBA is unable to proceed with the collection of the data, it will not have the benefit of receiving veteran feedback on what is important to them or how best to improve their service and to best serve the needs of veterans. The collection of the data will enable VBA to track and document improvements or declines in call center customer satisfaction and service delivery over time.

The design and administration of the suite of Call Center Satisfaction Surveys incorporates significant measures to minimize burden on respondents (see section 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.

8. Part A: If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

The 60-day Federal Register (FR) notice was published on July 31, 2009, pages 38262-38263. There are no comments received.

8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances, which preclude consultation every three years with representatives of those from whom information is to be obtained.

VBA’s contractor for this survey is J.D. Power and Associates. VBA is consulting with J.D. Power and Associates regarding their Call Center Certification Program, which benchmarks call center performance across a number of industries within the private sectors. The J.D. Power and Associates Call Center Certification program is an event-driven approach to measuring satisfaction. That is, for each call center that is benchmarked, they survey customers who have interacted with that center. Recency within the J.D. Power and Associates methodology is defined as between 48 hours up to 1-week post call. J.D. Power and Associates chose this time frame after conducting empirical studies aimed at determining which of two method (immediate or delayed surveying) yields the most valid/reliable results. Those studies indicated that immediate transfers overestimate satisfaction scores and yielded less reliable and less valid results than those obtained from the slight delayed surveying approach.

The ability to leverage J.D. Power and Associates methodology and benchmark data offers VBA a set of well-established and accepted industry performance benchmarks across numerous industries from which to compare and contrast various reporting metrics.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

An assurance of strict confidentiality is made in the introduction respondents receive with the live phone interviewer prior to being transferred to the IVR-hosted survey. Respondents are assured that answers given will be kept confidential under the Privacy Act and will be used for research purposes only. Respondents will also be given the opportunity to opt-out of completing the IVR-hosted survey during the introduction. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

None of the survey instruments contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDDENT GROUP
Population surveyed	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden hours
Veterans who have contacted a call center	2,700	1	.25	675
Totals	2,700	NA	NA	675

TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Estimate hourly wage	Average burden per response (hr.)	Total cost to all respondents
Veterans who have contacted a call center	2,700	15.57	.25	10,509.75
TOTALS	2,700	N/A	NA	10,509.75

The Department of Labor estimates the median hourly wage of veterans—the targeted respondents for the survey—at $[15.57] in 2008. ¹

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There is no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent (average of 15 minutes per respondent).

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is estimated at $85,830.00. Table 3 below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to a contractor and will be included in the contractor’s total cost.

TABLE 3: Estimated Cost to the Federal Government
Cost Item	Hours	Cost
VA-Labor	150	6,567
Contractor Costs		79,263
TOTAL	235	85,830.00

The VA Labor cost was estimated using a composite average salary and benefits figure of $43.78 per hour.² The amount paid to the contractor for the survey effort includes as its major components the survey of veterans who have contacted a call center for total cost of $79,263.³ These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for IVR administration, questionnaire pretest, administration of the instrument, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reason for any changes reported in Items 13 or 14 above.

This is a new information collection.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Scan Surveys and Monitor Response Rates

During the survey field-period, the surveys will be scanned as they are received to ensure the currency of the name and phone number data file. During the scanning process, VBA’s contractor will inspect and remove duplicate surveys in case any individuals inadvertently received a second phone call after completing an IVR-hosted survey.

The contractor will host a password-protected Web site that will provide response rates from the Web-based surveys for the call center customer satisfaction surveys. Throughout the period that the surveys are being fielded, data will be updated at a minimum on a weekly basis. VBA staff will be provided with passwords to access the site at any time. VBA will review the response rates on a biweekly basis and generate ideas to increase the response rates.

• Develop Prototype Reports

Prototype reports will be developed for each of the two surveys. VBA will determine the appropriate weighting methods (if any) for the national reports. The selection of analyses and report exhibits in the prototype reports will rely on the knowledge and experience of VBA staff.

Each prototype report will be developed with the end user in mind, providing a clear explanation of findings. Each report will consist of the following sections:

• Executive summary

• Project methodology

• Survey results

• Appendices (e.g., copies of questionnaires and interviewing materials).

The contractor will pretest automation routines for new designs in the prototype reports to ensure that report exhibits, graphics, and data tables can be reliably and accurately programmed for each report type. Development of the automated report system will consist of several concurrent tasks, including a data crosswalk of the survey iterations to map identical variables and response sets, SPSS syntax routines, and development of prototype report templates using Microsoft Excel with Visual Basic for Applications.

• Revise Prototype Reports

Based on discussion between the VBA staff and the contractor, the contractor will incorporate suggestions and changes/edits to the reports’ content and layout and will produce a revised report for the survey.

• Submit “Live” Draft Reports

The production of the draft report requires “quick” turnaround of data cleaning and automated report production. This section details the steps to be taken to produce draft reports for the survey using actual data from the survey administration. There are two significant subtasks involved in producing the draft report:

• Subtask 1: Clean and analyze survey data

• Subtask 2: Develop draft report.

Each of these subtasks is described below.

Subtask 1—Clean and analyze survey data. Shortly after the interviewing process commences, the contractor will examine the respondent data file to ensure that the IVR-hosted survey is functioning properly and that all of the variables are correctly placed and formatted. The sample data file will also be used to test the data cleaning programs. This process will identify any errors in the IVR-hosted survey process.

. Once the final interviews have been completed and a raw ASCII data file has been produced, the process of creating SPSS data files will begin. An SPSS syntax program will be run to convert the ASCII data into separate SPSS data files representing each call center and the national file. Each call center’s “raw” SPSS data file will be saved into its own sub-directory, and a copy of the original ASCII data file will be archived separately as a quality control measure. The contractor will analyze the SPSS data files, including conducting frequencies, cross-tabulations, and quadrant analyses. The analyses will be geared toward providing VBA staff with user-oriented results.

Subtask 2—Develop draft reports. A draft report will be produced based on data from the national file. Using the automated report generation system, the data will be analyzed and key findings will be identified and summarized in the executive summary. Production of a draft report provides VBA staff an opportunity to assess the accuracy, quality, and utility of the report structure and content prior to production of final reports.

• Conduct Informal Briefing

The contractor will develop data tabulations on survey highlights from the customer satisfaction surveys and present the results informally to VBA staff. The contractor will work closely with the VBA to determine the most salient findings to present in the briefing. Following this informal presentation, VBA staff will provide the contractor with feedback for revising the briefing content. No more than two separate briefings will be required.

• Conduct Formal Briefing

The contractor will incorporate feedback, suggestions, and edits from VBA staff on the informal briefing to develop the final version of the briefing. Following this, the contractor will provide one face-to-face, formal, professionally conducted briefing on the results of the surveys to senior VBA management.

• Submit Final Reports

The contractor will submit all reports in final form, incorporating suggestions from VBA staff where agreed. The contractor will deliver bound color copies of the final survey reports to each of the eight call centers. The package will also contain a CD, which will include only the final versions of the reports. The contractor will provide official receipts to the VBA for the mailing/shipping for each report package.

• Deliver CD of Final Products

The contractor will submit the final deliverable of CDs containing electronic files as specified in the SOW. Documents and materials will include:

• All background statistical analysis performed

• Raw survey data on CD in SPSS format (including all original information provided with the sample) for the call centers surveyed

• Copies of the reports in electronic format on CD

• A detailed response rate file

These CDs of electronic files will provide VBA with all materials used during the project, along with the final reports.

16B: PROJECT TIMELINE

The major activities for the VBA Call Center satisfaction survey project are structured by task, and are outlined below.

• Task 1: Conduct kickoff meeting and develop the project management plan (PMP)

• Task 2: Review survey materials and discuss enhancements to the survey instrument and methodology

• Task 3: Revise survey instrument

• Task 4: Revise and finalize survey instrument and interview materials

• Task 5: Draw sample containing all customer interactions for 48 hours going back from the survey start date

• Task 6: Conduct surveys

• Task 7: Scan surveys and monitor response rates

• Task 8: Develop prototype reports

• Task 9: Revise prototype reports

• Task 10: Submit “live” draft reports

• Task 11: Conduct informal briefing on survey results

• Task 12: Conduct formal briefing

• Task 13: Submit final reports

• Task 14: Deliver CD of final products.

TIMELINe – call center satisfaction SURVEYS

Field IVR Surveys	January 2010
Analyze Data	February 2010
Draft Report	March 2010
Finalize Report	April 2010

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

We are not seeking such approval, as this will be a phone survey with no vehicle for respondents to complete without answering via the telephone.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

1 The median hourly wage of veterans (i.e., general population) in 2008 is estimated by the Department of Labor at $15.57. Source: Occupational and Employment Wages, 2008. Accessed May 1, 2009, from http://www.bls.gov/news.release/pdf/ocwage.pdf

2 FY 2009 Budget Estimates

3 This total is based on a cost estimate for the proposed contracted work.