1 BVA_Call_Center_Justification A

Supporting Statement A

For

Board of Veterans’ Appeals

Veterans Information Office

Voice of the Veteran Call Center Survey

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The mission of the Board of Veterans’ Appeals (Board) is to hold hearings and issue timely, understandable, quality decisions on appellate matters.

Currently, the Board collects customer satisfaction on a very limited basis. Survey cards are distributed to the appellant if a hearing is conducted and the Board relies on respondents to mail in the postcard. The survey card only measures the appellant’s satisfaction with the hearing process and response rates are low. The Board will benefit from obtaining direct feedback from its Veterans and appellants regarding their recent Veterans Information Office (VIO) Call Center experience. Specifically, the Veterans’ feedback will provide the Board three key benefits: 1) identify what is most important to its Veterans and appellants in determining their satisfaction with their VIO Call Center experience; 2) determine what to do to improve the call center experience; and 3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to its Veterans.

The Board and the Contractor will survey only Veterans and appellants who have contacted the Board’s VIO Call Center. This survey will enable the Board to gauge the effectiveness of its VIO Call Center in delivering information and assistance to its Veterans, as well as assess the Veterans’ overall level of satisfaction with the VIO Call Center experience. In addition, the data will be used by the Board to make improvements to the VIO Call Center operational processes and service delivery, which in turn, will enable the Board to serve its Veterans in the most efficient and effective way possible.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

The information collected during the planned FY13 surveys will enable the Board to understand and quantify call center satisfaction levels of Veterans. Since currently there are no other means of evaluating satisfaction with the Veterans’ experience with the Board’s VIO Call Center, these results offer the Board critical inputs that can be used to formulate operational changes in the call center environment to ensure that appellants are effectively served.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

J.D. Power and Associates will conduct CATI (Computer Assisted Telephone Interview) surveys among Veterans who have recently contacted the Board’s VIO Call Center. Calls will be made to respondents no sooner than one week after the respondent speaks with a representative at the VIO Call Center. The electronic submission of survey responses is not feasible at this time. Respondents will be randomly selected to participate from sample files provided by the Board throughout each quarter until 1250 interviews are completed (417 per month). This will produce a total of 5000 interviews completed annually.

Telephone calls will be made between 11 a.m. and 4 p.m. on Saturdays (no telephone calls on Sunday) and between 2 p.m. and 7 p.m. on weekdays in the potential respondent’s local time zone. Telephone calls will not be made on designated holidays. A maximum of seven call attempts will be made to each potential respondent. Interviews will be conducted by live interviewers who will inform potential respondents who they are and why they are calling at the beginning of each call.

In conjunction with the Contractor, Board staff will develop, administer, and analyze the survey data. Also, the Contractor in collaboration with BVA will select the telephone survey provider(s) that will maintain effective security and privacy procedures (e.g., data encryption).

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The Board is currently not collecting any other call center satisfaction data from Veterans, so there is no duplication across other efforts conducted within the Board. Additionally, review of available data suggests that there are no outside sources of data which the Board could use to obtain a representative sample of the Veterans feedback on call client satisfaction.

To prevent duplication within the survey (i.e., calling the same client more than once) the Board and the Contractor will de-dupe the call list so that a Veteran or appellant is only contacted for their most recent VIO Call Center experience. Callers using the same telephone number will not be contacted more than one time in a six-month time period.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If the Board is unable to proceed with the collection of the data, it will not have the benefit of continuing to receive client feedback on what is important to them or how best to improve their service and to best serve the needs of our Nation’s Veterans. The collection of the data will enable the Board to track and document improvements or declines in VIO Call Center customer satisfaction and service delivery over time.

The design and administration of the call center satisfaction survey incorporates significant measures to minimize burden on respondents (see section 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.

8. Part A: If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

In compliance with 5 C.F.R. 1320.8(d), VA solicited comments from members of the public and affected agencies concerning the proposed new information collection “Board of Veterans’ Appeals, Veterans Information Office, Voice of the Veteran Call Center Survey.” See 78 Fed. Reg. 54957-58 (September 6, 2013).

Several errors appeared in the September 6, 2013, Federal Register Notice in paragraphs pertaining to “burden hours”. The errors occurred due to non-use of converting hours into minutes. The “Frequency of Response” was changed from “Quarterly” to “Annually”. However, these errors only appeared in the Federal Register Notice and were inconsistent with that that was reported in the Supporting Statement. A corrected Federal Register Notice was published to reflect these corrections. See 78 Fed. Reg. 59426 (September 26, 2013).

VA received one comment from the public. The comment did not specifically address whether the proposed collection of information is necessary for the performance of VA’s functions; the accuracy of VA’s estimates of the burden of the proposed collection of information; the quality, utility, and/or clarity of the information to be collected; or ways to minimize the burden of the information collection on those who are to respond. Therefore, no further action will be taken with regard to the proposed information collection request for the “Board of Veterans’ Appeals, Veterans Information Office, Voice of the Veteran Call Center Survey in response to the comment received.

8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances, which preclude consultation every three years with representatives of those from whom information is to be obtained.

The Board’s contractor for this survey is J.D. Power and Associates. The Board is consulting with J.D. Power and Associates regarding their Call Center Certification Program, which benchmarks call center performance across a number of industries within the private sector. The J.D. Power and Associates Call Center Certification program is an event-driven approach to measuring satisfaction. That is, for each call center that is benchmarked, the Contractor surveys only Veterans who have interacted with that center. Recency within the J.D. Power and Associates methodology is defined as between 48 hours up to 1-week post call. J.D. Power and Associates chose this time frame after conducting empirical studies aimed at determining which of the two methods (immediate or delayed surveying) yields the most valid/reliable results. Those studies indicated that immediate transfers overestimate satisfaction scores and yielded less reliable and less valid results than those obtained from the slight delayed surveying approach.

The ability to leverage J.D. Power and Associates methodology and benchmark data offers the Board a set of well-established and accepted industry performance benchmarks across numerous industries from which to compare and contrast various reporting metrics.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

An assurance of strict confidentiality is made in the introduction respondents receive with the live telephone interviewer. Respondents are assured that answers given will be kept private to the intent of the law and will be used for research purposes only. Respondents will also be given the opportunity to opt-out of completing the survey during the introduction. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code). The Privacy notification is included in a list of

frequently asked questions that respondents have. During the interview, J.D. Power responds as follows:

Question: Will my responses during this interview remain confidential?

Answer: Yes, your responses during this interview will remain completely confidential and will not affect your eligibility for current or future benefits. The answers you provide will not be linked to your name or contact information

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument does not contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden hours
Veterans who have contacted the Board VIO Call Center	5,000	1	.10	500
Total				500

TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Estimate hourly wage	Average burden per response (hr.)	Total cost to all respondents
Veterans who have contacted the Board VIO Call Center	5000	24	.10	12,000
Total	5000			12,000

According to the U.S. Bureau of Labor Statistics Average Hourly Earnings, the cost to the respondent is $2.40.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There is no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent (average of 6 minutes per respondent).

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is estimated at $124,517. Table 3 below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to Contractor and will be included in the Contractor’s total cost.

TABLE 3: Estimated Cost to the Federal Government
Cost Item	Hours	Cost
VA labor	360	$15,761
Contractor costs		$108,755.74
TOTAL	360	$124,517

The VA labor cost was estimated using a composite average salary and benefits figure of $43.78 per hour.¹ The amount paid to the Contractor for the survey effort includes as its major components the survey of Veterans who have contacted the VIO Call Center for total cost of $108,755.74.² These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for administration, administration of the instrument, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reason for any changes reported in Items 13 or 14 above.

There are no changes in Items 13 or 14.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Scan Survey and Monitor Response Rates

During the survey field-period, the survey will be scanned as they are received to ensure the accuracy of the name and telephone number data file. During the scanning process, the Board’s Contractor will inspect and remove duplicate surveys in case any individuals inadvertently received a second telephone call after completing a telephone survey.

The Contractor will host a password-protected FTP site that will provide response rates from the phone-based surveys for the Call Center customer satisfaction surveys. Throughout the period that the survey is being fielded, response data will be updated on a quarterly basis. The Board’s staff will be provided with passwords to access the site at any time. The Board will review the response rates on a quarterly basis and generate ideas to increase the response rates.

• Clean and Analyze Survey Data

Each quarter when the interviews have been completed, a raw ASCII data file will be produced, and the process of creating SPSS data files will begin. A SAS syntax program will be run to convert the ASCII data into separate SPSS data files representing quarter-to-date information for Call Center. VIO Call Center’s “raw” SPSS data file will be saved into its own directory, and a copy of the original ASCII data file will be archived separately as a quality control measure. The Contractor will analyze the SPSS data files, including conducting frequencies, cross-tabulations, and quadrant analyses. The analyses will be geared toward providing the Board’s staff with user-oriented results.

• Create Quarterly Sample Disposition Reports

Quarterly disposition reports, which include total calls made, the number and percent of refusals, terminates, number of completes, incidence, response, and cooperation rate reports on an electronic dashboard.

• Create Monthly Data Matrix

The Contractor shall provide the Board with monthly data matrices via a secure FTP site that is accessible to appropriate Board staff and management. Matrices shall provide cumulative summaries of all data month-to-date and year-to-date, on a secure FTP site, hosted by the contractor.

16B: PROJECT TIMELINE

The major activities for the Board Call Center Satisfaction Survey project are structured by task, and are outlined below.

• Task 1: Conduct kickoff meeting and develop the Project Management Plan (PMP)

• Task 2: A face-to-face meeting with Board staff to review survey materials, sampling plans, and ongoing study details

• Task 3: Written Call Center Methodology Plan, Survey Samples and Sampling Plan

• Task 4: Design survey instrument for VIO Call Center

• Task 5: Complete survey interviews

• Task 7: Quarterly disposition reports posted to Secure Site

• Task 8: Quarterly analysis reports posted to Secure Site

• Task 8.9: Monthly written progress reports

TIMELINe – call center satisfaction SURVEYS

Field Surveys	Continuous FY13-15
Analyze Data	Continuous FY13-15
Post Data to Reporting Site	Continuous FY13-15
Finalize Reports	Biannually FY13-15

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

We are not seeking such approval, as this will be a telephone survey with no vehicle for respondents to complete without answering via the telephone.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

1 FY 2009 Budget Estimates

2 This total is based on a cost estimate for the proposed contracted work.

- 9 -