OMB# 2900-0744
VBA National Call Center (NCC) Satisfaction Survey |
VBA Education Call Center (ECC) Satisfaction Survey |
VBA National Pension Call Center (NPCC) Satisfaction Survey |
JUSTIFICATION:
Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection of information.
The mission of the Veterans Benefits Administration (VBA) National Call Centers is to provide information and assistance to clients (e.g., Servicemembers, Veterans, beneficiaries, etc.) regarding the various programs, services, and benefits offered by VBA.
As part of VBA’s continuing commitment to improve the overall quality of service, VBA will benefit from obtaining direct feedback from its clients regarding their recent call center experience. Specifically, the clients’ feedback will provide VBA three key benefits: 1) identify what is most important to its clients in determining their satisfaction with their call center experience; 2) determine what to do to improve the call center experience and 3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to its clients.
VBA and the Contractor 1will survey clients who have contacted one of eight National Call Centers, the Education Call Center in Muskogee and the National Pension Call Center in Philadelphia. Interviewing began in January 2010 and is ongoing.
This plan enables VBA to track and compare call center performance overall, as well as at each call center. This will further enable VBA to gauge the effectiveness of its call centers in delivering information and assistance to its clients, as well as assess the clients’ overall level of satisfaction with the call center experience. In addition, the data will be used by VBA to make improvements to the call center operational processes and service delivery which, in turn, will enable VBA to serve its clients in the most efficient and effective way possible. This information collection supports VA’s commitment to improving the Veteran experience by obtaining direct feedback from those who interact with our contact centers.
Indicate how, by whom, and for what purposes the information is to be used; indicate actual use the agency has made of the information received from current collection.
VBA National Call Center (NCC) Satisfaction Survey, Education Call Center (ECC) Satisfaction Survey, and National Pension Call Center (NPCC) Satisfaction Survey are used to conduct computer assisted telephone interviews (CATI) for VBA Call centers. The information collection is optional. VBA utilizes a survey vendor (contractor) to complete the CATI surveys.
The data is tabulated by the Contractor and is distributed to VBA national, district andregional leadership. The data is also provided to the leadership and staff of each call center. Data is housed via an online reporting tool or internal database that are updated on a daily basis with survey responses collected from the previous day. Monthly sessions are held to review call center performance in aggregate. Each month, select call centers receive a station specific deep dive into call center customer satisfaction for each center. The frequency and accessibility of reporting allows VBA Leadership and call centers to implement actionable changes that will positively affect caller satisfaction. The surveys are designed to obtain customer feedback regarding contact center processes, availability and effectiveness of contact options (i.e., in-person, phone, or web-based), training, and technology within VBA contact centers.
The information collected from the surveys have enabled VBA to understand, quantify and compare call center satisfaction levels among its stakeholders. The results offered VBA critical inputs to formulate operational changes in the call center environment and ensured that VBA’s clients are effectively served. Based on the recommendations of the call center research program thus far, VBA has identified over 150 service enhancements and completed over 180,000 interviews since the inception of the program. Service enhancements, such as Computer Telephony Integration, Workforce Management, contact center quality management program redesign, voice-activated interactive voice response (IVR) system redesign and implementation of segemented training to improve the timeliness of new hire onboarding were implemented to improve the Veteran experience with the phone system and engagement with employees. Process improvements such as direct upload of documents into eFolder for contact center employees and interactive and collaboration sites’ leadership teams and and helped to ensure a knowledgeable, courteous, and efficient call center workforce. In addition, based upon the identification of ease of access as a key driver in customer satisfaction, VBA increased the staffing level for PCRs and support personnel to further improve access and reduce wait time in the NCCs.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.
Information Technology is not feasible at this time. There currently is no utility process in place that will allow the data submitted on the form to be fully incorporated with an existing centralized legacy database.The contractor will conduct CATI (Computer Assisted Telephone Interview) surveys among Veterans who have recently contacted VBA’s National Call Centers, the National Pension Call Center, and the National Education Call Center. The current methodology utilizes live intervierwers to contact respondents.
Calls will be made to respondents no sooner than 24-48 hours after the respondent speaks with a representative at the call center. Telephone calls will be made between 11 a.m. and 4 p.m. on Saturdays (no telephone calls on Sunday) and between 2 p.m. and 7 p.m. on weekdays in the potential respondent’s local time zone. Telephone calls will not be made on federal holidays. A maximum of six call attempts will be made to each potential respondent. Interviews will be conducted by live interviewers who will inform potential respondents who they are and why they are calling at the beginning of each call.
Respondents will be randomly selected to participate from VBA data files throughout each month until a minimum of 300 interviews per call center are completed for each month. This will produce a total of 3,000 completes per month for a total of 36,000 interviews completed annually.
In accordance with Memorandum for the Heads of Executive Departments and Agencies and Independent Regulatory Ageencies dated July 22, 2016, it is noted in the Office of Information and Regulatory Affairs, Flexibilities under the Paperwork Reduction Act for Compliance with Information Collection Requirements, that “Non-substantive changes may also be used to facilitate and finalize larger changes to a particular collection, as long as the public is provided with some opportunity to comment on possible options or changes, as well as the circumstances that will trigger those options, as part of theofthetheofthe original approval.”
As technology enhancements are made within VBA call centers, the ability to supplement this effort with automated surveys may be possible. Automation may include live transfer to survey interviewers and post-call interactive voice response (IVR) options. In the event such technology is implemented, a new survey questionnaire(s) will be developed to support these automated transactional surveys. The questions will be of a similar nature to those currently utilized within the information collection. These changes, if implemented, would be submitted as a non-substantive administrative change. After OMB approval is obtained, previously approved survey versions may be utilized until the new survey instruments have been programmed for interview.
In conjunction with the contractor ,VBA staff will develop, administer, and analyze the survey data. Sample files will omit social security numbers and be transmitted utilizing data encryption and VBA approved file transfer policies
Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.
VBA is currently not collecting any other call center satisfaction data from clients, so there is no duplication across other efforts conducted within the VBA. Additionally, review of available data suggests that there are no outside sources of data which VBA could use to obtain a representative sample of the clients feedback on contact center client satisfaction.
To prevent duplication within the survey (i.e. calling the same client more than once) VBA and the contractor will de-dupe the call list so that a client is only contacted for their most recent contact center experience. Callers using the same telephone number will not be contacted more than one time in a six month time period.
If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
The collection of information does not involve small businesses or entities.
Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.
If VBA is unable to proceed with the collection of the data, it will not have the benefit of continuing to receive client feedback on what is important to them, how best to improve their service, and how to best serve the needs of our clients. The collection of the data will enable VBA to track and document improvements or declines in call center client satisfaction and service delivery over time.
The design and administration of the Call Center Satisfaction Survey incorporates significant measures to minimize burden on respondents (see section 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.
Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.
There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.
There is no special circumstance requiring collection in a manner inconsistent with 5 CFR 1320.6 guidelines.
A. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.
The Department notice was published in the Federal Register on April 17, 2018, Volume 83, No. 74, page 16922. No comments were received in response to this notice.
Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances, which preclude consultation every three years with representatives of those from whom information is to be obtained.
VBA utilizes a third party contractor for the collection and tabulation of survey data. VBA originally consulting with J.D. Power regarding their Call Center Certification Program, which benchmarks call center performance across a number of industries within the private sector. The J.D. Power Call Center Certification program is an event-driven approach to measuring satisfaction. That is, for each call center that is benchmarked, they survey clients who have interacted with that center. Recency within the J.D. Power methodology is defined as one to five day dialing period post call to the call center. J.D. Power chose this time frame after conducting empirical studies aimed at determining which of two methods (immediate or delayed surveying) yields the most valid/reliable results. Those studies indicated that immediate transfers overestimate satisfaction scores and yielded less reliable and less valid results than those obtained from the slightlyslightly delayed surveying approach.
The ability to leverage J.D. Power methodology and benchmark data offers VBA a set of well-established and accepted industry performance benchmarks across numerous industries from which to compare and contrast various reporting metrics. In the event of contractor changes, VBA would consult with those parties to confirm their approach and the statistical validity of the results.
No payments or gifts to respondents have been made under this collection of information.
The information that respondents supply is protected by the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code. Additionally VA privacy policy is also in effect. Respondents are assured that answers given will be kept private to the extent of the law and will be used for research purposes only. Respondents are also given the opportunity to “opt-out” of completing the survey. At their option, respondents may also have their responses reported with their identifying information for VA review. The information gathered from the survey does not affect benefits or services provided to the respondent.
There are no questions of a sensitive nature.
TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDENT GROUP |
||||
Form |
Number
of |
Frequency of Response |
Average burden per response (in hours) |
Total burden hours |
VBA National Call Center (NCC) Satisfaction Survey |
28,800 |
1 |
0.1 |
2,880 |
VBA Education Call Center (ECC) Satisfaction Survey |
3,600 |
1 |
0.1 |
360 |
VBA National Pension Call Center (NPCC) Satisfaction Survey |
3,600 |
1 |
0.1 |
360 |
Totals: |
36,000 respondents |
|
|
3,600 hours |
TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP |
||||
Form |
Number
of |
Average |
Total Annual Burden Hours |
Total
Cost |
VBA National Call Center (NCC) Satisfaction Survey |
28,800 |
$ 23.86 |
2880 |
$ 68,716.80 |
VBA Education Call Center (ECC) Satisfaction Survey |
3,600 |
$ 23.86 |
360 |
$ 8,589.60 |
VBA National Pension Call Center (NPCC) Satisfaction Survey |
3,600 |
$ 23.86 |
360 |
$ 8,589.60 |
Totals: |
3600 respondents |
|
360 hours |
$ 8,589.60 |
T
The respondent population for VBA Forms National Call Center (NCC) Satisfaction Survey, Education Call Center (ECC) Satisfaction Survey, and National Pension Call Center (NPCC) Satisfaction Survey are composed of individuals who contact VBA contact centers by phone. VA cannot make further assumptions about the population of respondents because of the variability of factors such as the educational background and wage potential of respondents. Therefore, VBA used general wage data to estimate the respondents’ costs associated with completing the information collection.
The Bureau of Labor Statistics (BLS) gathers information on full-time wage and salary workers. According to the latest available BLS data, the median weekly earnings of full-time wage and salary workers are $954.40. Assuming a forty (40) hour work week, the mean hourly wage is $23.86 based on the BLS wage code – “00-0000 All Occupations.” This information was taken from the following website: (https://www.bls.gov/oes/current/oes_nat.htm, May 2016).
Legally, respondents may not pay a person or business for assistance in completing the information collection. Therefore, there are no expected overhead costs for completing the information collection. VBA estimates the total cost to all respondents to be $8,589.60 (360 burden hours x $23.86 per hour).
This submission does not involve any recordkeeping costs.
Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.
Estimated Costs to the Federal Government:
Grade |
Step |
Number of Employees |
Annual Hourly Time Burden per employee |
Annual Time Burden in Hours |
Hourly Rate |
Total Responses |
Total |
12 |
3 |
20 |
10 |
200 |
$32.51 |
36,000 |
$6,502.00 |
13 |
3 |
15 |
50 |
750 |
$38.65 |
36,000 |
$28,987.50 |
14 |
3 |
10 |
30 |
300 |
$45.68 |
36,000 |
$13,704.00 |
15 |
3 |
5 |
30 |
150 |
$53.73 |
36,000 |
$8,059.50 |
|
|
|
|
1400 |
|
|
|
Contract Cost |
$860,501.49 |
||||||
Processing / Analyzing Costs |
$57,253.00 |
||||||
Printing and Production Cost |
$5,000.00 |
||||||
Total Cost to Government |
$922,754.49 |
||||||
Grade |
Step |
Total Responses |
Cost Per Response |
12 |
3 |
36,000 |
$0.18 |
13 |
3 |
36,000 |
$0.81 |
14 |
3 |
36,000 |
$0.38 |
15 |
3 |
36,000 |
$0.22 |
|
|
|
$1.59 |
Additional overhead costs do not apply as Government is only involved with the analysis of results provided by the contractor. The increase in employee labor since the previous information request represents the increase use of the survey data for process improvement with Call Center operations.
Note: The hourly wage information above is based on the hourly 2018 General Schedule (Base) Pay (https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2018/GS_h.pdf). This rate does not include any locality adjustment as applicable.
The processing time estimates above are based on the actual amount of time employees of each grade level spend to review the results and deliverables produced under the contract, to distribute those results throught VBA, and to implement process improvement. The within-grade step (3) of each employee represents the average experience of employees within each grade.
Explain the reason for any burden hour changes since the last submission.
The total burden hours for this collection remain the same. However, VA has redistributed burden hours among instrument collections. Forty (40) burden hours were removed from the both the VBA Education Call Center (ECC) Satisfaction Survey and the VBA Pension National Call Center (NPCC) Satisfaction Survey for a total of eighty (80) burden hours. Those eighty (80) burden hours were reallocated to the VBA National Call Center (NCC) Satisfaction Survey.
For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.
Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.
16A: TABULATION AND PUBLICATION
Monitor Response Rates
The contractor will monitor fielding daily to ensure that the monthly interviewing quota are met for each call center. If any issues arise that may prevent quota fulfillment, the contractor will notify VBA and arrange for a make-up sample collection day to receive additional sample.
Clean and analyze survey data.
Each month when the interviews have been completed, a raw ASCII data file will be produced, and the process of creating SPSS/SAS data files will begin. A SAS syntax program will be run to convert the ASCII data into separate SPSS/SAS data files representing month-to-date, fiscal year-to-date, and quarter-to-date information for each call center and the national data. Each call center’s “raw” SPSS data file will be saved into its own sub-directory, and a copy of the original ASCII data file will be archived separately as a quality control measure. The contractor will analyze the SPSS data files, including conducting frequencies and cross-tabulations. The analyses will be geared toward providing VBA staff with user-oriented results.
Create monthly sample disposition reports.
Monthly disposition reports, which include total calls made, the number and percent of refusals, terminates, number of completes, incidence, response, and cooperation rate, will be provided to VBA at the end of each fielding month.
Create monthly data matrices.
The contractor will provide VBA with monthly data matrices via a secure FTP site that is accessible to appropriate VBA staff and management. Matrices shall provide cumulative summaries of all data month-to-date and year-to-date, on a secure FTP site, hosted by the contractor. The reports shall provide data at the National and call center levels for month-to-date, quarter-to-date, and fiscal year-to-date aggregations.
Conduct daily data loads to an online reporting tool.
The contractor will load the call center data collected to an online reporting tool (i.e. VBA’s Voice of the Veteran (VOV) Reporting Site) or internal tool as designated that is updated on a daily basis (Monday thru Friday, excluding holidays) with survey returns collected on the previous day. The online reporting tool will be accessible to VBA and display results aggregated at the National level as well as tiered by each individual call center. At the end of each month, data on the reporting tool will be finalized and weighted. The contractor will notify VBA each month once the final month end data has been uploaded.
Conduct biannual formal briefings.
The contractor will prepare semi-annual overviews of data collected for VBA leadership. Results are published within the Performance Accountability Report, and budget submissions. This overview will be delivered in-person to VBA leadership and outline data trends and recommendations based on the data collected.
16B: PROJECT TIMELINE
Timeline – Call Center Satisfaction Surveys
An example of an annual fielding timeline is included in the supporting documents of this information collection.
If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.
We are not seeking approval to omit the expiration date for OMB approval.
Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.
This submission does not contain any exceptions to the certification statement.
1 The current contractor is J.D. Power. Contract personnel are subject to change and will be updated as administrative changes. The term contractor is used throughout the document.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SF 83 SS VAF 21-8926 |
| Author | Veterans Benefits Administrat |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |