OMB Control Number: 2900-0782
A. JUSTIFICATION
In 2008, VBA recognized a need to develop and design an integrated, comprehensive Voice of the Veteran (VOV) Continuous Measurement (CM) program for its lines of business: Compensation Service (CS), Pension Service (PS), Education (EDU) Service, Loan Guaranty (LGY) Service and Vocational Rehabilitation and Employment (VR&E) Service. The VOV CM program provides insight regarding Veterans and beneficiaries interactions with the benefits and services provided by VBA. The VOV CM provides VBA leadership with actionable Veteran feedback on how VBA is performing. These insights help identify opportunities for improvement and measure the impact of improvement initiatives. This study is supported by contractor staff from J.D. Power and Associates.
VBA conducted a benchmark study in Fiscal Year 2013 (October 2012 through January 2013) in order to validate the survey instruments, identify Key Performance Indicators, and establish performance benchmarks. Findings and recommendations were presented to VBA Leadership and stakeholders within each line of business in April 2013.
Based on interviews conducted, VBA has separated the Veterans experience with VBA into two categories:
Access to a Benefit. This measures the enrollment experience transaction with the beneficiary or Veteran.
Servicing of a Benefit. This measures the ongoing relationship experiences with the beneficiary or Veteran.
Each business line desired to understand the components of the overall customer experience. Each VBA business line wanted to engage their Veteran population with relevant questions regarding their experience. The following outlines how that is approached with each of the lines of business.
Compensation and Pension Programs
During 2014 J. D. Power fielded three survey instruments for the Compensation and Pension programs. Discussions with stakeholders from both programs indicated that one survey instrument could be used for both Compensation and Pension Enrollment category claimants. In FY2015, Compensation and Pension identified the need to separate the Enrollment survey to better serve the business needs of each program.
The Compensation Enrollment survey pool for the VOV Continuous Measurement Study includes individuals who have received a decision on a compensation benefit claim within 30 days prior to the fielding period. This includes those who were found eligible on a new or subsequent claim and those who have been denied and lack a current appeal of the decision. The Pension Enrollment survey pool includes individuals who have received a decision on a pension benefit claim within the past 30 days. The Compensation Servicing survey pool includes individuals who received a decision and are receiving benefit payments. The Pension Servicing survey pool includes individuals who established and completed a claim in the previous fiscal year.
Education Program
VBA fielded two survey instruments for Education Service. The Education Enrollment survey pool includes individuals who received a decision on their education benefit application within 90 days (i.e., the original end-product was cleared within the past 90 days) prior to the fielding period. The Education Servicing survey pool includes beneficiaries who are currently receiving benefits. The definition of those receiving benefits varies based on the educational program. Chapter 33 beneficiaries who have received at least 2 payments for “tuition” in the past 9 months are included in the survey pool. Chapter 30, Chapter 1606, and Chapter 1607 beneficiaries who have received 5 monthly payments during the past 9 months are included.
Loan Guaranty and Specially Adapted Housing Programs
VBA fielded two survey instruments for Loan Guaranty Service. The survey pool for the tracking study for the LGY Enrollment questionnaire includes individuals from a 30 day period who closed on a VA home loan in the 90 days prior to the fielding period. The sample is stratified as follows: (1) those who closed on purchase loans, (2) those who received loans for interest rate reductions, and (3) those who obtained cash out or other refinancing. The survey pool for the tracking study for the SAH Servicing questionnaire includes individuals who are eligible for a specially adapted housing grant and in the past 12 months have: (1) received an approval on their grant and are currently somewhere in post-approval, (2) have had all their funds dispersed and final accounting is not yet complete, and (3) have had all of their funds dispersed and final accounting is complete.
Vocational Rehabilitation and Employment Program
VBA fielded three survey instruments for Vocational Rehabilitation & Employment Service (VR&E). The VR&E Enrollment survey pool includes individuals who applied within the last 12 months, entered Evaluation and Planning and (1) entered any of the following case statuses: Extended Evaluation, Independent Living (IL), Rehabilitation to Employment (RTE), or Job Ready Status (JRS) (excludes re-applicants), or (2) were found not entitled. The VR&E Servicing survey pool includes individuals who in the last 30 days were in a plan of services for more than 60 days, all rehabilitated participants, and MRGs. Participants who interrupted their plan are excluded. The VR&E Non-Participant survey explores why eligible individuals chose not to pursue the benefit entitlement. The VR&E Non-Participant questionnaire survey pool includes individuals who dropped out of the program prior to completing a rehabilitation plan. The sample is stratified as follows: (1) applicants who never attended the initial meeting with a counselor, (2) applicants who were entitled to the program but did not pursue a plan of service, and (3) applicants who started, but did not complete a rehabilitation plan (i.e., negative closures).
Findings
Findings and recommendations from the survey fielding periods were presented to VBA leadership and stakeholders within each line of business in FY 2014 and FY 2015. These discussions identified a need to refine the survey instruments due to technological and process changes within the VBA programs. J.D. Power collaborated with the lines of business in order to edit the survey instruments.
The results of the VOV Continuous Measurement Study allow VBA to measure the effectiveness of new initiatives and changes in processes identified within the VOV Study. Further, since the benchmark study, the fielded volume for each survey-type has been increased to provide statistically valid sample sizes for each Regional Office. These changes were reviewed by statisticians and VBA leadership.
These surveys will be conducted per the legislative requirement set forth in the Executive Orders 12862 - Setting Customer Service Standards and 13571- Streamlining Service Delivery and Improving Customer Service. The Executive Orders mandate that government agencies set standards and seek to measure agency performance against such standards while streamlining and improving delivery of benefits and services. These surveys are also being conducted in concurrence with the Government Performance Results Act (GPRA) and its requirements for improving customer service.
Specifically, VBA requires customer satisfaction information from Veterans in order to track the effectiveness and usability of the benefits offered, as well as to assess the level of satisfaction with the level of service provided by VBA. The data collected is used by VBA to make improvements to the benefit application and enrollment process, which in turn, enables VBA to maximize the efficiency and quality of service provided to its nation’s Veterans and their dependents.
The information to be collected with the planned surveys is used by VBA leadership and the lines of business: Compensation Service (CS), Pension Service (PS), Education (EDU) Service, Loan Guaranty (LGY) Service, and Vocational Rehabilitation and Employment (VR&E) Service to determine Veteran satisfaction with benefits and services. The major use of this data is three-fold: 1) to provide Veterans an opportunity to comment on their experience with benefits and services, 2) to formulate program and policy changes in the enrollment and servicing process to ensure the most effective delivery of benefits and services, and 3) to identify the most influential elements of Veterans’ experience on their satisfaction and establish performance benchmarks to gauge the effectiveness of program and policy changes in improving Veterans’ experience with benefits and services. Currently, there are no other means of continuously evaluating Veteran satisfaction with VBA’s benefit programs.
3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.
In conjunction with a contractor, JDPA, VBA staff develops, administers, and analyzes the Compensation, Pension, Education, and VR&E Enrollment surveys, Compensation, Pension, Education, and VR&E Servicing surveys, the LGY and SAH surveys, and the VR&E Non-Participant surveys.
The majority of the surveys are self-administered providing the respondent the choice of either a paper form or a web-based data collection. The exceptions are the Pension Access and Servicing, SAH Servicing, VR&E Servicing, and VR&E Non-Participant survey which are available in paper form only. This maximizes the timeliness, efficiency, and response rate of data collection from Veterans. Based on previous years fielding experience, it is expected that the completion rates will be composed of 25% online collection and 75% of respondents will be received by the mail option.
JDPA has strong capabilities in programming and hosting surveys on the worldwide web, and maintains effective security and privacy procedures (e.g., unique passwords for respondents, data encryption) when designing and programming web surveys. The invitation to the electronic surveys are included in the materials sent to Veterans advertising the surveys. Veterans can either connect to the appropriate web page and complete the survey or fill out the survey included in the materials packet.
4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.
As noted, VBA has not recently conducted customer satisfaction surveys specifically geared toward Veteran satisfaction with CS, PS, EDU, LGY or VR&E benefits and services on a continuous basis since 2005.
Our review of available data suggests that no sources outside of VBA’s VOV survey program are available that provide a reliable, representative, and statistically valid sample of the targeted audience who have used CS, PS, EDU, LGY or VR&E services and from which detailed customer satisfaction indicators can be derived.
No small businesses or other small entities are impacted by this information collection.
If the Voice of the Veteran tracking surveys are not conducted, VBA will not be able to fulfill its performance measurement requirements as articulated by the requirements of Public Law 103-62, the Government Performance and Results Act of 1993 and Title 38 USC, §527, Evaluation and Data Collection. Furthermore, VBA would miss an opportunity to learn from America’s Veterans about how to improve the CS, PS, EDU, LGY or VR&E benefit enrollment and servicing processes to best serve their needs. VBA would also lose an opportunity to track and document improvements or declines in Veteran satisfaction and service delivery over time.
There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys are designed and carried out with appropriate scientific rigor, and are intended to produce valid and reliable results that can be generalized to the universe of study.
Original Notice:
The 60-day Federal Register (FR) notice was published on 10/06/16, pages 69571-69572.
The 30-day Federal Register (FR) notice was published on 12/23/16, pages 94485-94486.
Comments:
Name: Andrew Straw
Docket Number: VA-2016-VACO-0001
Date Posted: 10/07/16
FR Publication: OMB No. 0782; FR Doc. #2016-23241
Response:
We appreciate the comment and interest in the Voice of the Veteran Customer Satisfaction Continuous Measurement Survey Program. Upon further review, the submitted comment is related to the proposed rule RIN: 2900-AP66, AP66 - Proposed Rule - Diseases Associated With Exposure to Contaminants in the Water Supply at Camp Lejeune, cited within the Federal Register, Volume 81, Number 175, dated Friday, September 9, 2016. The submitted comment was forwarded internally to the Veterans Benefit Administration Compensation Service, Regulation Staff, for the appropriate action.
8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances which preclude consultation every three years with representatives of those from whom information is to be obtained.
VBA is not aware of any tracking surveys in private or government sectors.
No payment or gift shall be provided to respondents.
An assurance that the information will be kept private to the extent of the law is made in the introductory letter/postcards as well as the reminder letter respondents receive with the survey, regardless of the mode of administration. Respondents are assured that answers given will be kept private to the extent of the law and will not affect their current or future entitlement to benefits and services. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code).
None of the survey instruments contain any questions of a sensitive nature.
The respondent population is composed of Veterans and other beneficiaries who applied for or received benefits from Compensation Service, Pension Service, Education Service, Loan Guaranty Service or Vocational Rehabilitation and Employment Service. VBA cannot make further assumptions about the population of respondents because of the variability of factors such as the educational background and wage potential of respondents. Therefore, VBA used general wage data to estimate the respondents’ costs associated with completing the information collection.
The Bureau of Labor Statistics gathers information on full-time wage and salary workers. Accordingly, the median weekly earnings of full-time wage and salary workers is $936.80. Assuming a forty (40) hour work week, the median hourly wage is $23.421.
Legally, respondents may not pay a person or business for assistance in completing the information collection and a person or business may not accept payment for assisting a respondent in completing the information collection. Therefore, there are no expected overhead costs for completing the information collection. VBA estimates the total cost to all respondents to be $765,834.00 (32,770 burden hours x $23.42 per hour).
In order to understand the different elements that make up the overall experience with VBA for a Veteran, questions were added to the surveys that result in a variation of length in the questionnaires. Questionnaires range from 10 minutes to 15 minutes depending on the benefit line, method of contact and Veteran experience.
The volumes below reflect the estimated burden per year for the duration of FY 2016. These burdens are estimated based upon the FY15 actual response rates.
TABLE 1. FY 2016 TIME BURDEN, RESPONDANT COST, AND FREQUENCY BY RESPONDENT GROUP |
|||||||
Population surveyed |
Number |
Number of |
Annual |
Annual |
Annual burden |
Total Annual Burden hours |
Annual |
Compensation Enrollment |
160,000 |
1 |
48,000 |
Annually |
15 |
12,000.00 |
$281,040.00 |
Compensation Servicing |
60,000 |
1 |
18,000 |
Annually |
15 |
4,500.00 |
$105,390.00 |
Pension Enrollment |
10,000 |
1 |
3,000 |
Annually |
15 |
750.00 |
$17,565.00 |
Pension Servicing |
10,000 |
1 |
3,000 |
Annually |
15 |
750.00 |
$17,565.00 |
Education Enrollment |
16,000 |
1 |
4,800 |
Annually |
15 |
1,200.00 |
$28,104.00 |
Education Servicing |
10,000 |
1 |
3,000 |
Annually |
15 |
750.00 |
$17,565.00 |
VR&E Enrollment |
60,000 |
1 |
18,000 |
Annually |
15 |
4,500.00 |
$105,390.00 |
VR&E Servicing |
60,000 |
1 |
18,000 |
Annually |
15 |
4,500.00 |
$105,390.00 |
VR&E Non-Participant |
5,000 |
1 |
1,500 |
Annually |
15 |
375.00 |
$8,782.50 |
Loan Guaranty Home Loan Process |
40,000 |
1 |
12,000 |
Annually |
15 |
3,000.00 |
$70,260.00 |
Specialty Adapted Housing Grant Process (Mail) |
5,000 |
1 |
1,500 |
Annually |
15 |
375.00 |
$8,782.50 |
Totals |
436,000 |
- |
130,800 |
- |
165 |
32,700 |
$765,834.00 |
According to the U.S. Bureau of Labor Statistics Average Hourly Earnings, the cost to each respondent is $5.86.
a. There is no capital, start-up, operation, or maintenance costs.
b. Cost estimates are not expected to vary widely. The only true cost is that for the time of the respondent (average of 15 minutes per respondent).
c. There are no anticipated capital start-up cost components or requests to provide information.
Table 2 below shows the labor and contracting costs for conducting the surveys for FY17. Operational costs are outsourced to a contractor and are included in the contractor’s total cost.
TABLE 3: ESTIMATED COST TO THE FEDERAL GOVERNMENT |
||||
Cost Item |
|
Hourly Wages |
Hours |
Cost |
VA-Labor |
GS-13, 5 |
$50.21 |
2,236 |
$112,265.48 |
|
GS-14, 5 |
$59.33 |
520 |
$30,851.50 |
Contractor |
- |
- |
- |
$1,529,882.72 |
TOTAL |
|
|
$2,756 |
$1,672,999.70 |
The VA Labor cost was estimated using the 2017 GS base hourly pay scale average salary of $38.12 per hour for 2,236 hours at a GS-13, step 3 and $45.05 per hour for 520 hours at a GS-14, step 3.
.2 The amount paid to the contractor for the survey effort includes as its major components the survey of recent claimants or existing beneficiaries for an annual cost of $1,529,882.72 in FY 2016. These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for Web administration, questionnaire pretest, validation, data processing, providing a clean data file, project management and analysis, and reporting.
There was a reduction in the survey volume for VR&E Enrollment, which decreased from 60,000 to 14,000 based upon the actual population of VR&E Enrollment recipients on a yearly basis.
During the survey field-period, the surveys are scanned as they are received to validate the name and address data file. During the scanning process, VBA’s contractor validates survey responses against the data file and removes the names of respondents who have completed a survey instrument during the fielding period.
The contractor hosts a secure Reporting Site that provides response rates from the survey. During the survey fielding period, the response rate data is updated at least monthly.
Program lines receive an overview of findings that outline the following:
Summary of key score matrices
Key attributes
Key diagnostics and demographics
Verbatim (open-ended) response report
Semi-annual program Line Briefings
Each VBA business line (CS, PS, EDU, LGY, and VR&E) receives a semi-annual briefing and report that summarizes survey results. The graphically interface for these reports is automated by the vendor. Development of the automated report system consists of several concurrent tasks, including: a data crosswalk of the survey iterations to map identical variables and response sets; Statistical Package for the Social Sciences (SPSS) syntax routines; and development of prototype report templates using Microsoft Excel with Visual Basic for Applications.
The contractor develops data tabulations on survey highlights and presents the results to CS, PS, EDU, LGY, VR&E and VBA staff.
The contractor submits the final deliverable containing electronic files as specified in the SOW. Documents and materials include:
Raw survey data in SPSS format
Copies of the reports in electronic format
Copies of the questionnaire and survey mailing materials
Survey respondent verbatims, grouped by Regional Office/Line of Business
The major activities for the VBA VOV survey project are structured by task, and are outlined below.
Task 1: Conduct kickoff meeting and develop the Project Management Plan (PMP) and Risk Management Plan (RMP)
Task 2: Monthly Status Meetings and written summaries
Task 3: Methodology, Sampling Plans, and Production Schedule
Task 4: Final Surveys, letters, postcards, and mailing materials to the print vendor
Task 5: Sample cleaning, Imaging and Data Collection, and Disposition Reports
Task 6: Host live survey websites and Disposition Reports
Task 7: Host toll-free telephone and web helpdesk assistance and corresponding call/inquiry logs
Task 8: Data Matrices, Verbatim Reports, Data Analysis
Task 9: Data Load to VOV Reporting Site and Training Sessions
Task 10: Reporting Site Training Sessions
Task 11: Business line analysis and presentations
Task 12: Semi-Annual Leadership Briefings
Task 13: Action Plan development and sessions
Task 14: Nonresponse Bias Reports per Office of Management and Budget (OMB) Standards and Guidelines for Statistical Surveys
TIMELINe – VOICE OF THE VETERAN SURVEYS embeded PMP and make sure it is current- As discussed, did not add
Conduct kickoff meeting and develop the project management plan (PMP) |
Continuous (annually) |
Monthly Status Meetings and written summaries |
Continuous |
Methodology and Sampling Plans, and Production Schedule |
Continuous (annually) |
Sample cleaning, Imaging and Data Collection, and Disposition Reports |
Continuous |
Host live survey websites and Disposition Reports |
Continuous |
Host toll-free telephone and web helpdesk assistance and corresponding call/inquiry logs |
Continuous |
Data Matrices, Verbatim Reports, Data Analysis |
Continuous |
Data Load to VOV Reporting Site and Training Sessions |
Continuous |
Business line analysis and presentations |
Continuous |
Semi-Annual Leadership Briefings |
Continuous (twice yearly) |
Action Plan development and sessions |
Continuous |
The collection survey instrument may be reproduced and/or stocked by respondents and Veterans organizations. Requiring VA to display an expiration date on the form would result in unnecessary waste of existing stocks. Inclusion of the expiration date could also result in a delay of the Department's action on the benefit being sought (respondent trying to obtain a newer version, while VA would have accepted the older version). For the reasons stated, VA continues to seek an exemption that waives the displaying of the expiration date.
There are no exceptions.
1 Occupational Code 00-0000. Wage and Salary Data obtained from Economic Release dated September 2016 https://www.bls.gov/news.release/ecec.t01.htm
2 2017 General Schedule Base Hourly Salary
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement for VBA Generic Customer Survey Clearance |
| Author | VBA |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |