Download:
pdf |
pdfService Level Measurements – Boards of Veterans
Appeals Survey
Sampling Methodology Report
Prepared by
Veteran Experience Office
Version 1.1
September 2018
1
Contents
Executive Summary ..................................................................................................................................... 3
Part I – Introduction.................................................................................................................................... 4
A. Background ........................................................................................................................................... 4
B. Basic Definitions ................................................................................................................................... 4
C. Application to Veterans Affairs ............................................................................................................ 5
Part II – Methodology ................................................................................................................................. 5
A. Target Population and Frame ............................................................................................................... 5
B.
Sample Size Determination................................................................................................................ 6
C.
Data Collection Methods ................................................................................................................... 8
D.
Reporting............................................................................................................................................ 9
E.
Quality Control .................................................................................................................................. 9
F.
Sample Weighting, Coverage Bias, and Non-Response Bias ........................................................... 10
G.
Quarantine Rules ............................................................................................................................... 11
Part III – Assumptions and Limitations ................................................................................................... 12
A. Respondent Satisfaction Bias ............................................................................................................... 12
B. Coverage Bias....................................................................................................................................... 12
Appendix 1. List of Data Extraction Variables .................................................................................... 13
Appendix 2. Survey Questions ............................................................................................................... 14
Appendix 3. References .......................................................................................................................... 23
2
Executive Summary
The Board of Veteran’s Appeals Survey is designed to measure Veterans’ (or their Beneficiary’s) satisfaction with the appeals process
regarding their benefit claims, as reviewed by the Board of Veterans Appeals. The Board passes final judgement on cases of denied or diminished
benefits originating within the Veterans Benefits Administration (VBA). The goal of the VEO survey operation is to continuously provide highly
reliable monthly overall estimates.
Those recently completing one of three aspects of the appeals process are eligible for the survey:
1. Initial Filing Survey
2. Hearing Proceeds Survey
3. Survey on the Reporting of the Board’s Decision
A portion of the Veterans recently engaging the Board in the three stages listed above will be contacted by email to complete a brief online
survey. This survey is conducted weekly, with reporting also available weekly, monthly, and quarterly. The questions aim to measure Veterans’
experience with the general process, including their interactions with Board systems, staff, and judges, as well as their clarity and understanding of
the Board’s procedures and decisions. The measurement scale is a 5-point Likert, and is based on a short list of multiple-choice questions that
effectively solicit Veterans’ feedback. Once the invitation is received, participants will have two weeks to complete the survey at their
convenience. Once data collection is completed, a series of internal quality checks are implemented. The participant responses in the online survey
are weighted so the samples will more closely resemble the overall population. Sample weighting will be applied using Age Group.
This report describes the methodology used to collect and disseminate the Board of Veterans Appeals Survey. Information about quality
assurance protocols as well as limitations of the survey methodology is also included in this report.
3
Part I – Introduction
A. Background
The Enterprise Measurement and Design team (EMD) is part of the Insights and Analytics (I&A) division within the Veterans
Experience Office (VEO). The EMD team is tasked with conducting transactional surveys of the Veteran population to measure their satisfaction
with the Department of Veterans Affairs (VA) numerous benefit services. Thus, their mission is to empower Veterans by rapidly and discreetly
collecting feedback on their interactions with such VA entities as NCA, VHA, and VBA. VEO surveys generally entail probability samples which
only contact minimal numbers of Veterans necessary to obtain reliable estimates. This information is subsequently used by internal stakeholders to
monitor, evaluate, and improve beneficiary processes. Veterans are always able to decline participation, and have the ability to opt out of future
invitations. A quarantine protocol is maintained to limit the number of times a Veteran may be contacted, in order to prevent survey fatigue,
across all VEO surveys.
The Board of Veterans Appeals administered by the Department of Veterans Affairs (VA) determines whether Veterans are entitled to
claimed benefits which have initially been denied or reduced. Its mission is to conduct hearings and pass proper judgement on appeals in a timely
manner. The vast majority of appeals involve claims for disability compensation, in addition to other veteran benefits. the Board allows Veterans
to submit medial and lay evidence at any point during the appeals process. Veterans Law Judges will eventually review benefit claims
determinations made by local VA offices and issue decisions on appeals.
In order to continue to provide quality services to Veterans, the Veteran Experience Office (VEO) has been solicited to measure the
satisfaction of Veterans regarding the Board appeals and hearing procedures for their benefit claims. VEO proposed to conduct a brief survey on
Veterans who recently completed various aspects of the appeals process.
B. Basic Definitions
Coverage
The percentage of the population of interest that is included in the sampling frame.
Measurement Error
The difference between the response coded and the true value of the characteristic being studied for a
respondent.
Non-Response
Failure of some respondents in the sample to provide responses in the survey.
Transaction
A transaction refers to the specific time a Veteran interacts with the VA that impacts the Veteran’s journey
and their perception of VA’s effectiveness in caring for Veterans.
4
Response Rate
The ratio of participating persons to the number of contacted persons. This is one of the basic indicators of
survey quality.
Sample
In statistics, a data sample is a set of data collected and/or selected from a statistical population by a defined
procedure.
Sampling Error
Error due to taking a particular sample instead of measuring every unit in the population.
Sampling Frame
A list of units in the population from which a sample may be selected.
Reliability
The consistency or dependability of a measure. Also referred to as standard error.
C. Application to Veterans Affairs
In general, customer experience and satisfaction are usually measured at three levels: the enterprise level, the service level patterns, and
point-of-service feedback. This measurement may bring insights and value to all stakeholders at VA. Front-line VA leaders can resolve individual
feedback from Veterans and take steps to improve the customer experience; meanwhile VA executives can receive real-time updates on systematic
trends that allow them to make changes.
1) To collect continuous customer experience data that make or break the service experience.
2) To help field staff and the national office identify areas of improvement.
3) To understand emerging drivers and detractors of customer experience.
Part II – Methodology
A. Target Population and Frame
The target population of the Board Survey is all Veterans who have completed a Board of Veterans’ appeals process items within the past
week. The survey frame, a listing of all appellants, is provided to VEO directly by the Board. This listing is then reconciled with an email database
from the Veteran’s Health Administration, and the survey is offered to appellants with a recent interaction for whom an email address is listed
within VHA. This will correspond to roughly 30% of the true underlying population.
5
Figure 1. Measurement Goals and Survey Mode
Survey
Types
• Filing
• Hearing
• Decision
Data
Collection
• Email Recruitment
• Online Survey Interface
• 2 Weeks to Complete
B. Sample Size Determination
To achieve a certain level of reliability, a sufficiently large sample is required. The sample size for a given level of reliability is calculated
below (Lohr, 1999). Note that the reliability of the survey estimates is restricted by the amount of available sample. Due to limitations on the
number of Veterans that have shared an email address with VA, coupled with the expected non-response, the the Board Survey may require to
contact all eligible Veterans who also have an email address. We proceed to depict the expected level of precision (i.e., reliability) against various
standards.
For a population that is large, the equation below is used to yield a representative sample for proportions:
𝑛0 =
2
𝑍𝛼/2
𝑝𝑞
𝑒2
where
-
𝒁𝜶/𝟐 = 1.96, which is the critical Z score value under the normal distribution when using a 95% confidence level (α = 0.05).
p = the estimated proportion of an attribute that is present in the population, with q=1-p.
o Note that pq attains its maximum when value p=0.5, and this is sometimes used for a conservative sample size (i.e., large enough
for any proportion).
e = the desired level of precision; in the current case, the margin of error e = 0.03, or 3%. Also referred to as MOE.
6
For a population that is relatively small, the finite population correction is used to yield a representative sample for proportions:
𝑛=
𝑛0
𝑛
1+ 0
𝑁
Where
-
𝒏𝟎 = Representative sample for proportions when the population is large.
N = Population size.
The margin of error surrounding the baseline proportion is calculated as:
𝑁 − 𝑛 𝑝(1 − 𝑝)
√
𝑀𝑎𝑟𝑔𝑖𝑛 𝑜𝑓 𝐸𝑟𝑟𝑜𝑟 = 𝑧𝛼/2 √
𝑁−1
𝑛
Where
-
𝒁𝜶/𝟐 = 1.96, which is the critical Z score value under the normal distribution when using a 95% confidence level (α = 0.05).
N = Population size.
n = Representative sample.
p = the estimated proportion of an attribute that is present in the population, with q=1-p.
Table 2A depicts the approximate monthly population for the the Board appeals, as well as the sample size determination with Confidence
Level (CL) and Margin of Error (MOE) at various levels. The sample size numbers listed indicate the number of responses from Veterans required
to attain the stated level of precision.
Based on the estimated number of appellants filing with the Board each month (6,000), the email availability rate (30%), and the expected
survey response rate (20%), the expected number of responses from contacted Veterans would be approximately 360. This number would
represent the number of responses if every qualified Veteran/Beneficiary were sent a survey invitation. However, this number is still much smaller
(360 vs. 907) than the desired level of precision corresponding to a 95% CL and 3% MOE (Lohr, 1999). Therefore, it is reasonable to sample all
available Veterans to produce the most reliable estimates possible – this would correspond to reliability levels at the 90% CL and 5% MOE. A
similar situation holds for the Decision population: 7,400 monthly decisions will result in . In the Board appeals hearing population, the expected
respondents with 100% sampling rate are still below any of the sample sizes at proposed CL and MOE, due to the small hearing population size.
Therefore, all Veterans in this population will also be sampled.
7
Table 2B provides the weekly and monthly sample targets for the appellant and hearing populations, presuming a return rate of 20%. The
current method obviates the need for any stratification or formal probability sampling.
Table 2A. Target Population Figures
Population
Type
Approximate
Monthly
Population
Approximate
Monthly Email
Population
Precision
Sample:
95% CL,
3% MOE
Precision
Sample:
90% CL,
5% MOE
Precision
Sample:
90% CL,
8% MOE
Filing
6,000
1,800
907
258
N/A
Hearing
1,700
510
656
233
100
Decision
7,400
2,220
933
260
N/A
Table 2B. Proposed Sample Targets by Time Period
Population Type
Weekly Target
Weekly Contact
Monthly Target
Monthly Contact
Filing
75
375
300
1,500
Hearing
25
125
100
500
Decision
100
500
400
2,000
Total
200
1,000
800
4,000
C. Data Collection Methods
At the beginning of every weekly measurement period, the Board will review records and send participant information to VEO for the
survey distribution and data collection. Given that Veteran’s email address is not available to the Board, these Veteran records will be mapped
using social security number to the VHA Corporate Data Warehouse (CDW) to collect email addresses. This will permit VEO to share the online
survey link to appellants who have shared their email address with VHA. Note that Veterans with private or other health care providers who do not
8
access VHA for health benefits will not be contacted. Thus, the email population may differ somewhat from the true underlying population,
possibly introducing coverage-bias into the survey estimates (see section below).
After the Board participants have their email address appended to their records, an invitation file is created and stored. Emails are
immediately delivered to all selected Veterans. Respondents will have 14 days to complete the survey. A reminder email will be sent after one
week to those who have yet to complete the survey. Estimates will be accessible to data users instantly within the Veteran Signals (VSignals).
D. Reporting
Researchers will be able to use the Veteran Signals (VSignals) system for interactive reporting and data visualization. VA employees with
a PIV card may access the system at https://va.voice.medallia.com/sso/va/. The scores may be viewed by Age Group, Gender, and
Race/Ethnicity in various charts for different perspective. They are also depicted within time series plots to investigate trends. Finally, filter
options are available to assess scores at varying time periods and within the context of other collected variable information.
Recruitment is continuous (weekly) but the results should be combined into a monthly estimate for more precise estimates, which is the
recommended reporting level. Weekly estimates are less reliable for small domains, (i.e., VAMC-level) and should only be considered for
aggregated populations. Monthly estimates will have larger sample sizes, and therefore higher reliability. Quarterly estimates are the most precise,
but will take the greatest amount of time to obtain (12 weeks of collection). However, Quarterly estimates are the most suitable for the analysis of
small populations.
E. Quality Control
To ensure the prevention of errors and inconsistencies in the data and the analysis, quality control procedures will be instituted in several
steps of the survey process. Records will undergo a cleaning during the population file creation. The quality control steps are as follows.
1. Records will be reviewed for missing sampling and weighting variable data. When records with missing data are discovered, they will be
either excluded from the population file or put into separate strata upon discussion with subject matter experts.
2. Any duplicate records will be removed from the population file to both maintain the probabilities of selection and prevent the double
sampling of the same Veteran.
3. Invalid emails will be removed.
The survey sample loading and administration processes will have quality control measures built into them.
1. The survey load process will be rigorously tested prior to the induction of the the Board Survey to ensure that sampled customers is not
inadvertently dropped or sent multiple emails.
2. The email delivery process is monitored to ensure that bounce-back records will not hold up the email delivery process.
9
The weighting and data management quality control checks are as follows:
1. The sum of the weighted respondents will be compared to the overall population count to confirm that the records are being properly
weighted. When the sum does not match the population count, weighting classes will be collapsed to correct this issue.
2. The unequal weighting effect will be used to identify potential issues in the weighting process. Large unequal weighting effects indicate a
problem with the weighting classes, such as a record receiving a large weight to compensate for nonresponse or coverage bias.
F. Sample Weighting, Coverage Bias, and Non-Response Bias
Weighting is commonly applied in surveys to adjust for nonresponse bias and coverage bias. Nonresponse is defined as failure of
selected persons in the sample to provide responses. This is observed virtually in all surveys, in that some groups are more or less prone to
complete the survey. The nonresponse issue may cause some groups to be over or under represented. Under-coverage is another common survey
problem, in which certain group of interest in the population is not included in the sampling frame. The reason that these people cannot participate
is because they cannot be contacted. For instance, individuals without a valid email address cannot be reached via email recruitment. In both cases,
no reliable conclusions can be drawn from the survey data, due to nonresponse and coverage bias.
Survey practitioners recommend the use of sampling weighting to improve inference on the population, so that the final respondent sample
is closely resemble the full population. Although all available Veterans from the appellant and hearing population are surveyed, differential
response rates may still be observed across different age groups. Weighting can help adjust for the demographic representation by assigning larger
weights to under-represented age group and smaller weights to over represented age group. Also, because the email population will have different
demographics than the overall population, the initial sample will be selected in a manner from the frame so that the final respondent sample
resembles the overall population. In both ways of adjustments, weighting may result in substantial correction in the final survey estimates when
compared to direct estimates in the presence of non-negligible sample error.
In summary, the Board Survey is weighted by respondent’s age, and the final respondent sample is representative of the full population.
The population values will be collected and recorded by VEO for every data collection period. Sample weights are generated for all estimates:
weekly, monthly, and quarterly.
It was reported earlier that the email population comprises 30% of the full population. Since 85% of older Americans utilize email (Choi
& Dinitto, 2013), we can presume that most Veterans choose not to share their email address with the VA or are simply unware of that option. It is
assumed that the level of patient satisfaction is not directly related to their email status (Missing at Random). Since age has been observed to be a
strong predictor of Veteran satisfaction in other VA surveys, the weighting methodology outlined above will adequately compensate for any bias
introduced by the incomplete frame of population.
If we let wij denote the sample weight for the ith person in age group j (j=1, 2, and 3), then the CW formula is:
10
𝑤𝑖𝑗 =
# 𝑉𝑒𝑡𝑒𝑟𝑎𝑛𝑠 𝑖𝑛 𝑎𝑔𝑒 𝑔𝑟𝑜𝑢𝑝 𝑗 𝑖𝑛 𝑡ℎ𝑒 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛
# 𝑉𝑒𝑡𝑒𝑟𝑎𝑛𝑠 𝑖𝑛 𝑎𝑔𝑒 𝑔𝑟𝑜𝑢𝑝 𝑗 𝑖𝑛 𝑡ℎ𝑒 𝑠𝑎𝑚𝑝𝑙𝑒
As part of the weighting validation process, the weights of persons in an age group is summed and verified that it matches the universe
estimates (i.e., population totals). Additionally, we calculate the unequal weighting effect, or UWE (see Kish, 1992; Liu et al., 2002). This statistic
is an indication of the amount of variation that may be expected due to the inclusion of weighting. The unequal weighting effect estimates the
percent increase in the variance of the final estimate due to the presence of weights and is calculated as:
𝑠
2
𝑈𝑊𝐸 = 1 + 𝑐𝑣𝑤𝑒𝑖𝑔ℎ𝑡𝑠
= ( )2
𝑤
̅
where
-
cv = coefficient of variation for all weights 𝑤𝑖𝑗 .
s = sample standard deviation of weights.
1
̅ = sample mean of weights, 𝑤
𝒘
̅ = 𝑛 ∑𝑖𝑗 𝑤 ij.
G. Quarantine Rules
VEO seeks to limit contact with Veterans as much as possible, and only as needed to achieve measurement goals. These rules are enacted
to prevent excessive recruitment attempts upon Veteran inpatients. VEO also monitors Veteran participation within other surveys, to ensure
Veterans do not experience survey fatigue. All VEO surveys offer options for respondents to opt out, and ensure they are no longer contacted for a
specific survey.
Table 4. Proposed Quarantine Protocol
Quarantine Rule
Description
Elapsed Time
Repeated Sampling
for the Board Survey
Number of days between completing online survey, and receiving
another online survey related to another complaint.
3 Months or 90
Days
Other Surveys
Veterans engaged that have recently completed other VEO
surveys will not be selected for 30 days.
1 Month or 30
Days
Anonymous
Callers explicitly wishing to remain anonymous will not be
contacted.
N/A
11
Opt Outs
Persons indicating their wish to opt out of either phone or online
survey will no longer be contacted.
N/A
Part III – Assumptions and Limitations
A. Respondent Satisfaction Bias
According to the survey design, all qualified Veterans will be surveyed to understand their satisfaction with the overall appeals/hearing
process. There is possibility that Veterans’ satisfaction may be interacted with the appeals decisions they received. In other words, Veterans who
received an approval or positive review on their claim submissions may perceive higher general satisfaction with the Board services than those
who did not receive an approval, regardless of the services per se. This hypothesis needs to be examined once the survey is implemented. VEO is
aware of this potential bias and will carefully review the survey results once the data collection is completed.
B. Coverage Bias
The Veterans’ contact information provided by the Board does not contain participants’ email address, which is the primary point of
contact for most VEO surveys. Social security numbers are collected by the Board, however, and this may be used to obtain a valid email address
by accessing email records within CDW, the VHA database system. However, not all Veterans have provided their email address to VHA. Indeed,
if the there is a substantial difference in customer satisfaction between appellants who frequent VA hospitals and those who do not, the final
estimates will suffer from coverage bias. Although we may assume that a Veteran’s satisfaction with the appeals process is independent of whether
they frequented a VA hospital in the past, there may be differences in the socio-demographic composition between those utilizing VA services and
those accessing private (or other) healthcare sources. Coverage bias will be mitigated through the use of sample weighting to ensure the VHA
population is modified to greater reflect the Board population, with respect to demographic variables such as Age Group, Gender, and
Race/Ethnicity.
Additionally, there is thought to be bias from the use of email addresses as the singular method of contact with survey respondents. Choi
and Dinitto (2013) showed that 86% of older adults use the internet for email purposes. Therefore, older Veterans that lack access to the internet
cannot be reached by the survey, and it is plausible that such Veterans may be in a socio-demographic class with differing attitudes of the appeals
process. Moreover, only 25% of VHA beneficiaries with a valid email address elect to share it for future contact. In all, approximately 30% of all
appellants utilizing VHA services will be matched to a valid email address. It is possible that Veterans withholding their email address do not
share similar perspectives, which may also lead to coverage bias.
12
It is assumed that the coverage bias from these two sources is negligible, and will not significantly alter customer experience estimates. In
order to assess this impact VEO will plan to conduct a study to examine customer satisfaction differences between online surveys (w/ email
recruitment) and automated telephone surveys (w/ phone recruitment).
Appendix 1. List of Data Extraction Variables
Survey Person ID
Veteran First Name
Veteran Last Name
Appellant First Name
Appellant Last Name
Date of Birth
Certification Date
Form 9 Date
Disposition
Regional Office
Home Phone
SSOC
Hearing Type
Work Phone
Decision Date
Trigger Date
Appeal Date
Program Type
VLJ
Appeal ID
Representative
Period of Service
Survey Type
MVI
Age
Gender
Race
13
Appendix 2. Survey Questions
Summary
This copy document provides email and survey copy for 2 distinct surveys measuring customer experience during the Board of Veterans’ Appeal
process. These surveys are triggered within 1-4 weeks after the completion of the following events:
[8.1] Board of Veterans’ Appeals Hearing Experience Survey: Email survey sent after the hearing has occurred.
[8.2] Board of Veterans’ Appeals Appellant Experience Survey: Email survey sent after appellant process occurs.
Target Audiences
To be included in the survey sample, respondents must be:
Veterans who have gone through the Board of Veterans’ Appeals process.
Notes
Text with in the copy indicates a string from a database.
Text with [brackets] indicates notes, comments, options, dummy text or annotates browser behaviors. [or] separates conditionals when there are
choices. indicates data does not exist.
Margin comments call out details or issues that may require follow up or a discussion with a designer, developer, or writer.
Headings label and demarcate sections of the experiences and separate surveys.
the Board APPELLANT SURVEY
the Board Experience Survey Email Copy
[From:] Veterans Experience Office
[Subject 8.1] Board of Veterans’ Appeals Hearing Experience Survey – 2 minutes
[Subject 8.2] Board of Veterans’ Appeals Appellant Experience Survey – 2 minutes
[Header]
Help us serve your better
[Heading]
14
OMB Number: XXXX-XXXX
Expiration: MM/DD/YYYY
Estimated Burden: 2 minutes
Your feedback is important to us. Please take two minutes to let us know how we are doing by answering this short survey about
[8.1] < your recent experience with the Board of Veterans’ Appeals hearing process >.
[8.2] < your recent experience with the Board of Veterans’ Appeals appellant process>.
If you provide feedback, you may be contacted by VA. Serving you is our top priority.
[Take Survey]
Thank you,
Veterans Experience Office
Department of Veterans Affairs
[Footer]
If you wish to share your feedback, please do so by .
You received this email because you provided your email address to VA.
The Veterans Crisis Line provides free, confidential support for Veterans in crisis and their families and friends. Dial 1 (800) 273-8255 (Press 1),
or text 838255 to receive confidential support 24/7, (System of Records Notice VA158VA10NC5) Visit https://www.veteranscrisisline.net/ for
more information.
The National Call Center for Homeless Veterans (NCCHV) provides free, confidential support for Veterans who are homeless or at risk of
homelessness—and their family members, friends and supporters. Veterans can make the call to or chat online with the National Call Center for
Homeless Veterans, where trained counselors are ready to talk confidentially 24 hours a day, 7 days a week. Dial 1 (877) 424-3838 or visit
https://www.va.gov/HOMELESS/ to receive confidential support. If you would like to opt out from receiving future surveys, please click here.
Unsubscribe from this VA Survey | VA Privacy Policy
Department of Veterans Affairs
Veterans Experience Office (30)
810 Vermont Avenue NW
15
Washington, DC 20420
the Board Experience Survey Reminder Email Copy
[SENT 1 WEEK AFTER INITIAL EMAIL IF THERE IS NO RESPONSE]
[From:] Veterans Experience Office
[Subject 8.1] Board of Veterans’ Appeals Hearing Experience Survey Reminder – 2 minutes
[Subject 8.2] Board of Veterans’ Appeals Appellant Experience Survey Reminder – 2 minutes
the Board Experience Survey Reminder Pre-Header
[8.1] < your recent experience with the Board of Veterans’ Appeals hearing process >.
[8.2] < your recent experience with the Board of Veterans’ Appeals appellant process>.
[Header]
Help us serve your better
[Heading]
OMB Number: XXXX-XXXX
Expiration: MM/DD/YYYY
Estimated Burden: 2 minutes
We are waiting for your response about your experience with the Board of Veterans’ Appeals. Your feedback is important for identifying areas
that need improvement.
Please take two minutes to let us know how we are doing by answering this short survey about
[8.1] < your recent experience with the Board of Veterans’ Appeals hearing process >.
[8.2] < your recent experience with the Board of Veterans’ appellant process >.
If you provide feedback, you may be contacted by VA. Serving you is our top priority.
[Take Survey]
Thank you,
16
Veterans Experience Office
Department of Veterans Affairs
[Footer]
If you wish to share your feedback, please do so by .
You received this email because you provided your email address to VA.
The Veterans Crisis Line provides free, confidential support for Veterans in crisis and their families and friends. Dial 1 (800) 273-8255 (Press 1),
or text 838255 to receive confidential support 24/7, (System of Records Notice VA158VA10NC5) Visit https://www.veteranscrisisline.net/ for
more information.
The National Call Center for Homeless Veterans (NCCHV) provides free, confidential support for Veterans who are homeless or at risk of
homelessness—and their family members, friends and supporters. Veterans can make the call to or chat online with the National Call Center for
Homeless Veterans, where trained counselors are ready to talk confidentially 24 hours a day, 7 days a week. Dial 1 (877) 424-3838 or visit
https://www.va.gov/HOMELESS/ to receive confidential support. If you would like to opt out from receiving future surveys, please click here.
Unsubscribe from this VA Survey | VA Privacy Policy
Department of Veterans Affairs
Veterans Experience Office (30)
810 Vermont Avenue NW
Washington, DC 20420
the Board Services Survey Rating Scale Questions
The Veterans Crisis Line provides free, confidential support for Veterans in crisis and their families and friends. Dial 1 (800) 273-8255 (Press 1),
or text 838255 to receive confidential support 24/7. Visit veteranscrisisline.net for more information.
8.1 Board of Veterans’ Appeals Hearing Experience Survey
Tell us about your recent experience with the Board of Veterans’ Appeals hearing process.
1. Why did you request a Board of Veterans’ Appeals hearing? (Mark all that apply)
• I thought it would help the outcome of my appeal
17
•
•
•
•
•
•
•
•
I thought it was required
I wanted to speak to someone in person
It’s my right to have a Board of Veterans’ Appeals hearing
It was recommended by a friend, family member, or other Veteran
It was recommended by a Veteran Service Organization representative
It was recommended by a personal representative
Other (specify)
Don’t know or not sure
2.
•
•
•
•
Was your hearing held as initially scheduled? (Mark only one)
Yes (if selected, move to question #5)
No (if selected, move to question #5)
No, my hearing was cancelled at least once (if selected, move to question #3)
No, my hearing was rescheduled at least once (if selected, move to question #4)
3.
•
•
•
•
•
•
Why was your hearing cancelled? (Mark all that apply) (OPTIONAL)
I canceled the hearing based on the advice of my representative.
I was advised that I could submit a statement in writing instead of having the hearing.
I was unable to travel to the hearing location.
I felt like it was delaying my appeal decision.
My request to reschedule my hearing was denied.
None of the above.
4.
•
•
•
•
•
•
Why was your hearing rescheduled? (Mark all that apply) (OPTIONAL)
I requested a different type of hearing.
I had a conflict with my initially scheduled date.
My representative was unable to attend my initially scheduled date.
Illness, transportation, or weather issues on the day of the hearing.
I didn't receive adequate notice of the date/time that the hearing was scheduled.
None of the above.
5.
•
•
•
Please rate your overall experience in requesting a hearing
Very Negative
Negative
Average
18
•
•
Positive
Very Positive
6.
•
•
•
•
•
Please rate your Overall experience with the hearing staff.
Very Negative
Negative
Average
Positive
Very Positive
7.
•
•
•
•
•
Please rate your Overall experience with the Judge.
Very Negative
Negative
Average
Positive
Very Positive
8.
•
•
•
•
•
Please rate your overall experience with the Board of Veterans’ Appeals hearing process.
Very Negative
Negative
Average
Positive
Very Positive
8.2 Board of Veterans’ Appeals Appellant Experience Survey
The Veterans Crisis Line provides free, confidential support for Veterans in crisis and their families and friends. Dial 1 (800) 273-8255 (Press 1),
or text 838255 to receive confidential support 24/7, (System of Records Notice VA158VA10NC5) Visit https://www.veteranscrisisline.net/ for
more information.
19
Tell us about your recent experience regarding your Board of Veterans’ Appeals appellant process.
1.
•
•
•
Did you receive an initial letter from the Board of Veterans’ Appeals notifying you that your appeal was received by the Board?
Yes
No
Don’t know or not sure
2.
•
•
•
•
•
Thinking about the initial Board of Veterans’ Appeals notification letter, was the purpose of the letter clear and easy to understand?
Not at all clear
Somewhat clear
Completely clear
Don’t know or not sure
I did not read the letter
3.
•
•
•
•
•
Which of the following online resources do you use to check the status of your appeal?
eBenefits
Vets.gov
Inquiry submitted via the Board’s Customer Service website
Other
I do not use any online resources to check the status of my appeal
4.
•
•
•
•
•
•
Please rate your experience using the Veteran websites for appeal status. (OPTIONAL)
Very Negative
Negative
Average
Positive
Very Positive
Not Applicable
5. In the past 6 months (excluding any contact with your local VA field office), did you contact anyone at the Board of Veterans’ Appeals
about the appeal process?
• Yes
• No
20
6.
•
•
•
•
•
•
•
Which of the following best describes the reason for your most recent contact?
Request a status update
Ask a question
Request a change to your contact information
Provide additional evidence/arguments in support of your appeal
Resolve a concern
None of the above
Not applicable
7.
•
•
•
•
•
Please rate your experience regarding the communication and information about your appeal.
Very Negative
Negative
Average
Positive
Very Positive
8.
•
•
•
•
•
•
My most recent contact was resolved to my satisfaction?
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Not Applicable
9.
•
•
•
•
•
Please Rate your experience with the customer service provided by the Board of Veterans’ Appeals.
Very Negative
Negative
Average
Positive
Very Positive
10. The Board’s decision was clear and understandable (whether I agree with it or not).
• Strongly Disagree
• Disagree
21
•
•
•
Neutral
Agree
Strongly Agree
11.
•
•
•
•
•
Please Rate your overall experience with your appeal process
Very Negative
Negative
Average
Positive
Very Positive
❐ Please check this box if you would like to volunteer your demographic information to help VA better serve you, otherwise just click “Next” to
submit your survey.
[Next]
[OMB Burden Response Copy]
We are asking for this information so that you can provide compliments, recommendations, or concerns to VA. This information is collected in
accordance with section 3507 of the Paperwork Reduction Act of 1995. Title 38, United States Code, allows us to ask for this information. We
estimate that you will need an average of two minutes to review the instructions and complete this survey. The results of this survey will be used to
inform opportunities for program improvement in the quality of VA services. Participation in this survey is voluntary and your decision not to
respond will have no impact on VA benefits or services to which you may currently be receiving. By filling out this survey, you are authorizing
VA database access to retrieve veteran contact information, to follow up with you accordingly for purposes of service recovery, potential crisis, or
to learn more about feedback you have shared regarding your experience with VA. VA cannot conduct or sponsor a collection of information
unless a valid OMB control number is displayed. You are not required to respond to a collection of information if this number is not displayed.
Valid OMB control numbers can be located on the OMB Internet Page at www.reginfo.gov/public/do/PRAMain.
[Footer]
Privacy Policy
Race, Ethnicity, and Gender Questions
Help VA Improve its Services
We are working to better understand our customers. The following questions are voluntary. By providing your data, your responses can
22
help us improve VA care and services. Thank you for your participation.
1. Are you Hispanic or Latino?
• Yes
• No
2.
•
•
•
•
•
What is your race? Select one or more.
American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or Other Pacific Islander
White
3.
•
•
•
•
How do you describe your gender?
Male
Female
Non-Binary / Third Gender
Prefer not to say
[Submit]
Appendix 3. References
Choi, N.G. & Dinitto, D.M. (2013). Internet Use Among Older Adults: Association with Health Needs, Psychological Capital, and Social Capital.
Journal of Medical Internet Research, 15(5), e97
Kish, L. (1992). Weighting for unequal P. Journal of Official Statistics, 8(2), 183-200.
Lohr, S. (1999). Sampling: Design and Analysis (Ed.). Boston, MA: Cengage Learning.
Liu, J., Iannacchione, V., & Byron, M. (2002). Decomposing design effects for stratified sampling. Proceedings of the American Statistical
Association’s Section on Survey Research Methods.
23
| File Type | application/pdf |
| File Modified | 2020-08-27 |
| File Created | 2020-08-27 |