SUPPORTING STATEMENT B
uSPEQ® CONSUMER EXPERIENCE SURVEY (REHABILITATION)
VA FORM 10-0467
OMB 2900-0752
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
Statistical methods used in this data collection are provided by the contractor in aggregating, summarizing, reporting, and benchmarking the results to local, regional, or national rehabilitation programs. uSPEQ Consumer Experience Survey (CES) was developed by experienced researchers. uSPEQ CES has been field tested in diverse health and human service organizations varying in size, type, and geographical location. Ongoing testing and analysis by the Commission on Accreditation of Rehabilitation Facilities (CARF) is employed to validate the crosscutting nature of the questionnaire, confirm the psychometric properties, and further refine the questionnaire. For more information about the psychometric properties of the survey instrument, please see the attached uSPEQ Consumer Experience Survey – Psychometric Evaluation technical report.
1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each strata. If this has been conducted previously include actual response rates achieved.
A potential respondent universe of 384,000 individuals annually is estimated. Many individual local rehabilitation programs will have a small number of annual participants/respondents. In order to provide useful and accurate information about each program, a census survey of all veterans in each program is required. The Commission on Accreditation of Rehabilitation Facilities (CARF) 2009 standards pertaining to Information Measurement and Management emphasize the importance of census surveying in the following words. Important words have been underlined for emphasis in the last two bullets below. In populations similar to those served by VA, there is a 67% survey completion and return rate of the uSPEQ consumer experience questionnaire.
“2.c.
Completeness. The organization takes steps to ensure that
the data used for decision making are as complete as possible; no
accredited services are omitted from the information and performance
improvement effort; no groups of persons served are omitted from the
data gathering or analysis; no data elements or indicators are
systematically missing; and any database is checked for completeness
of records before final analyses are run and decisions made. For
example:
The quality council and data manager collaborate on designing an information system regarding the persons served that includes necessary data elements for all services of the organization. They decide to design an organization wide system but identify each record with the particular service in which the person participates so that analysis can be done separately for all the services to be surveyed.
Staff training for the data-recording activities includes attention to the importance of recording each data field for every person served.
The data manager routinely cross checks the number of records in the database with the operations officer’s report of the number of persons served during a reporting period to ensure that data are available on all persons served before analysis is conducted and reports are generated. Missing records are located and entered into the database before analysis is conducted.”
2. Describe the procedures for the collection of information, including:
Statistical methodology for stratification and sample selection
Estimation procedure
Degree of accuracy needed
Unusual problems requiring specialized sampling procedures
Any use of less frequent than annual data collection to reduce burden
As noted above, in order to meet accreditation requirements for census sampling and to provide useful and accurate information about each program, a census survey of all Veterans in each program is required. Many individual local rehabilitation programs will have a small number of participants/respondents annually.
Many individual rehabilitation program sites will have less than one hundred participants/respondents annually. In order to reliably and validly detect bi-directional medium effect sizes with statistical power of at least .80 at the α=.05 level, it is necessary to have a sample size of at least 64 respondents (Cohen, 1988). As noted previously, in populations similar to those served by VA, there is a 67% survey completion and return rate of the uSPEQ consumer experience survey. For small rehabilitation programs, a marginally adequate sample size will be available to provide meaningful interpretation of results.
Adequate sample sizes will also be needed to ensure adequate detection and interpretation of bi-directional moderate effect sizes of each of the uSPEQ five component factors. It is essential to have adequate sample sizes to maintain within person reliability coefficients in the .80s and within-item reliability coefficients in the .90s.
A census survey of all veterans in each program is required for conformance with accreditation requirements and to provide useful and accurate information about each program, especially small local rehabilitation programs.
For many participants/respondents this uSPEQ survey of consumer experiences will be a one-time survey of perceptions following a traumatic event such as onset of paralysis, loss of vision, amputation, stroke, or an episode of homelessness or unemployment. Although sampling parameters have been described on an annual basis, for many of these individuals with severe injury or serious illness, this information will be collected only once or twice during a lifetime rather than on a recurring basis.
3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be approved if they call for answers to identical questions of 10 or more individuals.
The opportunity to complete the survey in a modality that is most comfortable for the participant/respondent maximizes response rates.
95% of surveys are completed on paper.
5% are completed by telephone follow-up.
Less than one percent are completed by internet or web.
Given that survey respondents/participants are likely to include individuals with cognitive difficulties due to trauma, strokes, and other etiologies, it is important to use an easily readable survey or questionnaire. An easily readable survey for people with a fourth grade education will maximize response rates and minimize respondent burden.
2.1 average sentences per paragraph
6.3 average words per sentence
4.0 average characters per word
3% passive sentences
79.5 Flesch reading ease
3.7 Fesch-Kincaid reading grade level
Some survey respondents are likely to have paralysis or impaired hand-function that would not allow them to complete a paper survey by themselves. In populations similar to those served by VA, 63.3% of the respondents reported completing the survey by themselves, 23.1% used assistance in completing the survey, and 13.6% had the survey completed on their behalf by a surrogate. This flexibility improves response rates and minimizes respondent burden.
uSPEQ Consumer Experience Survey (CES) was developed and refined by experienced researchers and has been field tested in diverse health and human service organizations varying in size, type, and geographical location. This extensive and comprehensive research has reduced an original pool of items from 80 potential items to 20 targeted and refined items that produce five component factors with minimal overlap and very good psychometric values consistent with item response theory and Rasch modeling.
4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.
uSPEQ has been extensively and comprehensively tested with populations similar to those served by VA by experienced researchers. It has been field tested in diverse non-VA health and human service organizations varying in size, type, and geographical location. The original uSPEQ pilot study included 1,698 respondents, while 2,439 respondents from seventeen separate organizations participated in the validation study. These populations included heterogeneous samples as categorized by age, ethnicity, education levels, and types of disabilities. Strong, confirmatory empirical evidence of reliability and validity also came from a subsequent independent survey sample of over 3,000 cases over a three-month period. Results have demonstrated strong reliability and validity indices for the whole instrument, individual items, and within component factor domains. The integrity of separation, reliability, and validity measure was maintained from longer to shorter versions and reliability at either the level of the person or item reliability.
The psychometric properties and research behind uSPEQ have been carefully reviewed as part of the OMB approval number for the Social Security Administration use of uSPEQ as OMB number 0960-0526.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other persons(s) who will actually collect and/or analyze the information for the agency.
Di Shen, Ph.D.
Chief Research Officer
CARF International
4891 East Grant Road
Tucson, Arizona 85712
Toll-Free: (888) 877-3788
Phone (520) 325-1044 Ext. 140
Fax (520) 318-1129
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | vhapughendrr |
| File Modified | 0000-00-00 |
| File Created | 2021-11-02 |