Supporting Statement A 9-15-22

Office of Management and Budget (OMB)

Clearance Request

for the

Post-Separation Transition Assistance Program (PSTAP)

Assessment

Supporting Statement A

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

Preparing Transitioning Service members (TSMs) for civilian life is an important goal for the U.S. Department of Veterans Affairs (VA). This goal has been highlighted in a 2014 report to Congress by the General Accountability Office (GAO), “Better Understanding Needed to Enhance Services to Veterans Readjusting to Civilian Life” [GAO-14-676). VA’s progress toward meeting this imperative for Veterans’ readiness for civilian life includes expansions to the Transition Assistance Program (TAP) to meet long-term outcomes in a Memorandum of Understanding (MOU) (Transition Assistance Program for Servicemembers Transition from Active Duty MOU signed 16 Dec 2016). The Interagency group that carries out the performance management activities of this MOU is the Performance Management Workgroup (PMWG) comprised of the following federal partner agencies:

1. U.S. Department of Defense

2. U.S. Department of Veterans Affairs

3. U.S. Coast Guard (representing Department of Homeland Security)

4. U.S. Department of Labor

5. U.S. Department of Education

6. U.S. Office of Personnel and Management

7. Small Business Administration

Active duty and reserve TSMs being discharged from their respective branches of service are required to take part in either an online or in-person TAP curriculum. After separation, Veterans, their family members, and caregivers remain eligible to take TAP. This training is comprised of a comprehensive set of courses that focus on the many aspects of transitioning to civilian life. Additional separate modules and tracks such as Entrepreneurship, (e.g., “Boots to Business”), can be selected. Contracted Benefit Advisors conduct VA TAP briefings at transition sites located at military installations across the country as well as locations Outside Continental United States (OCONUS). These briefings provide information on the full range of VA’s benefits, including but not limited to, VA healthcare, disability compensation, education, home loan program, insurance, vocational rehabilitation, and burial benefits. These benefits are collectively administered and monitored within the VA.

In April of 2019, OMB approved the Post-Separation Transition Assistance Program (PSTAP) Cross Sectional Assessment (OMB Control Number 2900-0864). The first administration of the Cross-Sectional Survey occurred in June of 2019. In March of 2020, OMB approved the addition of the Longitudinal Survey. Both surveys were then administered in 2021. The next administration is planned for 2022 and expected in 2023. These two assessments are combined into the PSTAP Assessment Outcome Study.

The goal of the PSTAP Assessment is to gauge the impact of TAP on TSMs through a Cross-Sectional Survey and then continue to track those participants for several years to assess long-term Veteran outcomes through the Longitudinal Survey. Participants in the Longitudinal Survey are recruited via the Cross-Sectional Survey. In the first year of the longitudinal study, Veterans from the six-month, one-year, and three-year cohorts from the prior year’s Cross-Sectional Survey who opt in will be provided the opportunity to participate in the longitudinal survey. In future years, only the 6-month cohort respondents who opt in from the prior year’s Cross-Sectional Survey will be added to the longitudinal study. The purpose of this information collection request fulfills Executive Order 13571—Streamlining Service Delivery and Improving Customer Service. Without collecting satisfaction information, VA would be unable to monitor and improve its TAP delivery and implementation. Additional legislative guidance for this improvement to TAP falls under the Veterans Opportunity to Work (VOW) to Hire Heroes Act (November 21, 2011, Public Law 112-56, §221-225, 125 Stat. 715-718) and the National Defense Authorization Act (Fiscal Year 1991, P.L. 101-510 amended Title 10 of the U.S.C.), specifically Chapter 58, which authorized comprehensive transition assistance benefits and services for Servicemembers transitioning from active duty, as well as their family members and caregivers.

VA is benefitting from the direct objective and subjective feedback from Veterans on their long-term outcomes and how they relate to the TAP training they received. Specifically, the survey will be administered to gauge the long-term effectiveness of TAP by: (1) examining the relationship between attendance in TAP courses and the use of VA Benefits; (2) analyzing the effect of participation in TAP courses on the long-term outcomes of Veterans in the broad life domains of employment, education, health and social relationships, financial, social connectivity and overall satisfaction and well-being, and; (3) identifying areas of improvement for TAP and the broader transition process to guide training and/or operational activities aimed at enhancing the quality of service provided to transitioning service members, Veterans, their families, and caregivers.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

The stakeholders of the PSTAP Assessment are Veterans, the VA, congressional committees with oversight of Veterans' issues, the Office of Management and Budget, the interagency partners that liaise with Congress and GAO regarding military and Veteran concerns, as well as companies and service providers that address Veterans’ needs (e.g., housing, healthcare, etc.). The interagency partnership was established to coordinate the federal government’s transition programs that support Veterans and the PMWG is the primary subgroup that addresses performance management for TAP.

VA is the primary stakeholder of this study and currently has several staff and contracts in place to deliver and determine the effectiveness of TAP training. Findings and recommendations from the PSTAP Assessment will be used to make informed program-related decisions and facilitate Veterans' access to benefits as they transition into civilian life.

This information collection request (ICR) is being submitted to request an extension to the expiration date as well as incorporate minor updates to the Cross-Sectional and Longitudinal Survey. These surveys enable VA to understand and quantify the understanding Veterans have of their benefits and how attendance in TAP training can affect their long-term outcomes over time and will be combined with the Cross-Sectional Survey to provide a very comprehensive holistic profile of Veterans. Results provide VA critical inputs that can be used to assist decision-makers in formulating operational changes in the VA environment to ensure that Veterans are effectively served. Through four years of survey administration, several findings have informed VA on how to improve TAP.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The PSTAP Assessment is collected using multiple modes. All participants receive a letter requesting they complete the survey online (accessed at www.VA-PSTAPSurvey.org) using a personal ID. Additionally, all participants with email addresses on file with VA receive a personalized email and link to the surveys. Participants who enlist in the Longitudinal Study can enter or update their email addresses on an annual basis. Email reminders are also sent to nonrespondents periodically. When possible, the contractor will also use other electronic methods to contact Veterans if they have agreed to provide other contact information to the VA. These methods may include text messages and social media campaigns. Nonrespondents to these initial methods are sent a paper survey to complete and send their responses using a business reply mail envelope. The paper survey letter will also include log-in credentials for the web survey. Throughout the survey, several reminders are also sent via email or other electronic methods to nonrespondents.

The paper instrument meets OMB guidelines for increased automation as it is a scannable instrument that decreases data entry time and errors due to potential manual keystrokes.

The initial administration of the Cross-Sectional Survey provided preliminary data on the average response rate for our target audience. Given the low response rates to paper mailings from younger Veterans, email invitations were also sent for both the Cross-Sectional and Longitudinal Surveys in 2020. This method increased response rates for the Cross-Sectional by over 10 percent and will continue to ensure ample data is collected to draw actionable conclusions.

The contractor provides a 508 compliant accessible survey to ensure all participants can easily complete the survey. Additionally, the online survey has been optimized for mobile devices and can be accessed from any device with internet access. These methods reduce response burden and increase response rates which allows the contractor to conduct valid data analysis.

All survey procedures used for data collection are in accordance with the Government Paperwork Elimination Act (GPEA) requiring agencies to comply by October 21, 2003.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

This effort is a request for renewal of the surveys approved by OMB (OMB Number 2900-0864). This assessment will continue collecting the same or similar data from participants over time. Other than these surveys, there is no data currently being collected for use by the federal government to understand the retention of TAP training and readiness for civilian life by TSMs. Although there was a prior study that concluded in 2012, the TAP curriculum has since been greatly redesigned. Presently, OPA conducts an anonymous course assessment of TAP participants at the end of their training. While this data is useful for monitoring TAP training quality as well as participation rates, it neither determines which modules TSMs find useful once they have transitioned nor does it measure any domains of their lives related to their readiness to become a civilian.

There is a similar longitudinal survey that was initially designed to study post-traumatic stress disorder (PTSD). This study, the Veterans Metric Initiative (TVMI), led by health researchers and receiving private funding from the Henry M. Jackson Foundation, has an expanded scope including additional domains related to civilian life.

While the TVMI does share some features with the PSTAP Assessment, it does not capture any information pertaining to the TAP curriculum or long-term outcomes. In addition, it is not a federal study and was established through a public-private partnership. In an effort to make these studies as comparable as possible, during planning for this proposed study permission to use and adapt select batteries where possible for this study was sought and obtained from the TVMI instrument (as well as other validated instruments).

There are several minor additions to each survey this year to reflect some changes in TAP since beginning survey administration. These changes are highlighted in the attachments to this submission.

Finally, these questionnaires were shared with the PMWG and interagency partners (listed in Item B5). This group provided input to the survey to ensure there is no duplication. Given this working group’s expansive reach across so many federal agencies, it is highly unlikely that there is another federal study that duplicates the proposed information collection request.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

Not applicable. All data is collected from individuals, so no small businesses or other small entities are impacted by this information collection. Veteran-owned small businesses are not impacted because Veterans are being surveyed in their capacity as individuals.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

If VA is unable to proceed with the collection of the data, VA will not be responsive to the recommendation of GAO to Congress to prepare TSMs for transition to civilian life. VA will not have the benefit of receiving feedback on what is important to Servicemembers during TAP or how best to improve their service and how best to serve the needs of our Veterans as they transition into civilian life. The longitudinal collection of data will enable VA to track and document improvements or declines in TAP effectiveness over time. Absent a rigorous and statistically sound survey, VA will only have anecdotal information available to improve its programs. Understanding the long-term outcomes of TSMs is integral to continuing to build an effective and efficient transition program.

The design and administration of the PSTAP Assessment incorporates several best practices in survey research to minimize burden on respondents (see Item 3 in Supporting Statement B). There are currently no known technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

• Requiring respondents to report information to the agency more often than quarterly;

• Requiring respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it;

• Requiring respondents to submit more than an original and two copies of any document;

• Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• In connection with a statistical survey, that is designed to produce valid and reliable results that can be generalized to the universe of study;

• Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• That includes a pledge of confidentiality that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• Requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

The PSTAP Assessment is administered annually. Respondents for the Cross-Sectional Survey are TSMs and only respondents who agree to participate in the Longitudinal Survey, by selecting yes on the Cross-Sectional Survey, are included in the annual survey. There are no special circumstances that would require respondents to prepare or submit the documents outlined above or provide written responses in fewer than 30 days since the survey is voluntary. The surveys have been designed and carried out with appropriate scientific rigor and will continue to produce valid and reliable results that can be generalized to the universe of study.

No information will be collected in the PSTAP Assessment that will require respondents to keep records or produce written information. All survey questions rely on recall of the respondent about their experiences with TAP or responses to questions about their current life as a civilian.

All data collected in this study will be kept secure and confidential. Economic Systems Inc., along with its subcontractor, Westat, will employ all necessary data security policies and processes to ensure participant confidentiality and data safety. These contractors are well-versed in the data security requirements that need to be implemented. The contractors will also employ reporting guidelines and procedures to ensure that participants will not be identifiable in reports generated throughout the study.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of the instructions and record keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported. Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years – even if the collection of information is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

A 60-day notice was published in the Federal Register on June 28, 2022, (Volume 87, Number 123, pages 38458-38459) soliciting comments on VA’s intent to request Office of Management and Budget (OMB) approval for data collection on the PSTAP Longitudinal Study. There were no comments received during the 60-day period.

This PSTAP Longitudinal Assessment draws upon validated batteries of questions such as those from the TVMI, Short Scale for Measuring Loneliness in Large Surveys, Lubben Social Network Scale, and Personal Well-Being Inventory. Utilizing these questions helps to boost response rates by using questions that have previously been tested and fielded with success. Permission was sought and obtained by the authors of these studies to use for this survey by VA.

In addition to drawing upon existing research, VA is a member of the aforementioned PMWG. VA worked closely with this group to coordinate the development of this survey. This included briefings at various points about the status of survey development, allowing members to review and offer input about the instrument, and the group was also used to help identify pretesting candidates.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payments or gifts shall be provided to respondents. This is a public service and minimal burden; therefore, no special remuneration shall be offered.

10.Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

An assurance of strict confidentiality is provided in the introduction respondents receive with the survey on the top of page 1 (included below). Respondents are assured that answers given will be kept confidential under law and will be for statistical reporting purposes only. Respondents will also be given the opportunity to opt out of completing the voluntary survey during the introduction. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code). In addition, three phone numbers are provided to respondents in the event they are in crisis, have questions about the survey, or have questions about their VA benefits.

The Department of Veterans Affairs (VA) wants to better understand Servicemembers’ transition into civilian life following their military service. To accomplish this, we are requesting you complete the enclosed survey about your experience with the Transition Assistance Program (TAP). The survey asks about your employment, education, training, finances, health, and well-being. Your responses will be used to make improvements to help Veterans transition to civilian life after their service. This survey is being conducted by Economic Systems Inc., an independent consulting company, and their subcontractor, Westat.

Participation in this survey is voluntary. You may skip any question you do not wish to answer. Your decision on whether to participate will not in any way impact your current or future VA benefits. The survey takes 15-20 minutes to complete. All survey responses are, and will remain, confidential (protected by the Privacy Act of 1974, 5 U.S.C. §522a and 38 U.S.C. §5701) and will only be used for statistical purposes.

Your responses will be restricted to authorized researchers directly involved in survey data collection and analysis and are password protected. A few questions deal with information that could be viewed as sensitive. We have taken additional steps to ensure the safety and privacy of your data. Your responses will be combined with all other survey responses so that you cannot be identified in any report.

If you have already completed the survey online, please disregard this letter.

If you choose to participate, you can complete the survey either online or by paper.

• Online: https://va-pstapsurvey.org and enter your personal survey ID: [PIN]

• By paper: Complete enclosed survey and mail in the provided self-addressed, postage-paid envelope by [DATE]. Returning this survey indicates your consent to participate.

If the Veteran addressed herein passed away, we are sorry for your loss and sincerely apologize for the inconvenience. These invitations were sent using the most current information available. Please contact us at the email address below, so that we can update our records immediately.

If you require immediate assistance with a crisis, please call the VA Crisis Line: Dial 988 and Press 1. If you need assistance with any VA program or have general VA questions, please call the VA Assistance Line: 1-800-827-1000.

If you need assistance with this questionnaire or have questions about the study, please contact the survey support center at 1-855-252-5725 or by email at [email protected]. If you have questions about your rights and welfare as a research participant, please call the Westat Human Subjects Protections office at 1-888-920-7631. Please leave a message with your first name, the name of the research study that you are calling about (the PSTAP survey), and a phone number beginning with the area code. Someone will return your call as soon as possible.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The PSTAP Assessment contains minimal topics of a sensitive nature. The majority of the questions are attitudinal measures about TAP training and topics such as the Veteran’s current financial situation and health status are minimally invasive and discussed more below. The survey instrument does not contain questions that require more careful human subject review such as surveying groups that require special protections.

Mental and physical health are important long-term outcomes to measure for Veterans. While measuring these topics could raise issues related to Health Insurance Portability and Accountability Act (HIPAA) and other protections for respondents, the questions used do not ask about a specific health condition. This approach was intentional because it is not designed to measure the incidence or prevalence of any given condition but will instead measure if Veterans are experiencing any type of health condition.

Two vulnerable subgroups of Veterans that often overlap are those who are suffer from PTSD and/or homelessness. Given that these populations may be hard to survey, it is a given that there will be under-reporting among these groups. All surveys experience this same type of sampling bias and it is why different research techniques need to be employed when such subgroups studied.

In an abundance of caution, this survey does offer a crisis hotline (provided in Item 10) and again at the end of the survey. For respondents who are experiencing a crisis of any kind, this precaution is offered to help Veterans in distress.

12. Estimate of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

• If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 13.

Annually, three cohorts will receive the Cross-Sectional Survey. In 2023, the Longitudinal Survey will include 6 cohorts from previous Cross-Sectional Surveys. Each additional year of the study will add one additional cohort. The survey is currently estimated to take 18.5 minutes to complete per Veteran based on testing.

The total average annual burden of the PSTAP Assessment is estimated to be 5,954 hours across both surveys. The full description of this calculation is detailed in Item 1 of Supporting Statement B and summarized in this table below.

Average Annual Hourly Burden Calculation

Cross-Sectional Survey
	Cohorts 1-3		Cohorts 4-6		Cohorts 7-9		Minutes	Hourly Burden
	Retention Rate*	Responses	Retention Rate*	Responses	Retention Rate*	Responses
Year 1	n/a	15,250					18.5	4,702
Year 2	n/a		n/a	15,000			18.5	4,625
Year 3	n/a		n/a		n/a	14,750	18.5	4,548
					Average Annual CS Burden			4,625
					Total CS Respondents			45,000
Longitudinal Survey
	Cohorts 1-5		Cohort 6		Cohort 7		Minutes	Hourly Burden
	Retention Rate*	Responses	Retention Rate*	Responses	Retention Rate*	Responses
Year 1	n/a	3,800					18.5	1,172
Year 2	88%	3,344	n/a	950			18.5	1,324
Year 3	81%	3,078	88%	836	n/a	925	18.5	1,492
					Average Annual Long Burden			1,329
					Total Long Respondents			12,933
					Total Respondents			57,933
					Total Annual Respondents			19,311
					Total Annual Burden			5,954

* Retention rates are derived from the attrition rates reported in the previous three years of the PSTAP Assessment.

12. Provide an estimate for the total annual cost burden to respondents or recordkeepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

    • The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment, and record storage facilities.

    • If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain reasons for the variance. The cost of purchasing or contracting out information collection services should be part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with rulemaking containing the information collection, as appropriate.

    • Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.

14. Provide estimates of annualized costs to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The average annual total cost to the Federal Government is estimated at $418,834. The table below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to the Contractor and are included in the Contractor’s total cost.

Estimated Annual Cost to the Federal Government

Cost Item initial fielding	Hours	Cost
VA labor	600	$41,130
Contractor costs		$377,704
TOTAL	600	$ 418,834

The VA labor cost was estimated using a composite average salary and benefits figure of $68.55 per hour based on 2022 estimates. The amount paid to the Contractor for the survey effort includes as its major components, the survey of Veterans who have attended TAP for total cost of $377,704. These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for administration, administration of the instrument, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I.

There is no change. This is an additional collection under OMB Control Number 2900-0864.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The results from the PSTAP Assessment will continue to be published annually and made available to the public on the VA website (https://benefits.va.gov/TRANSITION/tap-assessments.asp). The results will also be available internally at VA and within the federal government to facilitate informed programmatic and policy decisions, including those made by Congress. The plans for developing these results are discussed below under two sections of tabulation and publication and a project timeline.

Tabulation and Publication

• Scan Survey and Monitor Response Rates

During the fielding period, VA’s Contractor will scan surveys and examine the data file continuously to ensure the accuracy of the data. During the scanning process, the Contractor will inspect and remove duplicates.

The Contractor will host a password-protected website (www.va-PSTAPsurvey.org) that will provide response rates from the Web-based and paper surveys. Throughout the fielding period, the Contractor will update data monthly at minimum and provide VA’s leadership with passwords to access the site at any time.

• Clean and Analyze Survey Data

The Contractor will clean raw data files by examining responses for inaccuracies and correcting inaccuracies prior to analyses (e.g., non-applicable responses to question items that the respondent should have skipped given the responses to prior questions, invalid responses, etc.). The Contractor will create an analysis file by combining and harmonizing data from the web survey and mail survey. The analysis file will include a flag to indicate the survey mode (web or mail).

Prior to analysis, the Contractor, in consultation with VA, will consider recoding categories and up-coding open-ended responses in to existing or new categories for reporting purposes. The Contractor will then analyze the data and create reports for VA. The analyses will provide an accessible summary of responses designed to inform VA’s leadership of the outcomes for TSMs.

The analyses will consist of frequencies, cross-tabulations, regressions, and variable means. The Contractor compares responses to those provided on previous surveys, and will create derived variables indicating change over time as appropriate. All statistics include the weighted standard error when applicable.

• Create Disposition Reports

Disposition reports that include total surveys mailed, number of completes, and the number and percent of refusals are provided by the Contractor on an electronic dashboard.

• Reporting

The results of the contractor’s analysis result in two reports covering the findings from the cross-sectional and longitudinal studies separately. Each report includes an executive summary of key findings along with more in-depth analysis of the survey data. For the longitudinal study, the focus is on identifying trends in long-term outcomes for Veterans of different groups based on their demographic characteristics, use of VA programs, attendance in TAP, and other factors. Each report also includes a methodology section that outlines the overall project and provide details on data collection, response rates, and analytical methodologies.

In addition to the two reports focusing on survey responses, a summary report is also developed annually. This report provides a high-level synopsis of the key findings of each report.

Project Timeline

The major activities for the PSTAP Longitudinal Assessment project are structured by task and are outlined below.

TASKS

Task 1: The Contractor will maintain a list of survey participants based on their voluntary agreement to be included in the longitudinal survey when completing the cross-sectional survey. The contractor will clean the list and request updated administrative data on an annual basis from VA.

Task 2: Respondents will initially be mailed a postcard inviting them to complete the survey online. This postcard shall be validated by VA Section 508 office and be in conformance with 508 standards. Additionally, emails will be sent to all participants who provided email addresses in the cross-sectional survey. Those respondents who do not complete the survey online will be subsequently mailed a survey packet containing a cover letter, scannable paper survey, and postage-paid return envelope. The paper survey will be mailed roughly one month after the postcard.

Initial postcards shall provide respondents with the Internet address and location of the corresponding online survey (www.VA-PSTAPSurvey.org). Postcards shall also include the survey recipient ID number on the mailing label to allow for easy access to the survey. Email and/or other electronic method reminders will also be sent to nonrespondents periodically throughout the survey administration window.

All printing and mailing activities for fielding the surveys shall be conducted by the GPO Print Vendor (costs to be incurred by the government for this service are not available at this time). Printing costs will be included in the GPO awarded contract to the selected vendor. The Contractor shall work with VA and its approved GPO vendor for printing all materials associated with the surveys.

The Contractor shall receive all completed survey materials and paper-based responses for processing of data results, image/scan all returned survey instruments with quality assurance check (manual) for 10% of instruments returned with a maximum of 100,000 paper survey instruments will be sent out.

Task 3: The Contractor shall provide the live links to VA staff for certification that the questionnaires are live and accessible. The Contractor shall be required to host the website and to ensure it meets the VA data security requirements and is 508 compliant. The term "fieldwork" refers to the time period for which each survey is ‘open for response’. For the duration of the fieldwork portion of the surveys, VA requires 95% up-time for the site. Technical difficulties which could affect the access and use of the questionnaires shall be reported to VA staff on the same business day they are discovered. The Contractor shall be responsible for ensuring that 95% up-time requirements are met and for rectifying any technical or accessibility issues related to this task.

Task 4: For the duration of the fieldwork portion of the surveys, the Contractor shall provide both toll-free phone and Internet-based assistance to survey respondents who require it. The Web helpdesk shall be staffed so that all requests for assistance are addressed within one business day. The toll-free telephone lines shall also have voicemail capabilities so that inquiries can be addressed no more than 24 hours after receipt.

The Contractor shall log all calls and Web-based inquiries received and record the disposition of each. The Contractor shall provide VA with ongoing logs in Excel format of inquiries monthly throughout the duration of fieldwork. After the fieldwork portion of the surveys, the Contractor shall provide final logs to the TAP subject matter expert for review.

The Contractor shall also divide the log into two separate files. The first will be standard calls that do not require any VA follow-up (for example, those who have general questions about the survey or require assistance with the survey), and the second will be calls that require VA to follow-up directly with the respondent (for example, callers who request that VA call regarding their benefits).

Task 5: The Contractor shall provide VA with data matrices. The Contractor shall send an email to the Contracting Officer’s Representative each time data is loaded.

Task 6: After the Contractor creates the data matrices and processes the reports necessary for loading data to the PSTAP Reporting Site, the Contractor shall load the data to a reporting page for VA access. The Contractor shall send an email to the COR each time data is loaded.

Task 7: The Contractor shall also provide six (6) Reporting Site training sessions for VA personnel. During these sessions, the Contractor will provide an extensive overview of the reporting site, to include all pages available to VA personnel, as well as the various functions and features. Online training materials/resources will be available to users.

Task 8: The Contractor shall prepare presentations and provide briefings on data collected with VA TAP management annually.

These detailed presentations and briefings will be delivered in-person and/or via teleconference and will provide a summary of key findings and recommendations based on the data collected. The Contractor shall follow OMB guidelines for data analysis and reporting. The Contractor shall provide 20 printed versions of the findings and recommendations (overviews) for review during the meetings (40 Total).

Task 9: The Contractor shall prepare annual formal VA Executive Leadership briefings on data collected. These annual formal briefings will be delivered in-person to VA leadership to validate program objectives, data trends and recommendations across surveys based on the data collected. The Contractor shall provide VA with 25 color copies of the presentation as handouts for leadership at the meeting. The Contractor shall provide written summaries with all conclusions derived from the data. The written summaries/reports shall provide a high-level overview with key outcomes, recommendations, and best practices for the reporting period. The Contractor shall work with VA to ensure all edits are incorporated into the final presentations and written summaries.

Task 10: The Contractor shall use data trends and categorize the trends into suggested priority groupings. The Contractor shall review the suggested priority groupings with VA stakeholders to determine up to four relevant topics for participation in two action planning meetings annually. Action planning meetings shall be no longer than a half-day. The Contractor shall coordinate and facilitate an action planning meeting per topic to discuss causes, explore solutions, and determine actions, action owners and desired outcomes. The Contractor shall assist VA by providing senior-level guidance to action owners. Contractor shall then reconvene the stakeholders for each of the topic areas to discuss progress, challenges and future actions. The follow-up meetings shall be continued until action plans are finalized.

Task 11: The Contractor will process the new data collected and add to the data matrices. Further, the Contractor will program any corresponding questionnaire changes to the VOV Reporting Site on all applicable pages as well as update all materials for follow on survey collection.

Task 12: Federal agencies must design the survey to achieve the highest practical rates of response commensurate with the importance of survey uses, respondent burden, and data collection costs to ensure that survey results are representative of the target population so that they can be used with confidence to make informed decisions. Nonresponse bias analyses must be conducted when unit or item response rates or other factors suggest the potential for bias to occur. Per OMB’s Standards and Guidelines for Statistical Surveys (September 2006), a nonresponse bias evaluation is recommended if any surveys yield a response rate below 80%. The nonresponse bias evaluations will be conducted to ascertain the possible causes of variance in response rates among different respondent demographics and/or to determine the underlying cause of low response rates for the sample as a whole. The Contractor shall conduct such statistical analysis (e.g., weighting and imputation) in accordance with the guidelines outlined in the OMB Standards and Guidelines for Statistical Surveys (September 2006). The Contractor shall provide VA with a written report which discusses possibilities for survey sample biases based on the necessary OMB standards and guidelines for statistical surveys.

TIMELINE

The project timeline will be based largely on the date of approval for this OMB package. Below is a table of deliverables and projected dates.

Task	Start Date	End Date
Survey Administration	June 1, 2023	September 1, 2023
Data Cleaning	September 1, 2023	September 15, 2023
Data Weighting	September 18, 2023	August 22, 2023
Open-Ended Comments Report	September 18, 2023	September 29, 2023
Draft Report	October 2, 2023	November 30, 2023
Action Planning	November 1, 2023	November 30, 2023
Final Report	December 4, 2023	December 15, 2023

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

Not applicable. The OMB approval number will be displayed on each paper survey as well as web survey.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions requested under this clearance package.

Page 17 of 17