Request for approval under the "Generic Clearance for the Collection of Routine Customer Feedback"

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 2900-0770)

TITLE OF INFORMATION COLLECTION:

2014 Post-9/11 GI Bill Communications Assessment

PURPOSE:

The purpose of this information collection is to assess and improve outreach and communications to veterans and other key stakeholders.

The objectives are to:

• gather perceptions on key areas of concern regarding communication of the Post-9/11 GI Bill benefits to Veterans and other stakeholders,

• assess the effectiveness of VA Education Service’s current methods of communication about the Post-9/11 GI Bill as well as other education programs and initiatives, and

• identify additional methods that will improve outreach and communication efforts and activities for the Post-9/11 GI Bill initiative.

DESCRIPTION OF RESPONDENTS:

The potential groups of respondents include student veterans at 15 preselected colleges and universities with significant numbers of student veterans. Other potential respondents will include individuals from:

• Veteran service organizations (American Legion, Iraq/Afghanistan Veterans of America, Veterans of Foreign Wars)

• Student Veterans of America,

• Tragedy Assistance Program for Survivors (TAPS)

• National Association of State Approving Agencies (NASAA)

• Individual State Approving Agencies

• National Association of Veteran Program Administrators (NAVPA)

• Veterans Affairs Committee on Education (Service members Opportunity Consortium)

See Attachment: Post-9/11 GI Bill Communication Assessment for more details

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [x] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[ ] Focus Group [ ] Other: __________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: ________________________________________________

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ ] Yes [X] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No (Not applicable)

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No (Not applicable)

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [x] No

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
External Stakeholders	100	45 20 minutes	75 33 hours
Totals

FEDERAL COST: The cost has been included in a contract, which will assist with this study.

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe?

[X] Yes [ ] No

*A copy of the sampling plan is attached. See Attachment: Post-9/11 GI Bill Communication Assessment for more details

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[ ] Web-based or other forms of Social Media

[x] Telephone

[x] In-person

[ ] Mail

[ ] Other, Explain

2. Will interviewers or facilitators be used? [X] Yes [ ] No

Attachments:

1. Stakeholder Interview Questionnaire

2. Post-9/11 GI Bill Communication Assessment – Proposed Plan, dated: February 24, 2014

1. Stakeholder Interview Questionnaire

Thank you for agreeing to participate in stakeholder interviews conducted by the MITRE Corporation on behalf of the United States Department of Veterans Affairs’ Education Service. The purpose of these interviews is to assess and improve outreach and communications to veterans and other key stakeholders.

Our objectives are to:

• Gather your perceptions on key areas of concern regarding communication of the Post-9/11 GI Bill benefits to Veterans and other stakeholders.

• Assess the effectiveness of Education Service’s current methods of communicating to you about the Post-9/11 GI Bill as well as other education programs and initiatives. In addition, we want to identify additional methods that could improve outreach efforts.

Please rate the degree to which you agree with the following statements, on a scale of 1 to 5, with 1 being “strongly disagree” and 5 being “strongly agree.”

1. Communications from VA about the Post-9/11 GI Bill have improved since 2012 [or since we last spoke with you in ____ FOR PREVIOUS INTERVIEWEES ONLY].

2. Communications from VA help veterans better understand the Post-9/11 GI Bill.

3. Communications from VA about the Post-9/11 GI Bill are accurate.

4. Communications from VA about the Post-9/11 GI Bill are timely.

5. Communications from VA about the Post-9/11 GI Bill are sufficient in helping you provide information to Veterans.
   
   

Today, we would like to better understand the current level and nature of your concerns about issues related to the Post-9/11 GI Bill.

For these issues (and related questions), we’re interested in open-ended responses. Please share your thoughts or impressions about what you have experienced with the VA’s outreach and communications to Veterans and other key stakeholders concerning the Post-9/11 GI Bill.

Communications:

• Timeliness/accuracy of information

• Effective outreach to veterans

• Effective outreach to other stakeholders

Providing Guidance to Veterans:

• GI Bill complexity (makes the decision process difficult)

• Inaccurate or inconsistent information (could result in poor choice of programs)

• Irrevocability of choice (no going back if a poor choice is made)

Payment Issues:

• Delay in payments to schools and/or veterans

• Overpayments charged to veterans

• Potential for fraud by schools/educational institutions

1. Which VA communication resources do you find most useful (e.g., the VA GI Bill website; VA GI Bill Facebook page; VA blogs; conference calls; brochures, letters, and other mailings; presentations at conferences / meetings; webinars)?

   1. VA recently redesigned the GI Bill website. Have you noticed any changes that you enjoy? Any additional suggestions for VA about the website?

2. How would you characterize your understanding of the GI Bill enhancements resulting from the President’s Principles of Excellence? These include the new school comparison tool and the feedback (or complaint) system featured on the GI Bill website.

1. What have been your primary sources of information about the comparison tool and feedback system?

2. What communications, if any, have you received from VA about these enhancements? Do you have suggestions for how VA can improve its communications about the comparison tool and feedback system?

3. What have you heard about the helpfulness of these enhancements? Any suggestions for improvements?

4. What have you heard, if anything, about how schools are adhering to the Principles of Excellence? What’s going well? What are the challenges? Do you think schools know about their role regarding the feedback system?

3. What is your perception of the role of State Approving Agencies (SAAs) with the Post-9/11 GI Bill? Do you think schools view the SAAs as intermediaries between themselves and VA?

1. VA Education Service launched quarterly webinars to share information about the Post 9/11 GI Bill and other education programs with a broad audience including SAAs, School Certifying Officials, VetSuccess on Campus Counselors, and more. Are you aware of these webinars? Any suggestions for improvements in the way VA communicates with SAAs?

1. Do you have suggestions for additional ways that VA could communicate with you about the Post-9/11 GI Bill or other education benefit programs?

5. What didn’t we ask you that we should have?

2. Post-9/11 GI Bill Communication Assessment – Proposed Plan, dated: February 24, 2014

Prepared for: The Department of Veterans Affairs Veterans Benefits Administration Education Service
Post-9/11 GI Bill Communication Assessment
Proposed Plan
February 24, 2014
Version 4.0
The views, opinions, and/or findings contained in this report are those of The MITRE Corporation and should not be construed as official government position, policy, or decision unless so designated by other documentation. This document was prepared for authorized distribution only. It has not been approved for public release. © 2014, The MITRE Corporation. All Rights Reserved.
MITRE
7515 Colshire Drive McLean, VA 22102

1. Purpose

As set forth in Task 4.1 of the Performance Work Plan, this assessment is part of MITRE’s organizational change and communication support to VA’s Education Service. The findings from this assessment will serve as the basis for calibrating current outreach and communication activities, and will provide recommendations for planning and implementing future outreach and communication activities for the GI Bill initiative.

2. Scope

MITRE will assess previous communication activities and conduct follow-up interviews with key stakeholders to assess performance and concerns relative to the continued implementation of the Automate GI Bill Initiative and related programs. MITRE will:

• Assess implementation and effectiveness of previous communication efforts

• Using previous stakeholder analyses as the baseline, interview key stakeholders to assess current level of concern relative to the key issues identified in 2012

• During interviews, elicit feedback on issues, concerns, problems with communications and their implementation. Elicit “what is going right” to identify potential best practices. Assess methods of delivery, messaging and effectiveness

• Provide recommendations for ongoing and new outreach and marketing activities, and for integration of communication best practices and tools into future plans

3. Approach and Schedule

The following individuals are identified as leads for this effort:

• VA: Robert Worley (RW), Brandye Terrell (BT), Bill Spruce (BS), Barrett Bogue (BB)

• MITRE: Martha Goldstein (MG), Jack Gribben (JG), Nicki Crane (NC), Theda Parrish (TAP), Tricia Lennon (TL)

Below is an outline of our recommended approach and schedule:

Steps	Tasks	Leads	Due Dates
General approach to assessment	Provide both qualitative and quantitative assessments: Update the 2012 protocol to re-interview key stakeholders, with focus on Veterans; use 2012 assessment feedback and previous stakeholder analyses as a baseline; evaluate implementation and effectiveness of previous communication activities and results; recommend a set of ongoing communication activities, as well as new activities to communicate about the GI Bill	Draft: MITRE JG/NC/TAP Review: MG	February 17 thru February 28 DONE
Determine delivery details	Report delivered on June 30, 2014 in Word and PowerPoint to: EDU Director, Brandye Terrell, Bill Spruce, and Barrett Bogue	MITRE MG	February 17 thru February 28 DONE
Define potential scope, draft outline for assessment, develop list of stakeholders to interview, develop draft interview protocols	How far back to assess? June 2012 What is included in assessment? Major communication initiatives such as Website redesign (reevaluate, with 2012 evaluation as baseline), social media, print, direct mail, and specific outreach initiatives Stakeholders? Focus groups and selected individuals for follow-up interviews and analysis (see Stakeholder Interview section for more detail)	Draft: MITRE JG/NC/TAP Review: MG	February 17 thru February 28 IN PROGRESS
Meet with customer to validate outline and proposed stakeholders; discuss other questions	Meeting with Brandye, Bill, and Barrett Monday, February 24 Feedback from EDU: Areas of Inquiry for External Stakeholders Vets: Level of knowledge about complaint system – do they know about it, would they use it, do they have complaints. Are schools adhering to POE and PL? What are perceived strengths and weaknesses of communications about POE and PL? Do schools know about their role with the complaint system? Do schools now see SAAs as intermediaries? Vets: Are more Vets using Facebook, other social media as an information source? Is the GI Bill website better / easier to use post-redesign? Perceptions of quarterly webinars with SAAs (also includes ELRs, SAAs, CELOs). Areas of Inquiry for Internal Stakeholders Field: Are we successfully disseminating information through the following flow: Field Office>ELR>SAA>Schools?	MITRE MG JG/NC/TAP Review/ Approve: EDU BT/BS/BB	February 17 thru February 28 DONE
Conduct assessment	Review documents Schedule and conduct interviews Report weekly on status to EDU and MITRE leads	MITRE JG/NC/TAP	March 3 thru April 16
Analyze data and draft preliminary recommendations	Continue with interviews and focus groups, and begin analysis of initial data collected Validate preliminary findings with EDU and MITRE leads Prepare first draft of assessment	Draft: MITRE JG/NC/TAP Review: EDU BT/BS/BB	April 16 thru May 21
Deliver first draft of assessment for review	Review of first draft by EDU leads	Review: EDU BT/BS/BB	May 21 thru June 4
Revise assessment based on feedback	Incorporate feedback from EDU leads Obtain final signoff from MITRE VA PM Finalize formatting and prepare electronic and print copies for delivery	MITRE JG/NC/TAP MG/TL	June 4 thru June 18
Deliver final draft of assessment	Revise as needed and deliver final assessment to COTR and EDU leads	MITRE JG/NC/TAP MG	June 30
Brief EDU Director	Conduct briefing to review findings with EDU Director	MITRE JG/NC/TAP MG EDU RW/BT/BS /BB	TBD

4. Review of Foundational Documents

MITRE will review documents, plans, surveys, and assessments that formed the basis for previous communication activities and will have an impact on future communication planning. These include, but are not limited to, those detailed in the following table.

Documents/Assessments to Review	Organization/Leads	Proposed Action
Feb.2009 Communication Strategy and Plan and subsequent updates (2012 is most recent update)	EDU/ Brandye Terrell, Barrett Bogue	Assess effectiveness of specific activities
Legislation and executive actions that have impacted the Post-9/11 GI Bill since the last assessment (e.g., Public Law 112-249; Principles of Excellence)	EDU/James Ruhlman	Identify areas to be addressed in communication planning
Regulations and Policies	EDU/James Ruhlman	Assess new regulations that require communication activity
VBA Call Center	EDU/Pam Stephens	Collect anecdotal information about most frequent questions and complaints regarding GI Bill
Surveys (Veterans, schools, other)	EDU/Brandye Terrell	Assess results and determine if follow-up surveys are feasible; potential coordination with PA&I
Website: Quantitative evaluation (user data, Web analytics) and qualitative evaluation (comments, feedback)	EDU/Barrett Bogue	Collect data and evaluate; leverage VA’s Google Analytics to assess overall usage. Consult data from Internet Inquiry System on FAQs, common inquiries.
Social Media (Facebook, Twitter, You Tube, VA Blogs): quantitative (number of users, comments) and qualitative (comments, feedback)	EDU/Barrett Bogue	Collect data and evaluate Interview VA social media leads
Print: Fact sheets, brochures, posters, briefings, giveaways, etc.	EDU/Barrett Bogue	Review for consistency, accuracy, determine need to update to reflect new legislation
Portable Media: DVDs, podcasts, Webinars, videos	EDU/TBD	Review usage, determine relative value and whether new versions are needed
Marketing Activities	EDU/ Brandye Terrell, Barrett Bogue	Review results; determine if marketing efforts should be included in future plans

5. Stakeholder Interviews

MITRE has conducted stakeholder interviews concerning Post-9/11 GI Bill communications on several occasions, including three rounds of interviews in 2009, and subsequent rounds in 2011 and 2012.

We recommend holding conversations with stakeholders included in the previous analyses to 1) assess effectiveness of previous communication activities in addressing their earlier concerns, and 2) identify ongoing and new concerns and risks. We also suggest using pre-existing relationships from previous assessments, where applicable.

Potential groups include (in descending order of priority):

Student Veterans

For the 2012 assessment, MITRE conducted focus groups at colleges and universities with significant numbers of student Veterans. MITRE ultimately convened focus groups with 15 colleges and universities to collect feedback from a broad sample of these institutions. We identified them by the following characteristics:

• School type (public, state, community college, and for-profit)

• Geography (Northeast, Mid-Atlantic, South, Southwest, West, and Midwest)

• School size (large: 20,000+, medium: 6,000–19,999, and small: <5,999)

• Veteran population (large: >1,000, medium: 300–999, small: <299)

• Yellow Ribbon Program

The following are colleges and universities we propose targeting in the 2014 assessment. We are proposing that one-third (5) of the sample be made of up of schools that have not participated in previous communication assessments. The new schools are highlighted in RED TEXT.

Student Veterans (con’t)

School	Type	Geography	School Size	Vet Pop Size	Yellow Ribbon
Brookdale Community College, NJ	Community	Northeast	M	S	No
Cameron University, OK	State	Midwest	S	M	Yes
Colorado State University – Ft. Collins	State	West	L	M	Yes
George Mason University, VA	State	Mid Atlantic	M	L	Yes
Kaplan University, CA	For Profit	All	L	L	No
Mississippi State University, MS	State	South	L	M	Yes
Northern Virginia Community College (NVCC), VA	Community	Mid Atlantic	L	M	No
Portland Community College, OR	Community	West	L	L	No
Santa Fe College, FL	Community	South	L	M	No
St. Louis Community College	Community	Midwest	L	S	No
Texas A&M	Public	Southwest	L	M	Yes
University of Nevada – Reno	State	West	M	S	No
University of Phoenix, AZ	For Profit	All	L	L	Yes
West Virginia University	Public	Mid Atlantic	L	M	Yes

Other External Stakeholders

Organization	Individual(s)
American Legion	Steve Gonzalez, Economic Division - Assistant Director
Iraq/Afghanistan Veterans of America	Tom Tarantino, Chief Policy Officer
Student Veterans of America	D. Wayne Robinson, President Darren Phelps, Program Coordinator Kelsey Hill, Communications and Outreach Coordinator
Veterans of Foreign Wars	Ryan Gallucci, Legislative Service Deputy Director
Tragedy Assistance Program for Survivors (TAPS)	Ashlynne Haycock, Education Support Specialist
American Council on Education (ACE)	Tanya Ang, Associate Director of Veterans Programs
National Association of State Approving Agencies (NASAA)	Joe Wescott, President
Individual SAAs	Individual TBD (Note: Janice to help identify) Individual TBD (Note: Janice to help identify) FYI. MITRE spoke with the following SAAs in 2012 regarding outreach on VRAP: Texas: Connie Jacksits, Rufus Coburn Utah: Bernie Davis Virginia: Annie Walker Washington: Michael Ball
National Association of Veteran Program Administrators (NAVPA)	R.K. Williams, President
Veterans Affairs Committee on Education (VACOE) Service-members Opportunity Colleges (SOC) Consortium	Kathy Snead, President and Director
Department of Defense	MAJ Justin M. DeVantier, Assistant Director, Accession Policy
Department of Education	Marc Cole, Department of Education
Consumer Financial Protection Bureau	Patrick Campbell, [Title]
Congress - House	Individual TBD (Majority) Individual TBD (Minority)
Congress - Senate	Individual TBD (Majority) Individual TBD (Minority)

Internal Stakeholders

Organization	Individual(s)
RPO – Muskogee	Phyllis Curtis, Education Officer
RPO – St. Louis	Louise Wright, Education Officer
RPO – Buffalo	Kim Wagner, Education Officer
RPO – Atlanta	Angela Seelhammer, Education Officer
VA Office of Public and Intergovernmental Affairs (OPIA)	Steve Westerfeld, Communications Director Terry Jemison, Public Affairs Specialist Tim Hudak, Staff Writer, Digital Engagement Team Bronwyn Emmet, Public Affairs Specialist
VBA – Benefits Assistance Service (BAS)	Mike Carr, Assistant Director for Social Media and Web Jennifer Rudisill, [Title]
VBA – Corporate Communications	Pat Mackin, Director of Corporate Communications
VBA -- Office of Economic Opportunity (OEO)	Curtis Coy, Deputy Undersecretary for Economic Opportunity
Education Service	Robert Worley, EDU Director Charmain English, EDU Deputy Director for Operations

Appendix: List of Previous Interviews (2012, 2011, 2009)

The tables in this section provide detail on the organizations and individuals that have participated in stakeholder interviews since 2009. A few items of note:

• Due to time constraints and other considerations, MITRE did not interview internal stakeholders (e.g., VACO, Education Service staff, Regional Processing Offices) for the 2011 assessment.

• The most recent round of interviews was completed in spring 2012

External Stakeholders – Student Veterans

2012	2011	2009
Baylor University, TX Boise State University, ID Brookdale Community College, NJ Cameron University, OK Des Moines Community College, IA Embry-Riddle Aeronautical University, AZ George Mason University, VA Kaplan University, CA Mississippi State University, MS Northern Virginia Community College (NVCC), VA Portland Community College, OR Santa Fe College, FL Towson University, MD University of Phoenix, AZ University of Washington-Seattle, WA	Boise State Univ., ID Cameron Univ., OK El Paso Community College, TX Florida State College , FL George Mason Univ., VA Mississippi State Univ., MS Northern Virginia Cty. College, VA Ramapo College, NJ San Diego Cty. College, CA Santa Fe College, Gainesville, FL Thomas Nelson Cty. College, VA Towson Univ., MD Univ. of Maryland, Univ. Col., MD	George Washington University, DC University of Maryland, MD

External Stakeholders – Veteran Service Organizations

Organization	2012	2011	2009
American Legion	Steve Gonzalez, Director of Education and Certifications	Joe Sharpe, Economic Division Director	Joe Sharpe, Economic Division Director Mark Walters
Iraq/Afghanistan Veterans of America	Tom Tarantino, Legislative Director	Tom Tarantino, Legislative Director	Patrick Campbell
Student Veterans of America	Michael Dakduk, Executive Director	Michael Dakduk, Executive Director	N/A
Veterans of Foreign Wars	Ryan Gallucci, Legislative Service Deputy Director Mark Marth, Service Officer (Virginia)	Shane Barker, Senior Legislative Assoc. Toby Beanblossom, VFW Department of Michigan, Ast. Dept. Service Officer	Toby Beanblossom, VFW Department of Michigan, Ast. Dept. Service Officer Eric Hillerman, Deputy Director for Legislative Affairs

External Stakeholders –Educational and Other Key Organizations

Organization	2012	2011	2009
American Council on Education (ACE)	Meg Krause, Assistant VP – Lifelong Learning	Meg Krause, Asst. VP, Lifelong Learning
National Association of State Approving Agencies (NASAA)	Joe Wescott, Incoming President Chad Schatz, President	Chad Schatz, President	Charles Rowe William Stephens
National Association of Veteran Program Administrators (NAVPA)	Dorothy Gilman, President	Dorothy Gillman, President; also Veterans Administrator, Ramapo College, NJ	RK Williams, President
Veterans Affairs Committee on Education (VACOE) Service-members Opportunity Colleges (SOC) Consortium	Kathy Snead, President and Director	Kathy Snead, VACOE Chair & Service-members Opportunity Colleges (SOC) Consortium President; SOC Director	James Bombard

External Stakeholders –Federal Agencies and Congress

Organization	2012	2011	2009
Department of Defense	Bob Clark, Deputy Under Secretary for Military Personnel Policy	Bob Clark, DoD Accessions Policy	Bob Clark, DoD Accessions Policy
Department of Labor	Christine Ollis, Employment and Training Administration	N/A	N/A
Department of Education	Karen Gross, Senior Policy Advisor	N/A	N/A
Congress - House	N/A	N/A	Michael Brinck, Veterans Affairs Subcommittee on Economic Opportunity (Republican Staff) Juan Lara, Veterans Affairs Subcommittee on Economic Opportunity (Democratic Staff)
Congress - Senate	N/A	N/A	Babette Polzer, Veterans Affairs Committee Staff William Edwards, Office of Senator James Webb

Note: EDU considered including congressional staff in the 2011 and 2012 interview rounds, but decided against contacting them.

Internal Stakeholders –VACO and Education Service

Organization/Position	2012	2011	2009
Under Secretary for Benefits	N/A	N/A	Michael Walcoff (Acting)
Deputy Under Secretary for Benefits	N/A	N/A	Yes
Office of Resource Management (ORM)	N/A	N/A	Yes
Congressional & Public Affairs	N/A	N/A	Yes
Office of Business Process Integration (OBPI)	N/A	N/A	Dianne Thompson
VSO Liaison	Kevin Secor	N/A	N/A
EDU Staff	Janice Fisher (SAA Contract Lead) James Ruhlman (Training Team Lead)	N/A	Keith Wilson Brandye Terrell Alison Rosen Eric Patterson Rodney Alexander James Palanchar (Metrics)

Internal Stakeholders –Regional Processing Offices

Organization/Position	2012	2011	2009
Call Center Director	Pam Stephens	N/A	N/A
RPO – Muskogee	Phyllis Curits (Education Officer) Robyn Noles Gayle Baldwin Michael Marks (CELO)	N/A	Phyllis Curtis Sam Jarvis
RPO – St. Louis	Nick Mickens (Asst. Director) Louise Wright (Education Officer) Lynn Flint	N/A	Louise Wright Dave Unterwagner
RPO – Buffalo	Donna Terrell (Director)	N/A	Jerry Miller
RPO – Atlanta	Al Bocchicchio (Director)	N/A	Al Bocchicchio