HSLS-09 HS Transcript and College Update FT Response to OMB passback

April 4, 2012

MEMORANDUM

To: Shelly Martinez, OMB

From: Laura LoGerfo, NCES

Through: Kashka Kubzdela, NCES

Re: Response to February 24, 2012, OMB Passback Meeting on HSLS:09 College Update and Transcript Field Test (2012) FT Package (OMB# 1850-0852 v.10)

This memorandum reviews changes to the HSLS:09 College Update and Transcript package from November 2011 and identifies materials that are being used to document changes in HSLS:09 College Update Supporting Statement Parts A and B and in Appendixes 1 and 2.

Changes to Part A of the Supporting Statement. Section A-9 has been amended to reflect a revised incentives plan, which in turn responds to OMB comments (February 24) on the earlier plan. The original plan was to offer $20 to a subset of difficult cases during nonresponse follow-up, payable to the student only (regardless of whether student or parent became the interviewee). The revised plan ties incentives to a Mahalanobis distance function which should reduce bias in the sample and is fully explained in A-9. Plans now include identifying a subset of nonresponding cases approximately 5 weeks after the start of data collection who are most unlike the set of respondents at the time. These targeted cases will receive a $5 pre-paid incentive with an offer of $10 more if they participate.

Changes to Part B of the Supporting Statement. Section B-3 has been amended to reflect the revised incentives plan in the discussion of methods for maximizing response rates.

Changes to Appendix 1 (communications/contact materials). Contact materials have been revised in two ways. First, references to 15 minutes for questionnaire completion have been changed to 20 minutes. During the instrument development stage, precise interview length was uncertain, and there were references to both 15 minutes and 20 minutes. Simulations with the near-final instrument confirm that the questionnaire will, on average, take about 20 minutes, and all references have been made consistent to reflect this fact.

Second, references in communications materials to incentive amount have been changed to reflect the revised incentives plan for those who are targeted for the incentive (i.e., target cases will receive a $5 pre-paid cash incentive in a reminder mailing, and the letter will promise that an additional $10 will be sent upon completion of the questionnaire).

Changes to Appendix 2 (questionnaire). The draft questionnaire has been revised. The changes respond to OMB’s comments (February 24) as well as reflect additional review within NCES. Issues addressed in the revision include aligning terminology with the typical vocabulary and depth of understanding of cohort members and ensuring that critical information will be collected from all respondents, not just college-going students. In addition, two tables are appended to this memo that summarize all changes made to the survey instruments: 1) a table of changes made to the questionnaire in response to OMB comments, and 2) a table that shows minor revisions which were implemented solely for consistency with the HSLS:09 first follow-up surveys.

HSLS:09 College Update Questionnaire Revisions

Changes in response to OMB comments

The table below reflects suggested changes to the HSLS:09 College Update field test questionnaire in response to OMB’s comments and additional review within NCES. These changes are tracked in the revised questionnaire.

"Original Page # or G" Column

• This reflects the page number and question number of the question in the original questionnaire.

• G indicates a general comment

• NA indicates that the question was not included in the submission.

• Line numbers refer to lines in the track changes version of the questionnaire.

"New Location" Column

• This reflects the new question number in the revised questionnaire.

“Comment” Column

• This column indicates how the question was revised and a rationale for the change.

• Comments with an “OMB” notation are NCES’ summaries of comments received from OMB.

• Comments with an “NCES” notation are NCES’ direct responses to OMB comments.

Original Page			New Location	Comment/Reply
# or G	Q#	Line
G				OMB: Inconsistent formatting is distracting NCES: Formatted consistently. Sub-items are lettered and response options are assigned a value.
G				Changed reference date from October 1st to November 1st, the day after data collection ends, so all respondents will be answering questions prospectively. Previously, the reference date was one month before the end of data collection so some respondents would have been answering retrospectively, potentially compromising comparability of the data.
2	Q5	62 - 75	Q4	GED recipients will be asked for the last high school attended. Now asked of all respondents, including those who are still enrolled in high school or who are homeschooled
2	Q6	81-82	Q5	GED recipients will be asked for the last high school attended rather than the high school from which they earned their GED
3	Q7	106	Q6	Removed the word “high” from “high school” to be sure teenagers report all types of secondary schools they have attended such as vocational-technical schools.
NA	NA	121-138	Q7	OMB: Need to add item on dual credits NCES: Added Q7 from HSLS:09 F1.
3	Q8	142 - 176	Q8	OMB: Wording discordant with understandings of nomenclature; item responses are too simple for the complexity of responses we expect NCES: Revised first option to “Taking classes at a college, university, or school that provides occupational training (e.g., cosmetology school or a school of culinary arts)” based on recommendation from Postsecondary, Adult, and Career Education Division (PACE) of NCES Added “ Studying for an industry certification or license (e.g., Microsoft Systems Engineer),” “Enrolling in an apprenticeship” and “ Another form of training (please specify)” to be more inclusive of occupational training “Pursuing other interests” omitted due to lack of policy relevance or analytic use.
3	Q9	178- 191	Q9	Revised question stem for ease of reading. Revised first response option to align with revised Q8 Added 3 options to align with revised Q8 “Pursuing other interests” omitted due to lack of policy relevance or analytic use.
4	Q10	213 - 223	Q10	OMB: There should be a response that allows for courses a student plans to take that are not necessarily in a program track. Add “community college courses to transfer to bachelor’s degree program.” Add “educational program resulting in a certificate.” NCES: Added “Associate’s degree program (usually a 2-year degree) with plans to transfer to a Bachelor’s degree program” Added “without plans to transfer to a Bachelor’s degree program” to the existing Associate’s degree option to better distinguish it from the new option. Added “No specific program, but [you plan/he plans/she plans] to take courses” Added “You don’t know” based on recommendation from PACE Added “(usually a 4-year degree)” to Bachelor’s option and “(usually a 2-year degree)” to Associate’s options based on PACE recommendation Revised “Occupational license or certification program” to “Certificate or diploma program from a school that provides occupational training (usually takes 2 years or less to complete, often leading to a license, such as cosmetology)” based on PACE recommendation Added “You don’t know” based on PACE recommendation
4	Q12	247-248	Q12	Revised terminology for school types as per recommendations of PACE team.
5	Q16	318-320	Q16	OMB: What about open enrollment? NCES: Added “Include schools that admit anyone who registers” to account for open enrollment institutions Revised terminology for school types as per recommendations of PACE team.
6	Q18	348 - 371	Q18	OMB: Ask only about 3 schools including the one the teenager plans to attend to reduce burden; Remove the word ‘etc’ NCES: This question now asks respondents for a maximum of 2 schools for a total of 3 schools (including the one the teenager plans to attend which is collected in Q12); in consultation with PACE opted to choose two institutions most seriously considered
6	Q19	373 – 387	NA	OMB: Ask only about 3 schools total NCES: Eliminated. No longer needed due to limiting Q18 to 2 schools
6	Q20	389 - 397	Q19	To simplify question stem removed “or admission status.” Added instruction to consider all schools regardless of admission status. Removed question about second choice school because it has limited analytic utility. Added “Q12 school” to list of choices
7	Q21	429- 416	Q20	OMB: Ask only about three schools total NCES: Now lists up to three schools to reduce burden
7	Q22	430-436	Q21	Removed question about second choice school because it has limited analytic utility. Added “Q12 school” to the list of choices for those who reported a school in Q12
7-9	Q23 - Q28	449 - 593	Q22-Q27	Asking about applications for and offers of financial aid is important for individuals who plan to be enrolled and who may not plan to be enrolled. Therefore, revised so that all respondents except those who plan to be in high school or in GED completion course are eligible for this series of questions on financial aid. Previously, those who did not apply to college and were not planning to continue education were routed around this series.
7/8	Q23/Q26	449 – 467 & 523-542	Q22/Q25	OMB: Want to know why students do not complete the FAFSA specifically. Not just why they do not apply for financial aid generally. NCES: Swapped Q23 for Q26 in response to OMB’s request to collect information about reasons why respondents do not complete FAFSA
8	Q26	449-455	Q22	OMB: Add “did not know what the FAFSA is”, put the full name in parentheses, and use FAFSA in the question stem. NCES: Added “(Free Application for Federal Student Aid)”. Added “You don’t know what a FAFSA is.” Removed “to apply for financial aid” from question stem to better test lack of knowledge of what FAFSA is.
8	Q24	469 – 478	Q23	OMB: Wants to know why students do not complete the FAFSA specifically. For field test, use ‘other specify.’ Add “you or your family did not know you could complete a FAFSA” NCES: Revised to refer specifically to FAFSA. Added 6th item: You or your family did not know you could complete a FAFSA. Added 8th item: “other reason (please specify).” 4th item: Revised for clarity. Wording from NPSAS:12 FT. 5th item: Revised for clarity. Wording from NPSAS:12 FT.
8	Q25	499-506	Q24	OMB: Wants to know why students do not complete the FAFSA specifically. For field test, use ‘other specify.’ NCES: Revised to refer to FAFSA aid. Added 6th item: “other reason (please specify)”
7	Q23	523-542	Q25	OMB: Typo: “if yes, skip question 26” NCES: replaced “skip” with “go to” Revised to refer to all other financial aid applications besides FAFSA Added “You don’t know” as a response option
9	Q27	545-561	Q26	OMB: What types of financial aid did they apply for? Pell Grants? Staffords? For field test, use ‘other specify.’ It is not clear if we are fully specifying ‘private loans’ that for-profit institutions accept. NCES: Added Stafford Loan as separate line and added “including private loans” to the existing loan item. Added Pell Grant and revised existing item to “Any other scholarship or grant.” Added “(please specify)” to “other type of financial aid” item. Removed “tuition discount or waiver” because PACE indicated this would apply to too few respondents for analytic utility Added a “don’t know” option for each type of aid for each school to allow for uncertainty
9	Q28	578- 582	Q27	Replaced “church or temple” with “place of worship” for inclusivity Added a “don’t know” option
9	Q29	598-602	Q28	OMB: Do we want perceptions of cost? Or actual cost of attendance? Should students ready their award letter in preparation of response to this survey? NCES: Revised question wording to emphasize that estimates are acceptable. Reference to financial aid letter removed.
10	Q30	613-622	Q29	Question wording revised to reflect items being requested To reduce burden, removed “pay out-of-pocket” because it can be inferred. Removed “work-study” because this is akin to paying out-of-pocket. Removed “tuition waiver or discount” because PACE indicated this would apply to too few respondents for analytic utility
10	Q31	646-652	Q30	OMB: This item is too focused on elite’s reasons. Perhaps we simplify and broaden for the non-elites. NCES: Reduced burden by consolidating items. Added option for “transferability to bachelor’s program” for those not planning to go to 4-year institutions.
11	Q33	687 - 713	Q32	OMB: Full list is overly intrusive, too specific, and too much burden NCES: Reduced burden by consolidating items and eliminating those that were intrusive and had low prevalence. Added “because [you/teenager] didn’t receive enough financial aid?” due to policy relevance.
11-13	Q34 – Q41	730 - 891	Q33-Q42	Asking about employment status for all individuals, regardless of postsecondary enrollment is preferable, therefore now all respondents who plan to be working or apprenticing in the fall are eligible for this series of questions about employment. Previously, those who were pursuing postsecondary education, attending high school, or completing a GED were routed around this series.
11	Q34	732-733	Q33	OMB: Add apprenticeship. Add occupational licensing NCES: Added instruction to include apprenticeships
NA	NA	780-793	Q36	OMB: Add apprenticeship. NCES: Added this question to identify apprenticeships.
NA	NA	795-807	Q37	OMB: Add occupational licensing NCES: Added this question to identify those working towards licensure.
12	Q38	828-834	Q39	To reduce burden, eliminated “family”, “friends”, “read an ad sign, or notice” and “other” options due to lack of policy relevance.
13	Q40	863-873	Q41	To reduce burden, question no longer applies to all respondents. No longer asked of those who indicated earlier in the questionnaire that they would not be working.
13	Q42	898-909	Q43	OMB: Streamline this to the following three main ideas: College admission, college financial aid, employment. NCES: Reduced items to: 1) “To gain admission to a college or university,” 2) “To apply for financial aid,” and 3) “To find a job”

HSLS:09 College Update Questionnaire Revisions

Minor changes

The table below reflects minor revisions made to the College Update questionnaire to bring it into line with the HSLS:09 first follow-up questionnaire where questions overlap. These changes are incorporated into the revised questionnaire, but not tracked for ease of review.

"Original Page # or G" Column

• This reflects the page number and question number of the question in the original questionnaire.

"New Location" Column

• This reflects the new question number in the revised questionnaire.

“Comment” Column

• This indicates how the question was revised and a rationale for the change.

Original Page		New Location	Comment/Reply
# or G	Q#
1	Q1- Q2	Q1	“Earned” replaced with “received” in question stem. Combined original Q1 and Q2 for efficiency. Response options now specify type of credential earned. “Diploma” qualified by “regular” for clarity. “Other equivalency certificate” and “certificate of attendance” combined and replaced with “other high school credential” due to low prevalence of certificate of attendance.
2	Q4	Q3	Credential wording aligned with new Q1
3	Q7	Q6	Schools already known are listed on screen for reference. Question wording now specifies “since you were a 9th grader in the fall of 2008.”
3	Q8	Q8	“Plan” changed to “expect” in question stem. 9th item: “Attending GED preparation or other high school equivalency preparation course” simplified to “Attending GED completion course.” “Don’t know” added as a response option for each item to measure uncertainty.
3	Q9	Q9	“MAIN activity” changed to “main focus” in question stem. Response options aligned with new Q8.
4	Q10	Q10	“Be attending” changed to “enroll in” in question stem.
4	Q11	Q11	Replaced “attend this program” with “enroll” in question stem. Added “don’t know” response option to measure uncertainty.
4	Q12	Q12	Question revised to fill school type selected in new Q10.
5	Q14	Q14	“Plan” replaced with “expect.” “Don’t know” added as a response option to measure uncertainty.
5	Q15	Q15	“Don’t know” added as a response option to measure uncertainty.
8	Q24	Q23	2nd item: added “or college” 3rd item: added “or may not qualify” 6th item: added “after high school”
8	Q25	Q24	Revised “be ineligible” to “not qualify” in question stem 5th item: added “You will attend school or college part-time”
8	Q26	Q22	Revised “fill out” to “complete.”
10	Q31	Q30	8th item: Simplified “[Your parents/you] or a family member went there to “A family member went there.”
11	Q34	Q33	Replaced “have a job” with “working for pay, not counting work around the house” for clarity.
12	Q36	Q35	Revised from “related to your career goals” to “the job you want to have when you have completed your education” to be inclusive of those who do not think in terms of a career as per recommendations of PACE team.
13	Q39	Q40	Removed “Every two weeks/twice a month” to streamline set of response options.

0