OMB83 Change Memo Oct 2013

HSLS-09 2013 Update FS Collection Period & Incentives Change Memo.docx

High School Longitudinal Study of 2009 (HSLS:09) High School Transcript and 2013 Update Full Scale Study and Panel Maintenance

OMB83 Change Memo Oct 2013

OMB: 1850-0852

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 1850-0852 can be found here:

Document [docx]

Download: docx | pdf

M e m o r a n d u m October 23, 2013

TO: Shelly Martinez, Office of Management and Budget

FROM: Elise Christopher, National Center for Education Statistics

THROUGH: Kashka Kubzdela, National Center for Education Statistics

RE: High School Longitudinal Study of 2009 (HSLS:09) 2013 Update Full Scale Collection – Extension of Collection Period and Incentive Treatment Change Request (OMB# 1850-0852 v.14)

The purpose of this change memo is to seek approval for several strategies to improve response and to further reduce the risk of nonresponse bias for the High School Longitudinal Study of 2009 (HSLS:09) 2013 Update full scale data collection, including extension of the collection period and increasing the current incentive treatment. Due to the partial government shutdown from October 1 through 16, 2013, and the associated shutdown of the systems that host the ED/IES websites, data collection was temporarily halted effective October 1, 2013. The HSLS:09 2013 Update data collection was scheduled to end on October 31, 2013. As of October 1, a total of 15,005 of the 23,415 sample members had responded (64 percent).

Original Responsive Design Approach

As a reminder, the following 6-phase responsive design strategy has been implemented for the HSLS:09 2013 Update full scale data collection, as detailed in Part B of the OMB package:

Phase 1 (June 1 to June 23): A three-week web collection period. At the start of the first phase of data collection, each of the parents and students in the 2013 Update sample received a letter asking them to log on to the web to complete the questionnaire. Students identified as ever dropping out of school were offered a $40 incentive for participating because this group has high analytical value and yet a small sample size, making high response rate particularly important. The extra incentive was found to be effective in ELS:2002 and is the same amount that was offered to this group for completing the HSLS:09 first follow-up survey.
Phase 2 (June 24 to July 28): A five-week web plus CATI data collection period. The addition in phase 2 is the CATI component. After the three-week web data collection period (phase 1), in addition to the web remaining an option, outbound calling to sample members commenced and continued for five weeks. The $40 incentive continued to be offered to students who had ever dropped out.
Phase 3 (July 29 to August 22): First stage of a targeted nonresponse follow-up period. After phase 1 and phase 2, predicted-likelihood values were determined for all nonrespondents (other than ever dropout cases). Cases with the highest predicted-likelihood values (i.e., those who may have the greatest influence on nonresponse bias) were identified for special intervention. These targeted cases were given a $5 prepaid incentive. Those cases who have ever dropped out also received the $5 prepaid incentive and continued to be offered $40 to complete the Update. All other cases received no incentive during that phase.
Phase 4 (August 23 to September 23): Second stage of a targeted nonresponse follow-up period. Four weeks after the start of phase 3, predicted-likelihood values were recalculated for remaining nonresponding cases. Phase 4 targeted cases were offered a $15 incentive conditional on completing the survey. Those cases who have ever dropped out continued to be offered $40 to complete the Update. All other cases received no incentive.
Phase 5 (September 24 to September 30; originally planned through October 9): Third stage of a targeted nonresponse follow-up period (we were in the midst of phase 5 when the partial shutdown occurred). Four weeks after the start of phase 4, predicted-likelihood values were recalculated again for remaining nonresponding cases. Phase 5 targeted cases have been offered a $25 incentive conditional on completing the Update. Nonresponding phase 4 targeted cases who are not targeted for phase 5 have continued to be offered $15 to complete the Update. Those cases who have ever dropped out have continued to be offered $40 to complete the Update. All other cases have received no incentive.
Phase 6 (originally planned for October 9 through October 31): all nonresponding sample members will be offered an abbreviated questionnaire that can be completed in 5 minutes. The offered incentive, if any, as determined in previous phase(s), will apply for this phase as well.

For cases eligible for an incentive, student respondents receive the incentive upon completion of the questionnaire. Parent respondents are presented with the option of receiving the incentive money or having the check made payable to the student sample member. The response rates to date by phase and type of case are shown in Exhibit 1.

The responsive design approach has provided some encouraging results with respect to participation so far. The multiple interventions for the ever-dropout cases have been particularly effective. This group has high analytic importance, but has historically been less likely to participate in longitudinal follow-ups, yet the response rate to date is 71 percent, compared with the 64 percent response rate overall and the 37 percent response for other targeted cases. The ever-dropout cases were offered a $40 incentive to complete the questionnaire from the start of data collection and also received a $5 prepaid incentive at the start of phase 3.

The interventions in phases 3, 4, and 5 have been helpful to encourage response among the cases with the highest predicted-likelihood to contribute to nonresponse bias unless interviewed, though it is important to gain participation from even more of these cases. The use of the incentive-based interventions seems to encourage participation by student sample members: 63 percent of the ever-dropout respondents are students, 61 percent of other targeted respondents are students, and 48 percent of non-incentivized respondents are students. It is also worth noting that this design may be effective in focusing the use of incentives to those needing them: 48 percent of the sample has participated without any incentive. We would be happy to provide further details, if desired, regarding the implementation to date of our responsive design.

At the time of this memo, 8,410 cases have not responded and 2,093 of these cases have been finalized with a non-interview status, leaving 6,317 pending cases available to work with.

This data collection has experienced certain specific challenges. We have been unable to establish confirmed telephone contact with roughly 20 percent of the sample for reasons such as avoidance, unavailability, mobility, and change in phone numbers. Additionally, some sample members have declined to participate in HSLS:09 stating that this was a high school study and the sample member was no longer in high school, or that they do not want to participate now that the student sample member is at college.

The partial government shutdown has resulted in a break in the data collection with the potential for lost contact with sample members. The shutdown has also resulted in telephone interviewer staff attrition and the need for attrition training of interviewers to replace those that do not return. We suspect that the shutdown itself could be a source of frustration with the government on the part of sample members and have an impact on participation once data collection is resumed.

Exhibit 1: Response rates by phase and type of case.

Phase		# Cases	# Responses	% Responses	# Student	% Student	# Parent	% Parent
TOTAL	Total	23415	15005	64.08%	7755	51.68%	7250	48.32%
	Dropout	1974	1407	71.28%	889	63.18%	518	36.82%
	Targeted	6518	2380	36.51%	1462	61.43%	918	38.57%
	No Incentive	14923	11218	75.17%	5404	48.17%	5814	51.83%
1	Total	23415	3678	15.71%	1648	44.81%	2030	55.19%
	Dropout	1974	487	24.67%	299	61.40%	188	38.60%
	No Incentive	21441	3191	14.88%	1349	42.28%	1842	57.72%
2	Total	19737	6131	31.06%	3384	55.19%	2747	44.81%
	Dropout	1487	492	33.09%	314	63.82%	178	36.18%
	No Incentive	18250	5639	30.90%	3070	54.44%	2569	45.56%
3	Total	13606	2541	18.68%	1421	55.92%	1120	44.08%
$5 prepay	Dropout	995	278	27.94%	189	67.99%	89	32.01%
	Targeted ($5 prepay)	6379	1019	15.97%	648	63.59%	371	36.41%
	No Incentive	6232	1244	19.96%	584	46.95%	660	53.05%
4	Total	11065	2158	19.50%	1045	48.42%	1113	51.58%
$15 offer	Dropout	717	118	16.46%	65	55.08%	53	44.92%
	$5 prepay	651	100	15.36%	46	46.00%	54	54.00%
	Targeted ($15 offer)	4780	966	20.21%	581	60.14%	385	39.86%
	No Incentive	4917	974	19.81%	353	36.24%	621	63.76%
5	Total	8907	497	5.58%	257	51.71%	240	48.29%
$25 offer	Dropout	599	32	5.34%	22	68.75%	10	31.25%
	$5 prepay	413	12	2.91%	5	41.67%	7	58.33%
	$15 offer	40	5	12.50%	5	100.00%	0	0.00%
	$5 prepay plus $15 offer	279	16	5.73%	11	68.75%	5	31.25%
	Targeted ($25 offer)	3701	262	7.08%	166	63.36%	96	36.64%
	No Incentive	3875	170	4.39%	48	28.24%	122	71.76%

Proposed Revisions to the Responsive Design Approach

We propose the following strategies to target an 82 percent response rate, while continuing to reduce the potential risk to data quality posed by nonresponse bias.

Extend data collection. Given the current level of participation, and the time lost due to the partial government shutdown, it will be necessary to extend data collection beyond October 31. We recommend conducting data collection through December 31, 2013. This will allow time to ramp up data collection now that the shutdown is lifted, conduct attrition training, and proceed with focused efforts for the balance of the data collection period, with a particular emphasis on the targeted cases (i.e., those cases with the highest predicted-likelihood to contribute to nonresponse bias unless interviewed).
Consider phase 5 to have been completed (prematurely) as of September 30 as a result of the shutdown.
Add a new phase 6 between the originally approved phases 5 and 6. To ensure that we collect the maximum number of full interviews, the abbreviated questionnaire will not be implemented until 3 weeks prior to the end of data collection, as described in the original OMB submission. The abbreviated questionnaire period would then become phase 7.
Modify the incentive package to increase overall response while maintaining our goal of reducing the risk of nonresponse bias. In an effort to minimize nonresponse bias while targeting an 82 percent response rate, we propose a multifaceted approach for the remainder of data collection. These incentives will be included as the new phase 6 of the modified responsive design approach.
- Increase the number of cases offered a $25 incentive by approximately 1,500 cases. We propose to move the predicted-likelihood cutoff which had been established for phase 5, thus identifying additional cases as having higher likelihood to contribute to nonresponse bias unless interviewed.
- Send a $5 prepaid incentive to a subset of cases that did not receive a prepaid incentive in phase 3, given that the lapse in data collection and possible frustration with the partial government shutdown may make sample members more hesitant to respond. We propose extending this prepaid incentive to the approximately 2,000 sample members with the lowest predicted-likelihood values who have not previously received a prepaid incentive. To encourage these sample members to open their mail and see the prepaid incentive check, an email will be sent on the day the mailing goes out indicating that a check is on its way. Revised contact materials are provided in Appendix A.
Phase 7: The implementation of the abbreviated questionnaire, previously approved as phase 6, will now be phase 7, and will comprise the last 3 weeks of data collection. As part of phase 7, we also propose to add a hardcopy abbreviated questionnaire option at the same time that the electronic abbreviated questionnaire is offered. The content of the hardcopy questionnaire matches that of the electronic abbreviated questionnaire. A hardcopy abbreviated questionnaire was used in the HSLS:09 base-year and first follow-up parent surveys and accounted for five and six percent of the parent responses, respectively. Sample members who have been previously offered an incentive would still receive the incentive regardless of the mode in which they complete the questionnaire: Web, CATI, or hardcopy abbreviated questionnaire. The hardcopy abbreviated questionnaire is provided in Appendix B.

As previously mentioned, 8,410 cases have not responded and 2,093 of these cases have been finalized with a non-interview status, leaving 6,317 pending cases available to work with. The selection of cases for the strategies proposed above would not factor in the finalized cases; therefore, not all selected cases would receive the treatment. Exhibit 2 shows the number of cases to be selected for additional treatment in the new phase 6; the number of pending cases that are available to be worked with, and thus receive the treatment; and the number of cases that are finalized as non-interview cases.

Exhibit 2. Proposed phase 6 treatments for non-interviewed cases.
Proposed phase 6 treatments	All pending and finalized non-interviewed cases	Estimated pending cases	Estimated finalized cases
$25 offer (additional cases identified)	1,500	1,125	375
$5 prepaid incentive (additional cases identified)	2,000	1,525	475

No changes to the burden estimate or cost to the federal government are anticipated as a result of the proposed data collection changes.

Appendix A. Additional Contact Materials

Student new prepaid incentive letter

[DATE]

<TEENAGER NAME> Web Address: surveys.nces.ed.gov/hsls

<ADDR1><ADDR2> Study ID: <XXXXXXX>

<CITY STATE ZIP> Password: <XXXXXXXX><PW_ind>

Dear <Teenager Name>:

Please take 15 minutes to participate in the High School Longitudinal Study of 2009 (HSLS:09). By participating, you have the opportunity to help educators, researchers, and policymakers fill an important gap in knowledge of students’ and parents’ decisionmaking at this critical transition point after high school.

<If prepaid: We have enclosed a $<pre-paid-inc-amt> check as a token of appreciation for your continued participation.> <If both prepaid and promised incentive: In addition,> <If incentive: You/you will <now> receive a $<incent> check after you finish the survey.>

Either you or your parent can complete the questionnaire. To complete the online survey, please log in using the web address, study ID, and password provided at the top of this page. If you have questions or would prefer to participate on the telephone, please contact one of our professional telephone interviewers at 1-855-500-1440. <<If teenager over 18: A letter was also sent to <<parent_name>> with their login information since either you or your parent/guardian can complete the questionnaire. If you would prefer that we not contact your parent, please call us toll-free at 1-855-500-1440.>>

Thank you in advance for your continued participation in HSLS:09!

Dan Pratt

HSLS:09 Project Director

RTI International

NCES is authorized to conduct HSLS:09 under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C., § 9543). According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number of this voluntary information collection is 1850-0852. The time required to complete this information collection is estimated to average 15 minutes for the questionnaire including the time to review instructions and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate or suggestions for improving the interview, please write to: High School Longitudinal Study of 2009 (HSLS:09), National Center for Education Statistics, 1990 K Street NW, Washington, DC 20006.

Parent new prepaid incentive letter

[DATE]

<PARENT NAME> Web Address: https://surveys.nces.ed.gov/hsls

<ADDR1><ADDR2> Study ID: <XXXXXXX>

<CITY STATE ZIP> Password: <XXXXXXXX><PW_ind>

Dear <Parent or Guardian Name>:

<If Prepaid: A $<pre-paid-inc-amt> check has been sent to <Teenagername> as a token of our appreciation for your continued participation.> <If both prepaid and promised incentive: In addition,> <If incentive: You/you will <now> receive a $<inc_amt> check after you finish the survey.>

Either <teenager name> or <his/her> parent/guardian can complete the questionnaire.

To complete the questionnaire online, please log in using the web address, study ID, and password provided at the top of this page. If you would prefer to participate over the telephone, you may call 1-855-500-1440.

Thank you in advance for your continued participation in HSLS:09!

Dan Pratt

HSLS:09 Project Director

RTI International

<<If student under 18: Enclosure: Young adult Letter>>

Student abbreviated questionnaire letter

[DATE]

<TEENAGER NAME> Web Address: surveys.nces.ed.gov/hsls

<ADDR1><ADDR2> Study ID: <XXXXXXX>

<CITY STATE ZIP> Password: <XXXXXXXX><PW_ind>

Dear <Teenager Name>:

We know your time is valuable. We are now offering an abbreviated survey for the High School Longitudinal Study of 2009 (HSLS:09) that will take only about 5 minutes to complete. Please complete online using the log in information above or call 1-855-500-1440 to complete by phone <If incentive: and we will send you a $<inc_amt> check after you finish the survey>.

If you prefer, we have enclosed a paper version of the questionnaire, which you can complete and return in the enclosed business reply envelope. We encourage you to participate online or by telephone, <If incentive: as it will take longer to process the form and send you your $<inc_amt> check>, but the choice is available for your convenience. Note that you should only complete one form, either electronically or hardcopy, but not both.

Please act quickly because the study ends December 31st.

Thank you in advance for your participation in this important study. Your responses will help improve high school education for generations to come, and your cooperation is greatly appreciated.

Sincerely,

Elise Christopher, Ph.D.

NCES Project Officer

National Center for Education Statistics

U.S. Department of Education

Parent abbreviated questionnaire letter

[DATE]

<PARENT NAME> Web Address: https://surveys.nces.ed.gov/hsls

<ADDR1><ADDR2> Study ID: <XXXXXXX>

<CITY STATE ZIP> Password: <XXXXXXXX><PW_ind>

Dear <Parent or Guardian Name>:

Please act quickly because the study ends December 31st.

<<If student under 18: If <teenagername> prefers to complete the questionnaire, please give <him/her> the enclosed letter to indicate that you have given your permission for <him/her> to participate.>><<If student over 18: We also sent a letter to <teenagername> with <his/her> log-in information since either you or <teenagername> may complete the questionnaire.>>

Thank you in advance for your participation in this important study. Your responses will help improve high school education for generations to come, and your cooperation is greatly appreciated.

Sincerely,

Elise Christopher, Ph.D.

NCES Project Officer

National Center for Education Statistics

U.S. Department of Education

<<If student under 18: Enclosure: Young adult Letter>>

2013 UPDATE OF HIGH SCHOOL LONGITUDINAL STUDY OF 2009 Case ID: <caseid>

ABBREVIATED QUESTIONNAIRE

OMB No.: 1850-0852 Exp. Date: 4/30/2016

The public reporting burden for collection of this information is estimated to average 5 minutes. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202-4651. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: High School Longitudinal Study of 2009 (HSLS:09), National Center for Education Statistics, 1990 K Street NW, Washington, DC 20006.

Q1. Who is completing this questionnaire?

^1-<first name>

^2-His/her parent or guardian

Q2. Has <first name> earned a high school credential such as a high school diploma, certificate of attendance, GED or other high school equivalency?

^1-Yes

^0-No Go to Q5

Q3. What type of high school credential has <first name> earned?

^1-High school diploma

^2-GED or other high school equivalency

^3-Certificate of attendance

Q4. In what month and year did <first name> receive his/her high school credential?

Month:_______________ Year:________________ Go to Q9 on the inside

Q5. Is <first name> currently attending high school, not attending high school, or being homeschooled? If <first name> is out for a high school break, illness, injury, or vacation, please consider him/her as attending high school.

^1-Attending high school Go to Q7

^2-Not attending high school

^3-Being homeschooled Go to Q7

Q6. Does <first name> plan to get a high school credential such as a high school diploma, certificate of attendance, GED or other high school equivalency?

^1-Yes

^0-No Go to Q9 on the inside

Q7. What type of high school credential does <first name> plan to earn?

^1-High school diploma

^2-GED or other high school equivalency

^3-Certificate of attendance

Q8. About what month and year does <first name> expect to receive his/her high school credential?

Month:_______________ Year:________________

PLEASE CONTINUE TO Q9 ON THE INSIDE.

Q9. What is the full name, city, and state of the high school <first name> last attended or is currently attending? (Do not enter abbreviations.)

Name:__________________________________________________________________________

City:____________________________________________________________________________

State:__________________________________________________________________________

Q10. On or around November 1^st, was <first name> taking classes from a college, university, community college, trade school, or other occupational school (such as a cosmetology school or a school of culinary arts)?

^1-Yes

^2-No Go to Q11

^3-Don’t know Go to Q11

Q11. Which of the following are reasons why <first name> was not attending school as of November 1st? Would you say <first name> was not attending school…

because <first name> did not want to go to school?	Yes	No
because <first name> did not want to go to school?	^1-	^0-
because <first name> did not get in?	^1-	^0-
because <first name> could not afford to go to school?	^1-	^0-
for other reasons?	^1-	^0-

Shape13

SKIP TO Q15 ON THE TOP OF THE NEXT PAGE.

Q12. Was <first name> enrolled in school full-time or part-time as of November 1st?

^1-Full-time

^2-Part-time

^3-Don’t know

Q13. What is the name, city and state of the school or college <first name> was attending as of November 1st? (Please write in the full name. Do not use abbreviations.)

Name:__________________________________________________________________________

City:____________________________________________________________________________

State:__________________________________________________________________________

Q14. What type of program was <first name> enrolled in?

^1-Bachelor’s degree program (usually a 4-year degree)

^2-Associate’s degree program (usually a 2-year degree)

^3-Certificate or diploma program from a school that provides occupational training (usually takes 2 years or less to complete, often leading to a license, such as cosmetology)

^4-No specific program, but he/she was taking courses

^5-Other

^6-Don’t know

PLEASE CONTINUE TO Q15 ON THE NEXT PAGE.

Q15. Did <first name> or another family member complete a FAFSA, that is the Free Application for Federal Student Aid, for <first name>’s education?

^1-Yes Go to Q17

^2-No

^3-You don't know what a FAFSA is Go to Q17

^4-You don't know if <first name> or another family member completed a FAFSA Go to Q17

Q16. What are the reasons <first name> did not complete a FAFSA? Would you say <first name> did not complete a FAFSA...

because you do not or your family does not want to take on debt?	Yes	No
	^1-	^0-
because you or your family can afford school or college without financial aid?	^1-	^0-
because you or your family thought <first name> may be ineligible or may not qualify?	^1-	^0-
because you or your family did not have enough information about how to complete a FAFSA?	^1-	^0-
because you or your family thought the FAFSA forms were too much work or too time-consuming?	^1-	^0-
because you or your family did not know you could complete a FAFSA?	^1-	^0-
because <first name> does not plan to continue his/her education after high school?	^1-	^0-

Q17. On or around November 1^st, was <first name> participating in an apprenticeship program?

^1-Yes

^2-No

^3-Don’t know

Q18. On or around November 1^st, was <first name> working for pay?

^1-Yes

^2-No If participating in an apprenticeship program, go to Q19. Otherwise, skip to Q20.

^3-Don’t know If participating in an apprenticeship program, go to Q19. Otherwise, skip to Q20.

Q19. As of November 1^st, was <first name> working full-time, that is 35 hours or more per week?

^1-Yes

^2-No

^3-Don’t know

Q20. Is <first name> currently working for pay, not counting work around the house? Include apprenticeships.

^1-Yes

^0-No Go to Q27 on the back page.

Q21.On <first name’s> current job, how much does he/she currently earn before taxes are taken out? Please include tips if applicable. (If he/she has more than one job, please report on the one at which he/she works the most hours.)

$__________.___ per ^1-hour

^2-week

^3-month

^4-year

PLEASE CONTINUE TO Q22 ON THE BACK PAGE.

Q22. On <first name’s> current job, how many hours does he/she usually work per week?

__________ hours per week

Shape23

IF EARNINGS IN Q21 ARE PER HOUR OR PER WEEK GO TO Q25.

IF EARNINGS IN Q21 ARE PER MONTH GO TO Q23.

IF EARNINGS IN Q21 ARE PER YEAR GO TO Q24.

Q23. On <first name’s> current job, how many weeks does he/she usually work per month?

__________ weeks per month  GO TO Q25

Q24. On <first name’s> current job, how many months does he/she usually work per year?

__________ weeks per year

Q25. Is <first name’s> current job related to the job <first name> wants to have when he/she has completed his/her education? Would you say…

^1-Closely related

^2-Somewhat related

^3-Not at all related

Q26. Was <first name> working at his/her current job on or around November 1st?

^1-Yes

^0-No

Q27. Which of the following activities was <first name> doing on or around November 1st?

	Yes	No	Don’t know

Serving in the military including ROTC	^1-	^2-	^3-
Starting a family or taking care of your children	^1-	^2-	^3-
Attending high school or homeschool	^1-	^2-	^3-
Taking a course to prepare for the GED or other high school equivalency exam	^1-	^2-	^3-

THANK YOU FOR YOUR COOPERATION.

Please return the completed form in the enclosed postage-paid envelope or to this address:

RTI International  Research Operations Center  One North Commerce Center 5265 Capital BoulevardRaleigh, NC 27616-9779

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Debbie Herget
File Modified	0000-00-00
File Created	2021-01-28