OMB Memo

MEMORANDUM OMB # 1850-0852 v.4

DATE: March 6, 2010

TO: Brian Harris-Kojetin, Shelly Martinez

Office of Information and Regulatory Affairs, Office of Management and Budget

FROM: Kashka Kubzdela

National Center for Education Statistics

THROUGH: Laura LoGerfo, Jeff Owings

National Center for Education Statistics

SUBJECT: HSLS:09 Parent Recruitment Change Request

Background

The High School Longitudinal Study of 2009 (HSLS:09) is a nationally representative longitudinal study of more than 20,000 9th graders in 944 schools, who will be followed through their secondary and postsecondary years. The study focuses on understanding students’ trajectories from the beginning of high school into university or the workforce and beyond. What students decide to pursue when, why, and how are crucial questions for HSLS:09, especially, but not solely, in regards to science, technology, engineering, and math (STEM) courses, majors, and careers. This study includes a new student assessment in algebraic skills, reasoning, and problem solving and like past studies, surveys students, their parents, math and science teachers, school administrators, and school counselors. This first wave of data collection for HSLS:09 will produce not only a nationally representative dataset but also ten individual state representative datasets for each of ten states. This change request memo reflects an addition of an incentive experiment to increase parent response rates in filling out their survey.

Request

We are seeking clearance to amend the approved base year data collection plan of the High School Longitudinal Study of 2009 (HSLS:09) in order to alleviate the unexpectedly low parent participation rates. This amendment involves an incentive experiment to improve response rates among HSLS:09 parents, a flyer to publicize this incentive to the target parents, use of abbreviated versions of the parent questionnaire, and a 16-day extension to data collection from parents to allow the incentive experiment opportunity to have an impact without risking excessive staffing and administration costs.

This change request includes a copy of the revised flyer; two versions of an abbreviated questionnaire for parents to fill out; and a copy of two versions of a letter addressed to non-responsive parents I hopes of gaining their participation.

One of the factors determining whether a target parent respondent is considered as having a decreasing likelihood of participating in a survey is the number of calls attempted to reach and encourage the parent to fill out the survey. Regarding call-count, there are various "cut-points" at which diminishing returns of success are experienced. Analyses of HSLS calls to parents suggest that the experimental design treatment should begin with the 15th call -- i.e., after 14 or more calls have been attempted. Among completed interviews: 88% required fewer than 15 calls; 12% required 15 or more calls. To break this down further:

• For households not reached but not in tracing (the ‘4507’ – call screening, voicemail, etc.): 58% of the cases have had 15+ call attempts; 42% of the cases have had fewer than 15 call attempts.

• For households reached but not interviewed (the ‘4157’ – someone, but not the target, is answering): 59% of the cases have had 15+ call attempts; 41% of the cases have had fewer than 15 call attempts.

To determine whether monetary incentives improve parent response rates, once we reach the 15-call threshold for a given household, we propose that the parents receive one of three experimental incentive conditions: $0, $10, or $20. All parents within the same school must receive the same incentive, so these conditions will be clustered by school membership. This call-threshold plan will apply to both of the aforementioned groups (reached and not reached).

The splits for the incentive groups will be determined at the school-level to preserve the experimental design: so we will let the splits fall where they may for the pending parent cases, aiming for a balanced set of cases (roughly 1/3) for each aspect, including the refusal-conversion cases. We recommend $0, $10, and $20 for the refusal-conversions as well to keep the incentive-by-school design intact.

For refusal-conversion cases (the ‘1000’ plus any possibilities from the ‘final refusal’ group): We propose the same 3 incentive levels of $0, $10, $20 to ensure that parents in the same school are not receiving different incentive offers. However, we recommend that all refusal-conversion cases be allowed to take the abbreviated interview.

Note: The abbreviated interview differs from the critical-item hard-copy version (both attached). The abbreviated interview is available online and uses a subset of the already approved full parent questionnaire that is in use; it takes 10 to 15 minutes for parents to complete. The critical-item hard-copy version of the survey will span just two pages and include only the most important data to collect from parents, namely the variables that enter the calculation of socioeconomic status.

"Final refusals" (the ‘800’): We should NOT attempt anything with the final reviewed refusals to risk withdrawal. However, we should review possible final-refusals (one step away from absolute final refusal designation) and shift a subset to refusal-conversion attempts.

For dead-end-trace cases (the ‘1107’): A potentially-good address is associated with about 82% of these cases, and a potentially-good e-mail address is associated with about 26%. By definition, we question all phone numbers for these cases because those phone-numbers were "dead-ended" in CATI and therefore the cases have made it to the tracing pipeline.

We recommend allowing these cases to move through the tracing pipeline and offer three options only to those that dead-end and for which we have a mailing address (not to cases that are still being traced): 1) to complete the very brief two-page hardcopy critical-item form, or 2) to do the regular half-hour survey on the web, or 3) to call in to complete the regular half-hour survey. If they accept Option #2 or Option #3, they will receive the designated incentive amount according to the conditions of the incentive experiment, $0, $10, or $20. Options #2 and #3 refer to the full interview generally, however, there is one exception, pending refusals. Pending-refusals who take Option #2 or Option #3, with whom we tread lightly, can take the “abbreviated” web/CATI interview. Cases that are both tracing-dead-ends and pending refusals (i.e., both circumstances apply), the pending refusal status dictates that the abbreviated interview is the relevant interview. In other words, pending refusal status drives the selection of which survey version to take.

If dead-end trace cases accept Option #1 and complete the critical-item hard-copy two-page version of the survey, we will activate a $5 gift card (sent through the mail with this two-page critical-item hard-copy version) upon receipt of the completed survey. Our intent is to give the $5 gift card to all who complete this brief two-page hardcopy critical-item form. This form differs so drastically from the regular survey in time commitment, thus warranting a different approach from the experiment. Yet this approach applies a consistent treatment across all groups. In essence, the differential incentive experiment is implemented for the “excessive call” cases and “pending refusals;” the $5 gift card approach is planned for the tracing-dead-end cases (e.g., those without an active phone). These are different groups of cases and would be completing very different forms of the survey.

It's possible that this "hard-copy" set could be expanded as a last-ditch effort. But, we first give parents the opportunity to provide more information if they choose to complete the survey online or call in. On a daily, ongoing basis, cases can move into incentive groups, for example, if a case becomes a pending refusal. Thus, this plan requires constant monitoring of the remaining cases. But, this dynamic complexity is contained within a clearly structured experimental design.

Finally, we will extend the data collection period through April 16 to allow for the return of any hard-copy questionnaires. The deadline to return the questionnaires will be stated previous to April 16, which allows for the postal system time lag.

A report of the experiment’s outcomes will be shared with OMB as soon as results are known.

In sum, this plan keeps treatment within school intact; identifies a reasonable threshold for "difficult" cases; offers special provisions for refusal-conversion; has differential approaches for the 3 types of cases (tracing-dead-ends; pending refusals; hard to reach); preserves the experiment while allowing potential for increasing numbers through a hard-copy; does not add to respondent burden or to total government cost.