Justification

Volume I:

Request for Clearance for Pretest of the

Fast Response Survey System (FRSS) 104: Dual Credit and Exam-Based Courses in Public High Schools: 2010–11

and

Postsecondary Education Quick Information System (PEQIS) 18: Dual Enrollment of High School Students at Postsecondary Institutions: 2010–11

OMB# 1850-0803 v. 50

May 9, 2011

National Center for Education Statistics

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct up to two rounds of pretests each on two complementary surveys about dual credit and dual enrollment programs offered to high school students. One survey will be conducted at public high schools and is part of the Fast Response Survey System (FRSS). The other survey will be conducted at postsecondary institutions and is part of the Postsecondary Education Quick Information System (PEQIS). The surveys were requested and funded by the Office of Elementary and Secondary Education (OESE), U.S. Department of Education. The secondary school survey, referred to as FRSS 104, will provide nationally representative data on prevalence and enrollment of dual credit and exam-based courses in public high schools. The postsecondary institution survey, referred to as PEQIS 18, will provide nationally representative data on the prevalence of college coursetaking by high school students both within and outside of dual enrollment programs offered by postsecondary institutions. Both surveys were previously conducted by NCES in 2002–03; Dual Credit and Exam-Based Courses in U.S. Public High Schools: 2002–03 (FRSS 85) and Dual Enrollment of High School Students at Postsecondary Institutions (PEQIS 14). One OMB package has been submitted for FRSS 104 and PEQIS 18 because the surveys are closely related in topic and schedule, and we have coordinated their survey development.

Two rounds of feasibility calls were conducted between February 2011 and April 2011 (OMB# 1850-0803) for both of the surveys to explore topics for potential new survey items, identify and correct issues with the content and format of the surveys before conducting pretests, and to assure that both surveys capture the intended meaning of the questions and minimize the burden imposed on respondents. Feasibility calls involved asking members of the target populations to review, but not complete, a draft survey and participate in a telephone discussion. Following feasibility calls, the surveys were reviewed by the NCES Quality Review Board (QRB), representing all four of NCES’s divisions, and revised accordingly to prepare them for the pretests described here. The purpose of the survey pretests is to further test both instruments prior to OMB submission to conduct the full surveys. Pretests will involve asking respondents to complete the survey and participate in a telephone debriefing. The request to conduct the full surveys will be submitted at a later date under OMB generic clearance for quick response surveys (OMB#1850-0733), which are authorized under the Education Science Reform Act of 2002.

Design

Overview of Survey Development

NCES has contracted Westat to conduct the surveys for FRSS 104 and PEQIS 18, including development of the survey instruments. The FRSS and PEQIS systems have established procedures for developing short surveys on a wide variety of topics. The techniques that we are using to shape the survey design on FRSS 104 and PEQIS 18 include literature reviews on dual credit and dual enrollment courses, several rounds of feasibility calls (which have been completed), and up to two pretests. The specific ways we plan to use the pretests are discussed below.

The upcoming surveys (2010–11) will be similar to the previous (2002–03) versions, with modifications based on feedback from respondents at secondary schools and postsecondary institutions. Many questions are consistent with the 2002–03 versions to facilitate making comparisons, but several new questions have been added and some existing questions and instructions have been updated due to changing circumstances. For example, questions that pertain to the topic of dual enrollment courses taught through distance education required revision and expansion because this instructional approach was not as prevalent in 2002–03 as it is now. During the pretests, we will test all questions on the surveys and obtain estimates of the respondent time required to complete each survey.

We anticipate conducting up to two rounds of pretests for each survey in the month of June 2011. Each round will include ten or fewer respondents. Pretest respondents will be asked to review, complete, and fax back the 3-page paper and pencil survey, and will be invited to provide their feedback by telephone. They will also be asked to record the time required to complete the survey. After the first pretest for each survey, we will discuss the results with NCES and OESE and modify the survey instruments if necessary. If needed, we will conduct a second pretest to test changes to the survey.

Consultations Outside of Agency

While the majority of survey topics are consistent with previous iterations of the surveys, additional topics were identified through literature reviews and in consultation with OESE. OESE has provided input throughout survey development, reviewing and approving necessary modifications to the survey instruments. Each survey was tested with respondents during feasibility calls.

Sample, Burden, and Cost

In this submission, we are requesting approval for pretest activities with members of the target populations. We will conduct up to two pretests for each of the two surveys, with 10 or fewer respondents per round. Secondary schools will be recruited to participate in the pretest for FRSS 104 based on various school characteristics including size, locale, and geographic region. Respondents for the FRSS 104 survey will be recruited by telephone and will be identified as the person most knowledgeable about the school’s dual credit, AP, and IB courses. In the 2002–03 survey and during feasibility testing, this was usually a school guidance counselor. Postsecondary institutions will be recruited to participate in the pretest for PEQIS 18 based on several institution characteristics including type, size, and geographic region. Respondents for the PEQIS 18 survey will be recruited by telephone from postsecondary institutions and will be identified as the person most knowledgeable about dual enrollment program(s) at the institution.

Telephone interviewers will recruit participants for the FRSS 104 and PEQIS 18 pretests using the recruitment scripts in attachments 1 and 2, respectively. Interviewers will schedule an appointment to complete the pretest interview with cooperating schools and institutions. Following telephone recruitment, interviewers will either email, mail, or fax a cover letter and draft questionnaire to the participating institutions (as discussed in the Data Collection Instruments section).

In order to recruit 10 respondents for FRSS 104, we anticipate contacting 40 public high schools¹ (Table 1). We anticipate contacting 45 postsecondary institutions to recruit 10 respondents per round for PEQIS 18 (Table 2). On average, recruitment calls with respondents who agree to participate in the pretest are expected to take about 10 minutes to explain the purpose of the call and set up an appointment to discuss the survey; all other recruitment calls are expected to take about 3 minutes.

Prior to the pretest, FRSS 104 and PEQIS 18 respondents will be asked to complete the draft survey, which should take approximately 30 minutes. The pretest debriefing interview should take approximately 30 minutes to complete. The total estimated burden time is 26.4 hours for FRSS 104, and 26.9 hours for PEQIS 18 for two pretests.

We anticipate that the estimated cost to the federal government will be approximately $10,000 for each pretest round for each survey.

Table 1. Maximum burden time for FRSS 104 pretest activities

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Each Round
Recruitment – Schools not participating in the pretest	30	30	0.05	1.5
Recruitment – Schools participating in the pretest	10	10	0.17	1.7
Survey completion and debriefing interview	10	10	1	10
Total per round	40	50	-	13.2
Total for two rounds	80	100	-	26.4

¹ Counts each contact (e.g., recruitment and debriefing interview are counted separately even when they are with the same respondents).

Table 2. Maximum burden time for PEQIS 18 pretest activities

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Each Round
Recruitment – Institutions not participating in the pretest	35	35	0.05	1.75
Recruitment – Institutions participating in the pretest	10	10	0.17	1.7
Survey completion and debriefing interview	10	10	1	10
Total per round	45	55	-	13.45
Total for two rounds	90	110	-	26.9

¹ Counts each contact (e.g., recruitment and debriefing interview are counted separately even when they are with the same respondents).

Data Collection Instruments

For each round of pretests, a cover letter and draft questionnaire will be emailed or faxed to each participating school or institution. The cover letter and questionnaire appear in attachments 3 and 4 for FRSS 104 and in attachments 5 and 6 for PEQIS 18. The cover letter thanks the respondent for agreeing to participate in the pretest, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should complete the questionnaire and fax it back to Westat, includes questions for respondents to consider while completing the survey, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter and on the cover of the survey, respondents are assured that their participation is voluntary and their answers may not be disclosed or used in identifiable form for any other purpose unless compelled by law. The public law is cited on the cover letters and the front page of the surveys (attachments 3 through 6). The materials for the second pretest round (if needed) will be similar, except the survey instrument will include the modifications that resulted from the previous round. The current instruments for each survey are discussed below.

FRSS 104 Secondary School Survey

The survey is designed to collect general information on dual credit and exam-based courses for public secondary school students. During the feasibility testing, some questions were deleted and other questions were edited based on respondent feedback. The definition of dual credit asks respondents to include courses or programs where high school students can earn both high school and postsecondary credits for the same courses.

The first four questions ask about types of exam-based courses, including Advanced Placement (AP) and International Baccalaureate (IB) courses. The first question asks whether students took AP or IB courses and is included to set up the skip pattern for Question 2. The second question asks for the number of enrollments in both types of exam-based courses. During feasibility calls, we found that students sometimes receive dual credit for passing an AP course without having to take the AP exam. Respondents were counting these courses in both the AP subsection and the dual credit subsection of the questionnaire. As a result, the next set of questions was added to address the potential overlap between AP and dual credit courses. Question 3 asks whether any students have taken any AP courses for which they could earn dual credit without taking the AP exam. Question 4 asks for the number of enrollments in such courses. In addition, we added an item to the dual credit definition to indicate that dual credit courses might include AP courses for which students can earn dual credit without taking the AP exam.

Questions 5-7 ask about dual credit courses and requirements related to taking such courses. Question 5 is provided to ‘screen out’ schools that do not have students taking dual credit courses. Question 6 asks whether the school has established any requirements that students must meet for participation in dual credit courses, and is included to set up the skip pattern for Question 7. Question 7 asks about specific student requirements for enrolling in dual credit courses. Items were added to the list in Question 7 as a result of feedback during feasibility calls.

Question 8 was added to address an NCES interest in whether any students earned postsecondary credentials by taking dual credit courses. It was tested during the second round of feasibility calls.

Questions 9-10 ask about course focus and enrollments regarding dual credit courses. Question 9 asks whether students took any dual credit courses with an academic focus or a career and technical/vocational focus. Question 10 asks respondents for the number of enrollments for courses taught primarily through distance education, on the high school campus, and on the postsecondary campus.

During feasibility calls, we found that distance education has become a more prevalent method of offering dual credit courses and respondents do not tend to think of these courses separately from other dual credit courses. We revised the questionnaire so that dual credit courses taught through distance education are included throughout the survey.

Questions 11-12 focus on dual credit courses taught on high school and postsecondary campuses. Question 11 asks about the instructors who taught the dual credit courses on the high school campus. Question 12 asks about the student composition of courses taught at the postsecondary institution.

Question 13 asks whether students were awarded postsecondary credit immediately upon course completion or were offered credit in escrow.

Questions 14-16 are included in the survey to collect information about the costs of enrollment in dual credit courses. Question 14 asks whether students or their parents generally paid out of pocket for tuition, fees, and books related to dual credit courses. Question 15 asks whether the school or district paid any of these expenses for students taking dual credit courses. Question 16 asks who is responsible for transportation costs associated with participation in dual credit courses.

PEQIS 18 Postsecondary Institution Survey

The survey is designed to collect information about dual enrollment programs offered to high school students by postsecondary institutions. During the feasibility testing, some questions were deleted and other questions were edited based on respondent feedback.

The first question asks if any high school students took courses during the previous academic year for college credit through the institution, including courses within and outside of a dual enrollment program. Respondents that answer “no” to this question are asked to return the survey without answering any additional questions.

Questions 2-3 pertain to courses offered to high school students outside of a dual enrollment program. Question 2 asks if any high school students took such a course, and Question 3 asks how many students took these courses.

Question 4 is included to determine if high school students took courses for college credit through the institution that were part of a dual enrollment program. For the remaining survey questions, respondents are asked to consider only courses within a dual enrollment program. These questions are designed to collect characteristics of the dual enrollment courses (e.g., location and type of teacher) that would not be meaningful for courses outside an organized program (e.g., high school students taking courses on their own who are treated as regular college students). Question 5 asks for the unduplicated head count of high school students taking these courses.

Question 6 asks where the dual enrollment courses were taken. This question has been edited based on feasibility testing to account for courses taught via distance education. During the first round of feasibility testing respondents inconsistently reported courses taught via distance education. Response option “a. Taught primarily through distance education where the student and teacher are separated by location” was added to address this issue. In addition, parenthetical instructions to exclude distance education courses were added to response options b. and c. to ensure respondents only included these courses in response option a.

Questions 7 and 8 collect information about the dual enrollment program instructors and instructor qualifications for dual enrollment courses taught on a high school campus. Question 7 asks who the instructors were, and Question 8 asks how the minimum qualifications for high school instructors who taught college-level courses compare to those required for college instructors.

Questions 9-11 ask about coursetaking patterns for high school students participating in dual enrollment program(s) at the institution. Question 9 asks about the typical number of courses taken per academic term, and Question 10 asks for the maximum number of courses allowed per academic term. Question 11 asks when high school students were generally awarded college credit for courses taken through the dual enrollment program(s).

Questions 12-14 are included in the survey to gather information about the eligibility requirements for high school students to participate in dual enrollment programs at the institution. Question 12 is included to collect information about grade level requirements for high school students participating in dual enrollment programs. This question was moved to earlier in the survey based on respondent feedback during feasibility testing, and the suggestion of the NCES QRB. This change addresses the concern that participants may write in “grade-level” as a requirement at Question 13 if Question 12 does not precede it. Question 13 asks what the eligibility requirements were for students to participate in the dual enrollment program(s). During feasibility testing some respondents felt they could not answer Question 13 because the high school established or monitored these requirements. To address this issues the response option, “Don’t Know” was added for each item at Question 13. Question 14 asks if the academic requirements for high school students to participate in the dual enrollment program(s) are the same or different from the admission standards for regular college students. During feasibility testing several respondents noted the answer to this question would be “different” because a high school diploma is required for regular college students and this is obviously not a requirement for high school students participating in the dual enrollment program(s). Because the intent of this question is to determine if the admission standards for high school students are different from regular college students excluding the requirement of a high school diploma, an instruction was added to exclude the requirement of a high school diploma when answering this question.

Question 15 asks whether the curriculum for courses in the dual enrollment program(s) are specially designed for high school students or the same as the curriculum for regular college students.

Questions 16-18 are included in the survey to collect information about the costs of enrollment in dual enrollment programs. Question 16 is a new question added to the survey based on respondent feedback during feasibility testing. Several respondents noted their institution discounts the rate of tuition for high school students participating in dual enrollment programs. Because the cost of dual enrollment program(s) is important to OESE, Question 16 was added to explicitly gather information about tuition discounts. Question 17 asks which sources paid tuition for the college-level courses taken in the dual enrollment programs. The parenthetical explanation at response option a. of Question 17 has been edited to include discounted tuition rates. During feasibility testing respondents inconsistently reported the discounted tuition rates at Question 17. This edit was made to ensure respondents consistently reported these discounts. Question 18 asks whether high school students or their parents generally paid out of pocket for tuition, fee, or books related to these courses.

Question 19 was added to address an NCES interest. This question is asked to gather information about awards high school students may receive through participation in the dual enrollment program(s).

Question 20 is a new question added at the request of OESE because OESE has a policy interest in the prevalence of comprehensive dual enrollment programs.

Questions 21-25 are included in the survey to gather information about dual enrollment programs that are geared specifically toward high school students at risk of educational failure. Question 21 asks if the institution has such a program. Questions 22-25 are only asked of institutions responding yes to Question 24. Question 22 asks for the number of students enrolled in these programs and Question 23 gathers information about the typical pattern of enrollments for these students. Questions 24 and 25 collect information about support services available to students enrolled in these types of dual enrollment programs.

Timeline

Recruitment for the pretests will begin immediately upon OMB’s approval. The goal is for the pretest activities to take place before the end of the 2010-2011 school year. Pretest activities are anticipated to take 1 month to complete. This includes the time to recruit respondents, conduct interviews with respondents after they have completed the survey, prepare the pretest report for NCES and OESE, and consult with NCES and OESE following pretest activities.

1 Estimates for the number of schools and institutions to be contacted for the pretest are based on the 2002–03 survey data on the percentage of high schools offering dual credit, AP and IB courses (for FRSS 104), and the percentage of postsecondary institutions offering dual enrollment programs (for PEQIS 18), as well as the estimated percentage expected to participate.

9