Justification

Volume I:

Fast Response Survey System (FRSS) 108: Career and Technical Education (CTE) Programs in Public School Districts – Feasibility Calls

OMB# 1850-0803 v. 144

September 25, 2015

National Center for Education Statistics

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct feasibility calls for Fast Response Survey System (FRSS) survey #108 on career and technical education (CTE) programs in public school districts. The Office of Career, Technical, and Adult Education (OCTAE) requested that NCES conduct this FRSS survey.

Nearly all public high school students (95 percent of ninth-grade students in 2009) attended a school that offered Career and Technical Education (CTE) instruction, either on campus or at a partnering school. In 2009, 85 percent of public high school graduates had completed one or more occupational CTE courses, 76 percent had earned at least one full credit in occupational CTE, and 19 percent were CTE concentrators who had earned at least three credits in the same CTE field.¹

Effective, high-quality CTE programs are aligned with college- and career-readiness standards as well as the needs of employers, industry, and labor. They provide students with a curriculum that combines integrated academic and technical content and strong employability skills. They also provide work-based learning opportunities that enable students to connect what they are learning to real-life career scenarios and choices. The students participating in effective CTE programs graduate with industry certifications or licenses and postsecondary certificates or degrees that prepare them for in-demand careers within high-growth industry sectors.²

According to the criteria identified by OCTAE in their blueprint specifications for the most recent reauthorization of the Perkins Act as characteristics that define high-quality CTE programs, a high-quality CTE program:

• Supports career pathways in in-demand industry sectors and occupations;

• Provides students with information about occupations in in-demand industry sectors or occupations and may offer career exploration activities as early as 7^th grade;

• Offers a non-duplicative, structured sequence of courses that begin at the secondary level and lead, as applicable, to an industry-recognized credential (in sectors where those credentials exist and are appropriate) and to a postsecondary certificate or degree that is needed for placement in an in-demand occupation that leads to economic self-sufficiency;

• Provides students with the academic, employability, and technical skills that employers require for entry into occupations in in-demand industry sectors or occupations;

• Offers opportunities for students to earn academic credit and postsecondary credit for completing high school career and technical education courses;

• Provides all participating students with work-based learning;

• Provides supplemental services to participating students who are members of underserved populations and provides support services to all participating students to ensure that all students have equitable access to career and technical education programs, in addition to equitable opportunities to participate and succeed in these programs; and

• Offers opportunities for participating students to develop leadership skills.

The purpose of this Fast Response Survey System (FRSS) survey is to collect nationally representative data from public school districts about CTE programs offered by the districts. These programs may be offered at district facilities or in a partnering off-site location, such as area CTE facilities or postsecondary institutions. The sample will focus on school districts with high schools because CTE programs are generally designed for high school students. The survey topics will focus on the extent to which districts provide high-quality CTE programs and how these programs are developed, designed, and implemented.

The purpose of feasibility calls is to explore topics for potential survey items, identify and correct any potential issues with the content and format of the survey before conducting pretests, and to ensure that the survey captures the intended meaning of the questions and minimizes the burden imposed on respondents. A request to conduct pretest activities will follow completion of the feasibility calls. Early rounds of the feasibility calls will ask school district respondents to participate in a telephone discussion about the CTE programs in their school district. This information will be used to develop the questionnaire. In later rounds of the feasibility calls, respondents will be asked to review, but not complete draft questionnaire items and ultimately, the draft questionnaire, and provide feedback by telephone. The pretests will involve asking respondents to complete the draft survey and participate in a telephone debriefing. Feasibility calls will be done before pretests to minimize the burden on respondents. Pretests will be done as a final test prior to OMB clearance submission requesting to conduct the full-scale survey. The request to conduct the full-scale survey will be submitted at a later date under OMB generic clearance for quick response surveys (OMB#1850-0733), which are authorized under the Education Science Reform Act of 2002 (ESRA, 20 U.S.C. §9543). NCES has contracted Westat for all stages of this survey.

Design

Overview of Survey Development

NCES has contracted Westat to conduct the survey for FRSS 108, including development of the survey instrument. FRSS has established procedures for developing short surveys on a wide variety of topics. The techniques that we plan to use to shape the survey design on FRSS 108 include literature reviews on CTE programs, input from the NCES Quality Review Board (QRB), several rounds of feasibility calls, and up to two pretests. The specific ways we plan to use feasibility calls are discussed below.

We anticipate conducting up to three rounds of feasibility calls, each with fifteen or fewer respondents. With new surveys such as the CTE programs survey, the initial feasibility calls use an open-ended interview guide rather than a questionnaire. As rounds of feasibility calls progress, respondents will be asked to review, but not complete draft questionnaire items and ultimately a draft questionnaire. Conducting multiple rounds of feasibility calls will systematically inform us about the availability and nature of high-quality CTE programs in order to draft a questionnaire, and in later rounds will provide in-depth information on respondents’ perceptions of the draft survey and response burden. The process will result in several iterations of the questionnaire items. The first round of calls will focus on learning more about the types of CTE programs offered by public school districts, and the terminology districts use regarding these programs. For the next round of feasibility calls, respondents will be asked to review draft survey questions, instructions, and definitions based on the initial round of feasibility calls. As a result of the feedback we receive, we will make any necessary changes to the survey items and draft a questionnaire. In the third and final round of feasibility calls, we will ask for respondents’ feedback on the draft questionnaire. The resulting draft of the survey will be reviewed by the NCES QRB and revised as necessary to prepare it for pretesting.

NCES Review and Consultations Outside of Agency

The NCES QRB members reviewed a draft list of questionnaire topics prior to this submission for the feasibility calls. Revisions were made to the list of topics based on input from the reviewers, and the list was used to develop an interview guide for the feasibility calls. In addition to staff from each of the three Divisions at NCES, the QRB also included staff from OCTAE and the Office of Planning, Evaluation, and Policy Development (OPEPD). The QRB members for this survey are listed below:

Braden Goetz, OCTAE

Sharon Lee Miller, OCTAE

Lul Tesfai, OCTAE

Kelly Fitzpatrick, OPEPD

Michael Fong, OPEPD (Policy and Program Studies Service)

Milagros Lanauze, OPEPD (Budget Service)

John Ralph, NCES (Annual Reports and Information)

Jing Chen, NCES (Assessment Division)

Heidi Silver-Pacuilla, NCES (Assessment Division)

Chris Chapman, NCES (Sample Surveys Division)

Sharon Boivin, NCES (Sample Surveys Division)

Lisa Hudson, NCES (Sample Surveys Division)

Elise Christopher, NCES (Sample Surveys Division)

Gigi Jones, NCES (Administrative Data Division)

Joseph Murphy, NCES (Administrative Data Division)

Kashka Kubzdela, NCES (Statistical Standards and Data Confidentiality)

Sample, Burden, and Cost

In this submission, we are requesting approval for feasibility calls with members of the target population. We will conduct up to three rounds of feasibility calls for the survey, with 15 or fewer respondents per round. School districts will be recruited to participate in feasibility calls based on various district characteristics including level (secondary or unified), size, urbanicity (locale), and geographic region. Respondents will be recruited by telephone and will be identified as the person in the district who is most familiar with CTE programs in their district.

Telephone interviewers will recruit participants for the feasibility calls using the recruitment script in Attachment 1. Interviewers will schedule an appointment to complete the feasibility calls with cooperating district-level personnel. Following telephone recruitment, interviewers will either email or fax a cover letter and draft questionnaire to the participating districts (as discussed below in the Data Collection Instrument section). In order to recruit 15 respondents per round, we anticipate contacting 45 public school districts per round (Table 1). On average, recruitment calls with respondents who agree to participate in the feasibility calls are expected to take about 10 minutes to explain the purpose of the call and set up an appointment to discuss the survey; all other recruitment calls are expected to take about 3 minutes. Prior to the feasibility calls, respondents will be asked to review (but not complete) either interview topics, draft questionnaire items, or a draft questionnaire, depending on the round of calls, which should take approximately 15 minutes. The feasibility call should take approximately 45 minutes to complete. The estimated burden for one round of feasibility calls is approximately 20 hours, and total estimated burden time for all three rounds of feasibility calls is approximately 60 hours. We anticipate that the estimated cost to the federal government will be approximately $6,000 for each round of feasibility calls.

Table 1. Estimated maximum burden time for up to three rounds of feasibility calls for FRSS 108

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Recruitment: Districts not participating in the feasibility call	30	30	0.05	2
Recruitment: Districts participating in the feasibility call	15	15	0.17	3
Survey review and feasibility call	15	15	1.0	15
Total per round	45	60	--	20
Total for three rounds	135	180	--	60

¹ Counts each contact (e.g., recruitment and feasibility call are counted separately even when they are with the same respondents).

Data Collection Instrument

For each round of feasibility calls, a cover letter and topic list, or in later rounds, draft questionnaire items or questionnaire, will be emailed or faxed to each participating school district. The cover letter and topic list for the first round of feasibility calls are included in this document as Attachment 2. The cover letter thanks the respondent for agreeing to participate in the feasibility call, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should review the topic list on which we will base the telephone discussion, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter, respondents are assured that their participation is voluntary and their answers may only be used for statistical purposes and may not be disclosed or used in identifiable form for any other purpose except as required by law. The law is cited on the cover letter. The materials for subsequent rounds of calls will be similar, except the questions for each round will include modifications or new items that resulted from the previous rounds.

Interview Guide: overview of topics and interview guide

The interview guide (see attachment 3) will be used to learn more about CTE programs at the secondary level in order to inform development of a questionnaire that is clear and relevant to respondents. Questions in the interview guide are based on criteria identified by OCTAE in their blueprint specifications for the most recent reauthorization of the Perkins Act as characteristics that define high-quality CTE programs. The interview guide will cover topics that include provision for career pathways; availability of CTE programs leading to industry-recognized credentials, or to postsecondary certificates or degrees; identification of in-demand industry sectors or occupations, and provision of skills required for entry into in-demand occupations; availability of work-based learning; and opportunities for CTE students to develop leadership skills.

Timeline

Feasibility call activities are expected to begin as soon as approval is received from OMB, and are anticipated to take about five months to complete, including up to three rounds of feasibility calls and the development of and revisions to a draft survey between each round.

Attachment 1: FRSS 108 Feasibility Call Recruitment Script

FRSS 108

Career and Technical Education Programs in Public School Districts Feasibility Call Recruitment Script

Hello, my name is __________________.

I am calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on career and technical education (CTE) programs in public school districts. I would like to ask the district-level person who is most familiar with CTE programs to provide us with information about the CTE programs offered by your district.

Who is the person in your district who is most knowledgeable about career and technical education programs offered by your district?

(This is often a director or coordinator of career and technical education programs at the district level.)

May I please speak to that person?

CONNECTED TO DISTRICT–LEVEL PERSON MOST FAMILIAR WITH CAREER AND TECHNICAL EDUCATION PROGRAMS IN THE DISTRICT

Hello, my name is __________________.

I’m calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on career and technical education (CTE) programs. We would like your help in developing a questionnaire about CTE programs in public school districts. Specifically, we would like you to participate in a telephone discussion in which we ask you about the CTE programs offered by your district. The call will take about 45 minutes.

1. How would you like me to send you the interview materials (email, fax)?

2. We ask that you review the interview materials before you talk to the survey manager. When would be a good time for the survey manager, Cindy Gray, to call you to discuss the interview topics and obtain your comments? How about [SUGGEST A TIME]. [Just to be sure, you are in the [Eastern, Central, Mountain, Pacific] time zone?]

3. What is the best telephone number at which the survey manager can reach you?

Thank you. Your insights will be very helpful.

Attachment 2: FRSS 108 Cover Letter

U.S. Department of Education  Institute of Education Sciences  National Center for Education Statistics

[Date] 2015

Dear Participant,

Thank you for agreeing to give us feedback on the development of a survey on career and technical education programs in public school districts. The National Center for Education Statistics (NCES), within the U. S. Department of Education is authorized to conduct this survey by the Education Science Reform Act of 2002 (ESRA, 20 U.S.C. §9543). Westat, a research company located in Rockville, Maryland, is administering this survey on behalf of NCES. As part of the survey development, we are looking for feedback from district-level personnel about topics and questions that might be included in the survey. Your input will be essential in ultimately developing a questionnaire that is clear and relevant, and not overly burdensome to respondents. All information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. § 9573).

We ask that you review the following list of interview topics prior to our telephone conversation.

1. The extent to which your district’s CTE programs are structured as career pathways;

2. Availability in your district of CTE programs that lead to industry-recognized credentials, or to postsecondary certificates or degrees;

3. Your district’s identification of in-demand industry sectors or occupations and how this impacts the CTE programs in your district;

4. The availability of work-based learning in your district’s CTE programs; and

5. Opportunities for students to develop leadership skills through your CTE programs.

My colleague and I will call you at the scheduled time to get your feedback on the materials and to discuss any comments or suggestions you may have. In the meantime, feel free to call me at Westat’s toll-free number, 800-937-8281, ext. 4336, if you have any questions. You may also reach me by email at [email protected].

Thank you for your much needed assistance!

Sincerely,

Cindy Gray

Westat Survey Manager

1 U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service (2014), National Assessment of Career and Technical Education: Final Report to Congress. Washington, DC.

2 U.S. Department of Education, Office of Career, Technical, and Adult Education, Investing in America’s Future: A Blueprint for Transforming Career and Technical Education (Summary) (2012). Washington, DC.