Justification

Volume I:

Fast Response Survey System (FRSS) 109: Public School Teachers’ Use of Digital Learning Resources (DLR) for Teaching – Feasibility Calls

OMB# 1850-0803 v.202

July 31, 2017

National Center for Education Statistics (NCES)

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), within the Institute of Education Sciences (IES) of the U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct feasibility calls for Fast Response Survey System (FRSS) survey #109 on public school teachers’ use of digital learning resources (DLR) for teaching. NCES is conducting this FRSS survey as part of the IES response to the request in the Every Student Succeeds Act of 2015 (ESSA 2015, 20 U.S.C. §6301 et seq.) to provide information about the educational impact of access to DLR outside of the classroom.

The expanding use of technology affects the lives of students both inside and outside the classroom. For this reason, the role of technology in education is an increasingly important area of research. While access to technology can provide valuable learning opportunities to students, technology by itself does not guarantee successful outcomes. Schools and teachers play an important role in successfully integrating technology into teaching and learning.

ESSA provides guidance to state governments on how to receive supplemental federal funding for public education. As part of the ESSA legislation, IES is required to produce a report on the educational impact of access to digital learning resources (DLR) outside of the classroom. Specifically, ESSA requests that IES conduct research in the five areas listed below.

1. An analysis of student habits related to digital learning resources outside of the classroom, including the location and types of devices and technologies that students use for educational purposes.

2. An identification of the barriers students face in accessing DLR outside of the classroom.

3. A description of the challenges that students who lack home internet access face, including challenges related to student participation and engagement in the classroom and homework completion.

4. An analysis of how the barriers and challenges such students face impact the instructional practice of educators.

5. A description of the ways in which state education agencies, local education agencies, schools, and other entities, including partnerships of such entities, have developed effective means to address the barriers and challenges students face in accessing DLR outside of the classroom.

ESSA refers to the term “digital learning” as “any instructional practice that effectively uses technology to strengthen a student’s learning experience and encompasses a wide spectrum of tools and practices” (20 U.S.C. §7112 Definitions). However, for this survey, the main focus of digital learning resources will be computers (e.g., laptops, desktops, notebooks), mobile devices (e.g., smart phones, tablets), and broadband internet access.

The purpose of this FRSS 109 survey is to collect nationally representative data from public school teachers about their use of DLR for teaching, and how their knowledge and beliefs about their students’ access to DLR outside the classroom affect the assignments they give. The survey will focus on information that can best be provided by teachers from their unique perspective and direct interaction with students.

The purpose of feasibility calls is to explore topics for potential survey items, identify and correct any potential issues with the content and format of the survey before conducting pretests, and to ensure that the survey captures the intended meaning of the questions and minimizes the burden imposed on respondents. A request to conduct pretest activities will follow completion of the feasibility calls. Early rounds of the feasibility calls will ask teacher respondents to participate in a telephone discussion about their use of DLR for teaching, and what they know about their students’ access to DLR outside of school. This information will be used to develop the questionnaire. In later rounds of the feasibility calls, respondents will be asked to review, but not complete draft questionnaire items and ultimately, the draft questionnaire, and provide feedback by telephone. The pretests will involve asking respondents to complete the draft survey and participate in a telephone debriefing. Feasibility calls will be done before pretests to minimize the burden on respondents. Pretests will be done as a final test prior to OMB clearance submission requesting to conduct the full-scale survey. The request to conduct the full-scale survey will be submitted at a later date as a renewal under the OMB clearance for FRSS teacher technology surveys (OMB#1850-0857), which NCES is authorized to conduct by the Education Sciences Reform Act of 2002 (ERSA 2002, 20 U.S.C. §9543). NCES has contracted Westat for all stages of the FRSS 109 survey.

Design

Overview of Survey Development

NCES has contracted Westat to prepare for and administer FRSS 109, including development of the survey instrument. FRSS has established procedures for developing short surveys on a wide variety of topics. The techniques that we plan to use to shape the survey design on FRSS 109 include input from the NCES Quality Review Board (QRB), several rounds of feasibility calls, and up to two pretests. The specific ways we plan to use feasibility calls are discussed below.

We anticipate conducting up to three rounds of feasibility calls, each with fifteen or fewer respondents. With new surveys such as the teacher use of DLR survey, the initial feasibility calls use an open-ended interview guide rather than a questionnaire. As rounds of feasibility calls progress, respondents will be asked to review, but not complete draft questionnaire items and ultimately a draft questionnaire. Conducting multiple rounds of feasibility calls will systematically inform us about public school teachers’ use of DLR for teaching and what they know about their students’ access to DLR outside of school. The gathered information will be used to draft a questionnaire, and in later rounds will provide in-depth information on respondents’ perceptions of the draft survey and response burden. The process will result in several iterations of the questionnaire items.

The first round of calls will focus on learning more about teachers’ knowledge and beliefs about their students’ access to technology and the Internet for doing school assignments outside of school, and how that influences the assignments they give; barriers and challenges they think their students face in using technology and the Internet for class assignments outside of school; how teachers use technology for instructional management practices and communication with students and/or parents; and digital devices and software that the district or school make available to the teachers’ students for use outside of class time. For the next round of feasibility calls, respondents will be asked to review draft survey questions, instructions, and definitions that will be developed based on the initial round of feasibility calls. As a result of the feedback we receive, we will make any necessary changes to the survey items and draft a questionnaire. In the third and final round of feasibility calls, we will ask for respondents’ feedback on the draft questionnaire. The resulting draft of the survey will be reviewed by the NCES QRB and revised as necessary to prepare it for pretesting.

NCES Review and Consultations Outside of Agency

The NCES QRB members reviewed a draft list of questionnaire and discussion topics prior to this request for the feasibility calls. Revisions were made to the list of topics based on input from the reviewers, and the list was used to develop an interview guide for the feasibility calls (Attachment 3). In addition to staff from NCES’s Statistical Standards group, the Annual Reports group, and each of the three Divisions, the QRB also included staff from ED’s Office of Educational Technology (OET) and the Policies and Programs Studies Service of the Office of Planning, Evaluation, and Policy Development (OPEPD); the U.S. Commerce Department’s National Telecommunication and Information Administration; and the IBM Center for The Business of Government. The QRB members for this survey are listed below:

Rafi Goldberg, National Telecommunications and Information Administration, Commerce

James Collins, Office of Educational Technology

Andrew Abrams, OPEPD (Policy and Program Studies Service)

Dan Chenok, the IBM Center for The Business of Government

Halima Adenegan, NCES (Assessment Division)

Jamie Deaton, NCES (Assessment Division, NAEP)

John Ralph, NCES (Annual Reports and Information)

Tom Snyder, NCES (Annual Reports and Information)

Mark Glander, NCES (Administrative Records Division, CCD)

Chris Chapman, NCES (Sample Surveys Division, Longitudinal Branch)

Amy Ho, NCES (Sample Surveys Division, Cross-sectional Surveys Branch)

Marilyn Seastrom, NCES (Statistical Standards and Data Confidentiality)

Kashka Kubzdela, NCES (Statistical Standards and Data Confidentiality)

Sample, Burden, and Cost

In this submission, we are requesting approval for feasibility calls with members of the target population. We will conduct up to three rounds of feasibility calls for the survey, with 15 or fewer respondents per round. Teachers will be recruited to participate in feasibility calls based on various school characteristics including level (elementary or secondary), size, urbanicity (locale), and geographic region. Respondents will be recruited by telephone and will be identified as a regular self-contained classroom teacher at the elementary level or a teacher of a core academic subject at the secondary level.

Telephone interviewers will recruit participants for the feasibility calls using the recruitment script in Attachment 1. Interviewers will schedule an appointment to complete the feasibility calls with cooperating teachers. Following telephone recruitment, interviewers will either email or fax a cover letter and draft questionnaire to the participating teachers (as discussed below in the Data Collection Instrument section). In order to recruit 15 respondents per round, we anticipate contacting 45 public schools per round (Table 1). On average, recruitment calls with respondents who agree to participate in the feasibility calls are expected to take about 10 minutes to explain the purpose of the call and set up an appointment to discuss the survey; all other recruitment calls are expected to take about 3 minutes. Prior to the feasibility calls, respondents will be asked to review (but not complete) either interview topics, draft questionnaire items, or a draft questionnaire, depending on the round of calls, which should take approximately 15 minutes. The feasibility call should take approximately 45 minutes to complete. The estimated burden for one round of feasibility calls is approximately 20 hours, and total estimated burden time for all three rounds of feasibility calls is approximately 60 hours. We anticipate that the estimated cost to the federal government will be approximately $6,000 for each round of feasibility calls.

Table 1. Estimated maximum burden time for up to three rounds of feasibility calls for FRSS 109

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Recruitment: Teachers not participating in the feasibility call	30	30	0.05	2
Recruitment: Teachers participating in the feasibility call	15	15	0.17	3
Survey review and feasibility call	15	15	1.0	15
Total per round	45	60	--	20
Total for three rounds	135	180	--	60

¹ Counts each contact (e.g., recruitment and feasibility call are counted separately even when they are with the same respondents).

Data Collection Instrument

For each round of feasibility calls, a cover letter and topic list, or in later rounds, draft questionnaire items or questionnaire, will be emailed or faxed to each participating teacher. The cover letter and topic list for the first round of feasibility calls are included in this document as Attachment 2. The cover letter thanks the respondent for agreeing to participate in the feasibility call, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should review the topic list on which we will base the telephone discussion, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter, respondents are assured that their participation is voluntary and their answers may only be used for statistical purposes and may not be disclosed or used in identifiable form for any other purpose except as required by law. The law is cited on the cover letter. The materials for subsequent rounds of calls will be similar, except the questions for each round will include modifications or new items that resulted from the previous rounds.

Interview Guide: overview of topics and interview guide

The interview guide (see Attachment 3) will be used to learn more about teachers’ use of DLR for teaching in order to inform development of a questionnaire that is clear and relevant to respondents. Questions in the interview guide are based on the research areas specified by ESSA and input from the initial meeting of the NCES QRB. The interview guide will cover topics that include how knowledgeable teachers think they are about their students’ access to technology and the Internet for doing school assignments outside of school; teachers’ estimation of the amount of access and availability their students have to technology and the Internet for doing school assignments outside of school; the extent to which teachers’ sense of their students’ access to technology and the Internet outside of school influences the assignments they give; the kinds of technology-based homework assignments teachers give their students; the kinds of barriers or challenges that teachers think their students face in using technology and the Internet for class assignments outside of school; whether their state, district, or school has policies about teachers’ use of technology and the Internet for homework assignments; whether they use technology for instructional management practices and to communicate with students and parents; and whether their district or school makes digital devices or technology such as software available to their students for use outside of class time.

Timeline

Feasibility call activities are expected to begin as soon as approval is received from OMB, and are anticipated to take about five months to complete, including up to three rounds of feasibility calls and the development of and revisions to a draft survey between each round.

Attachment 1: FRSS 109 Feasibility Call Recruitment Script

FRSS 109: Public School Teachers’ Use of Digital Learning Resources for Teaching

Feasibility Call Recruitment Script

Hello, my name is __________________.

I am calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on public school teachers’ use of technology and the Internet. We are in the early stages of developing this survey, and would like to talk to one of your teachers about how they use technology and the Internet for teaching and for communicating with students and parents. This input will help us develop a survey that makes sense to teachers and which they can easily answer.

Can you give me the name and school email address of a few of your teachers who might be interested in talking to us about this important survey? We would like to talk to [a regular self-contained classroom teacher {FOR ELEMENTARY SCHOOLS} / a teacher who teaches English, history or social studies, math, or science {FOR SECONDARY SCHOOLS}].

Is this phone number the best number on which to reach these teachers? When would be the best time to call?

SPEAKING TO A TEACHER

Hello, my name is __________________.

I’m calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on teachers’ use of technology and the Internet. We would like your help in developing a questionnaire about how teachers use technology and the Internet for teaching and for communicating with students and parents. Specifically, we would like you to participate in a telephone discussion in which we ask you some questions about these topics and get your input about what questions make sense to ask teachers. The call will take about 45 minutes.

1. How would you like me to send you the interview materials (email, fax)?

2. We ask that you review the interview materials before you talk to the survey managers. When would be a good time for the survey managers, Cindy Gray and Laurie Lewis, to call you to discuss the interview topics and obtain your comments? How about [SUGGEST A TIME]. [Just to be sure, you are in the [Eastern, Central, Mountain, Pacific] time zone?]

3. What is the best telephone number at which the survey managers can reach you?

Thank you. Your insights will be very helpful.

Attachment 2: FRSS 109 Cover Letter

U.S. Department of Education  Institute of Education Sciences  National Center for Education Statistics

[Date] 2017

Dear Participant,

Thank you for agreeing to give us feedback on the development of a survey on public school teachers’ use of technology and the Internet for teaching. The National Center for Education Statistics (NCES), within the U. S. Department of Education is authorized to conduct this survey by the Education Sciences Reform Act of 2002 (ERSA 2002, 20 U.S.C. §9543). Westat, a research company located in Rockville, Maryland, is administering this survey on behalf of NCES. As part of the survey development, we are looking for feedback from teachers about topics and questions that might be included in the survey. Your input will be essential in ultimately developing a questionnaire that is clear and relevant, and not overly burdensome to respondents. All of the information you provide is voluntary and may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

We ask that you review the following list of interview topics prior to our telephone conversation.

1. What kinds of technology-based homework assignments do you give your students?

2. How much access and availability for doing school assignments do you think your students have to technology and the Internet outside of school?

3. To what extent does your sense of your students’ access to technology and the Internet outside of school influence the assignments you give?

4. What kinds of barriers or challenges do you think your students face in using technology and the Internet for class assignments outside of school?

5. Does your state, district, or school have policies about teachers’ use of technology and the Internet for homework assignments?

6. Do you use technology for instructional management practices (e.g., an online gradebook or dropbox for turning in assignments), and to communicate with students and parents?

7. Does your district or school make digital devices (e.g., computers or tablets) or technology such as software available to your students for use outside of class time?

My colleague and I will call you at the scheduled time to get your feedback on the materials and to discuss any comments or suggestions you may have. In the meantime, feel free to call me at Westat’s toll-free number, 800-937-8281, ext. 4336, if you have any questions. You may also reach me by email at [email protected].

Thank you for your much needed assistance!

Sincerely,

Cindy Gray

Westat Survey Manager