Volume 1 FRSS 110 Ed Technology in Public Schools Feasibility Calls

Volume I:

Fast Response Survey System (FRSS) 110: Use of Educational Technology for Instruction in Public Schools – Feasibility Calls

OMB# 1850-0803 v. 244

January 2019

National Center for Education Statistics (NCES)

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), within the Institute of Education Sciences (IES) of the U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct feasibility calls for Fast Response Survey System (FRSS) survey #110 on use of educational technology for instruction in public schools. The Office of Educational Technology, ED, requested that NCES conduct this FRSS survey.

The expanding use of technology affects the lives of students both inside and outside the classroom. For this reason, the role of technology in education is an increasingly important area of research. While access to technology can provide valuable learning opportunities to students, technology by itself does not guarantee successful outcomes. Schools and teachers play an important role in successfully integrating technology into teaching and learning. The purpose of this FRSS 110 survey is to collect nationally representative data from public schools about their use of educational technology for instruction.

The purpose of feasibility calls is to explore topics for potential survey items, identify and correct any potential issues with the content and format of the survey before conducting pretests, and to ensure that the survey captures the intended meaning of the questions and minimizes the burden imposed on respondents. A request to conduct pretest activities will follow completion of the feasibility calls. Early rounds of the feasibility calls will ask school respondents to participate in a telephone discussion about their schools’ use of educational technology for instruction. This information will be used to develop the questionnaire. In later rounds of the feasibility calls, respondents will be asked to review, but not complete draft questionnaire items and ultimately, the draft questionnaire, and provide feedback by telephone. The pretests will involve asking respondents to complete the draft survey and participate in a telephone debriefing. Feasibility calls will be done before pretests to minimize the burden on respondents. Pretests will be done as a final test prior to OMB clearance submission requesting to conduct the full-scale survey. The request to conduct the full-scale survey will be submitted at a later date under the OMB generic clearance for quick response surveys (OMB#1850-0733), which are authorized under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543).

Design

Overview of Survey Development

NCES has contracted Westat to prepare and administer FRSS 110, including development of the survey instrument. FRSS has established procedures for developing short surveys on a wide variety of topics. The techniques that are planned to shape the survey design on FRSS 110 include input from the NCES Quality Review Board (QRB), several rounds of feasibility calls, and up to two pretests. The specific ways that feasibility calls will be used are discussed below.

We anticipate conducting up to three rounds of feasibility calls, each with fifteen or fewer respondents. With new surveys such as the FRSS 110 survey on school use of educational technology, the initial feasibility calls use an open-ended interview guide rather than a questionnaire. As rounds of feasibility calls progress, respondents will be asked to review, but not complete draft questionnaire items and ultimately a draft questionnaire. Conducting multiple rounds of feasibility calls will systematically inform us about public schools’ use of educational technology for instruction. The gathered information will be used to draft a questionnaire, and in later rounds will provide in-depth information on respondents’ perceptions of the draft survey and response burden. The process will result in several iterations of the questionnaire items.

The first round of calls will focus on learning more about the computers available at the school for student and teacher use (including whether the school has a 1:1 computer program), Internet access at the school, whether there is a staff member at the school whose job it is to support teachers with integrating technology into instruction, whether there is a staff member at the school whose job it is to provide technical support for educational technology at the school (e. g., troubleshooting/maintenance for hardware, software, or networks), the type and amount of professional development regarding educational technology that the school or district provides to teachers at the school, whether any courses at the school use online textbooks or other online resources, barriers or challenges faced by students and teachers in the school in using educational technology, and whether there are policies about technology use for school assignments. For the second round of feasibility calls, respondents will be asked to review draft survey questions, instructions, and definitions that will be developed based on the initial round of feasibility calls. As a result of the feedback we receive, we will make any necessary changes to the survey items and draft a questionnaire. In the third and final round of feasibility calls, we will ask for respondents’ feedback on the draft questionnaire. The resulting draft of the survey will be reviewed by the NCES QRB and revised as necessary to prepare it for pretesting.

NCES Review and Consultations Outside of Agency

The NCES QRB members reviewed a draft list of questionnaire and discussion topics prior to this request for the feasibility calls. Revisions were made to the list of topics based on input from the reviewers, and the list was used to develop an interview guide for the feasibility calls (Attachment 3). In addition to staff from NCES’s Statistical Standards group, the Annual Reports group, and each of the three Divisions, the QRB also includes staff from ED’s Office of Educational Technology (OET); the U.S. Commerce Department’s National Telecommunication and Information Administration; the National Science Foundation; and four education technology organizations. The QRB members for this survey are listed below:

Rafi Goldberg, National Telecommunications and Information Administration, Commerce

Lee Zia, National Science Foundation

Bernadette Adams, Office of Educational Technology

Halima Adenegan, NCES (Assessment Division; ED Tech Equity Initiative)

Tom Snyder, NCES (Annual Reports and Information)

Ross Santy, NCES (Administrative Records Division)

Chris Chapman, NCES (Sample Surveys Division,

Kashka Kubzdela, NCES (Statistical Standards and Data Confidentiality)

Maria Worthen, iNACOL

Christina Luke, Digital Promise

Susan Bearden, CoSN

Ji Soo Song, ISTE

Sample, Burden, and Cost

In this submission, we are requesting approval for feasibility calls with members of the target population. We will conduct up to three rounds of feasibility calls for the survey, with 15 or fewer respondents per round. School personnel will be recruited to participate in feasibility calls based on various school characteristics including level (elementary or secondary), size, urbanicity (locale), and geographic region. Respondents will be recruited by telephone and will be identified as the person at the school most knowledgeable about the use of educational technology for instruction.

Telephone interviewers will recruit participants for the feasibility calls using the recruitment script in Attachment 1. Interviewers will schedule an appointment to complete the feasibility calls with cooperating school personnel. Following telephone recruitment, interviewers will either email or fax a cover letter and draft questionnaire topics or items to the participating school staff member (as discussed below in the Data Collection Instrument section). In order to recruit 15 respondents per round, we anticipate contacting 45 public schools per round (Table 1). On average, recruitment calls with respondents who agree to participate in the feasibility calls are expected to take about 10 minutes to explain the purpose of the call and set up an appointment to discuss the survey; all other recruitment calls are expected to take about 3 minutes. Prior to the feasibility calls, respondents will be asked to review (but not complete) either interview topics, draft questionnaire items, or a draft questionnaire, depending on the round of calls, which should take approximately 15 minutes. The feasibility call should take approximately 45 minutes to complete. The estimated burden for one round of feasibility calls is approximately 20 hours, and total estimated burden time for all three rounds of feasibility calls is approximately 60 hours. We anticipate that the estimated cost to the federal government will be approximately $6,000 for each round of feasibility calls.

Table 1. Estimated maximum burden time for up to three rounds of feasibility calls for FRSS 110

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Recruitment: Schools not participating in the feasibility call	30	30	0.05	2
Recruitment: Schools participating in the feasibility call	15	15	0.17	3
Survey review and feasibility call	15	15	1.0	15
Total per round	45	60	--	20
Total for three rounds	135	180	--	60

¹ Counts each contact (e.g., recruitment and feasibility call are counted separately even when they are with the same respondents).

Data Collection Instrument

For each round of feasibility calls, a cover letter and topic list, or in later rounds, draft questionnaire items or questionnaire, will be emailed or faxed to each participating school staff member. The cover letter and topic list for the first round of feasibility calls are included in this document as Attachment 2. The cover letter thanks the respondent for agreeing to participate in the feasibility call, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should review the topic list on which we will base the telephone discussion, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter, respondents are assured that their participation is voluntary and their answers may only be used for statistical purposes and may not be disclosed or used in identifiable form for any other purpose except as required by law. The law is cited on the cover letter. The materials for subsequent rounds of calls will be similar, except the questions for each round will include modifications or new items that resulted from the previous rounds.

Interview Guide: overview of topics and interview guide

The interview guide (see Attachment 3) will be used to learn more about the computers available at the school for student and teacher use (including whether the school has a 1:1 computer program), Internet access at the school, whether there is a staff member at the school whose job it is to support teachers with integrating technology into instruction, whether there is a staff member at the school whose job it is to provide technical support for educational technology at the school (e. g., troubleshooting/maintenance for hardware, software, or networks), the type and amount of professional development regarding educational technology that the school or district provides to teachers at the school, whether any courses at the school use online textbooks or other online resources, barriers or challenges faced by students and teachers in the school in using educational technology, and whether there are policies about technology use for school assignments.

Cost to the Federal Government

The total estimated cost to the federal government for FRSS 110 is $793,665 (see Table 2). The estimated cost to the federal government for the feasibility calls requested in this submission is $18,000.

Table 2. Cost to the federal government for all FRSS 110 activities

FRSS 110 Activity	Cost
Feasibility calls (3 rounds x $6,000 per round)	$18,000
Pretest calls (2 rounds x $6,000 per round)	$12,000
Total survey development work (2 clearances under 1850-0803)	$30,000
Preliminary activities/research applications for national data collection	$200,000
Remaining activities national data collection (data collection, analysis and reporting, data files, project management)	$563,665
Total national data collection (2 clearances under $1850-0733)	$763,665
Total cost for FRSS 110	$793,665

Timeline

Feasibility call activities are expected to begin as soon as approval is received from OMB, and are anticipated to take about six months to complete, including up to three rounds of feasibility calls and the development of and revisions to a draft survey between each round.

Attachment 1: FRSS 110 Feasibility Call Recruitment Script

FRSS 110: School Use of Educational Technology for Instruction

Feasibility Call Recruitment Script

Hello, my name is __________________.

I am calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on school use of educational technology for instruction. I would like to ask the school-level person who is most familiar with education technology use at the school to provide us with information about how your school uses technology for instruction.

Who is the person at your school who is most knowledgeable about how your school uses educational technology for instruction?

(This is often the school principal or a school-level education technology coordinator.)

May I please speak to that person?

CONNECTED TO SCHOOL–LEVEL PERSON MOST FAMILIAR WITH EDUCATIONAL TECHNOLOGY USE FOR INSTRUCTION AT THE SCHOOL.

Hello, my name is __________________.

I’m calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education, regarding a survey on school use of educational technology for instruction. We would like your help in developing a questionnaire about instructional use of educational technology in public schools. Specifically, we would like you to participate in a telephone discussion in which we ask you about how your school uses educational technology for instructional purposes. The call will take about 45 minutes.

1. How would you like me to send you the interview materials (email, fax)?

2. We ask that you review the interview materials before you talk to the survey managers. When would be a good time for the survey managers, Laurie Lewis and Cindy Gray, to call you to discuss the interview topics and obtain your comments? How about [SUGGEST A TIME]. [Just to be sure, you are in the [Eastern, Central, Mountain, Pacific] time zone?]

3. What is the best telephone number at which the survey managers can reach you?

Thank you. Your insights will be very helpful.

Attachment 2: FRSS 110 Cover Letter

U.S. Department of Education  Institute of Education Sciences  National Center for Education Statistics (NCES)

[Date] 2019

Dear Participant,

Thank you for agreeing to give us feedback on the development of a survey on the use of educational technology for instruction in U.S. public schools. The study is being conducted by the National Center for Education Statistics (NCES) in partnership with the Office of Educational Technology, U. S. Department of Education. NCES is authorized to conduct this survey by the Education Sciences Reform Act of 2002 (ERSA 2002, 20 U.S.C. §9543). Westat, a research company located in Rockville, Maryland, is administering this survey on behalf of NCES. As part of the survey development, we are looking for feedback from school administrators and school technology coordinators about topics and questions that might be included in the survey. Your input will be essential in ultimately developing a questionnaire that is clear and relevant, and not overly burdensome to respondents. All of the information you provide is voluntary and may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

We ask that you think about the following list of interview topics prior to our telephone conversation.

1. Information about the computers available at your school for student and teacher instructional use (including whether your school has a 1:1 computer program);

2. Information about the type of Internet access available in the instructional areas of your school;

3. Whether there is a staff member at your school whose job it is to support teachers with integrating technology into instruction;

4. Whether there is a staff member at your school whose job it is to provide technical support for educational technology at your school (e.g., troubleshooting/maintenance for hardware, software, or networks);

5. The type and amount of professional development regarding educational technology that your school or district provides to teachers at your school;

6. Whether any courses at your school use online textbooks or other online resources, or use various emerging technologies (e.g., virtual reality, augmented reality, or games and simulations);

7. Barriers or challenges faced by students and teachers in your school in using educational technology; and

8. Whether there are policies about technology use for school assignments.

My colleague and I will call you at the scheduled time to get your feedback on the materials and to discuss any comments or suggestions you may have. In the meantime, feel free to call me at Westat’s toll-free number, 800-937-8281, ext. 8284, if you have any questions. You may also reach me by email at [email protected].

Thank you for your much needed assistance!

Sincerely,

Laurie Lewis

Westat Survey Manager