Justification with Pretest Script and Letter

Volume I:

Fast Response Survey System (FRSS) 105: Condition of Public School Facilities Pretest

OMB# 1850-0803 v. 70

August 8, 2012

National Center for Education Statistics

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct two rounds of pretest calls for a Fast Response Survey System (FRSS) survey #105 on the condition of public school facilities. Congress has appropriated funds for NCES to conduct an FRSS survey on the condition of public school facilities, with a First Look report on the results to be released in late 2013. FRSS previously conducted a survey on this topic in 1999.

The purpose of the pretest calls is to identify and correct any potential issues with the content and format of the survey before conducting full scale implementation, and to ensure that the survey captures the intended meaning of each question, and minimizes the burden imposed on respondents. The request to conduct the full survey will be submitted at a later date under OMB generic clearance for quick response surveys (OMB#1850-0733), which are authorized under the Education Science Reform Act of 2002. NCES has contracted Westat for this survey.

Design

Overview of Survey Development

The 2012 FRSS survey will cover many of the same topics as the 1999 survey, but will use a revised questionnaire. The current survey reflects lessons learned from the 1999 survey, topics and issues identified through literature review, with modifications based on two rounds of feasibility calls with public school district personnel most knowledgeable about school facilities. A few items from the 1999 survey are included on the 2012 questionnaire for comparison. As was done in 1999, schools will be sampled, but surveys will be sent to districts, where facilities personnel and records are located. Two rounds of feasibility calls, each with nine respondents, were conducted in May and June 2012 (OMB# 1850-0803 v.67). The feasibility calls were used to explore potential new survey items, and identify and correct issues with the content and format of the survey before conducting the pretest. The resulting draft of the questionnaire was then reviewed by the NCES Quality Review Board (QRB) and revised accordingly to prepare it for the pretest. During the pretest, we will test all questions on the questionnaire and obtain estimates of the respondent time required to complete the survey.

NCES Review and Consultations Outside of Agency

The NCES QRB reviewed study materials on two occasions prior to the submission for the feasibility calls. QRB members first reviewed a descriptive paragraph about the study, and later reviewed a draft questionnaire. The questionnaire was also reviewed by the Office of Innovation and Improvement (OII) in the U.S. Department of Education. In addition, the questionnaire was sent to the Environmental Protection Agency for review. Revisions were made to the instrument, and a few new items were added based on input from the reviewers. The questionnaire was tested with respondents during feasibility calls. The questionnaire was then revised and again reviewed by the NCES QRB and OII.

Sample, Burden, and Cost

In this submission, we are requesting approval to conduct up to two rounds of pretest calls with the revised questionnaire, with 10 or fewer respondents from school districts around the nation in each round. Since facilities personnel and records are generally located at the district, we are recruiting district-level personnel to participate in the feasibility calls. Pretest participants will be recruited to participate based on various district characteristics including size, locale, and geographic region. Respondents will be recruited by telephone and will be identified as the person in the district who is most familiar with the school facilities in their district.

Telephone interviewers will recruit participants for the pretest calls using the recruitment script in attachment 1. Interviewers will schedule an appointment to complete the pretest calls with cooperating district-level personnel. Following telephone recruitment, interviewers will email a cover letter and draft questionnaire to participating districts (as discussed below in the Data Collection Instrument section). Respondents will be asked to review, complete, and fax back the paper and pencil questionnaire. They will also be invited to participate in a telephone debriefing.

In order to recruit 10 respondents per round, we anticipate contacting 40 public school districts (Table 1). On average, recruitment calls with respondents who agree to participate in the pretest calls are expected to take about 10 minutes to explain the purpose of the pretest and set up an appointment to discuss the questionnaire; all other recruitment calls are expected to take about 3 minutes. The questionnaire is expected to take respondents approximately 30 minutes to complete. The pretest debriefing interview is expected to take approximately 30 minutes. The total estimated burden is 26.4 hours for two rounds of pretest calls. We anticipate that the estimated cost to the federal government will be approximately $5,000 for each round of pretest calls.

Table 1. Maximum burden time for each of up to two rounds of pretest calls for FRSS 105

Respondents	Number of Respondents	Number of Responses1	Burden Hours per Respondent	Total Burden Hours
Each Round
Recruitment – Districts not participating in the pretest	30	30	0.05	1.5
Recruitment – Districts participating in the pretest	10	10	0.17	1.7
Pretest questionnaire debrief	10	10	1	10
Total per round	40	50	-	13.2
Total for two rounds	80	100	-	27

¹ Counts each contact (e.g., recruitment and debriefing interview with the same respondents are counted separately).

Data Collection Instrument

For each round of pretest, a cover letter and draft questionnaire will be emailed to each participating school district. The cover letter and questionnaire appear in attachments 2 and 3. The cover letter thanks the respondent for agreeing to participate in the pretest, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should complete the questionnaire and fax it back to Westat, includes questions for respondents to consider while completing the questionnaire, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter and on the cover of the survey, respondents are assured that their participation is voluntary and their answers may not be disclosed or used in identifiable form for any other purpose unless compelled by law. The public law is cited on the cover letter and the front page of the survey. The materials for the second pretest round (if necessary), will be similar, except the survey instrument will include the modifications that resulted from the previous round. The current instrument is discussed below.

Questionnaire

The questionnaire is designed to collect information on the condition of the building systems/features in permanent and portable (temporary) buildings in schools, and satisfaction with the building environmental factors that result from them. Respondents will be asked (as they were in 1999) for their estimate of the total cost of repairs/renovations/modernizations to put the school’s buildings in good overall condition, and on which sources (e.g., facilities inspections, capital improvement master plans) this estimate is based. They will be asked about plans for major repair or renovation or replacement of building features and systems, and about plans for construction at the school in the next few years. Additional items ask about the school’s long-range educational facilities plan, and steps taken to improve energy efficiency.

Question 1 asks whether the school has two types of onsite buildings -- permanent and portable (temporary) buildings. Responses to the question indicate which parts of questions 2 and 7 should be completed.

Question 2 lists 17 building systems/features and asks about the condition of each in the school’s permanent and portable (temporary) onsite buildings. The question includes a 4-point rating scale (excellent, good, fair, poor) and a “school does not have system/feature” option. Building features include things such as roofs, plumbing/lavatories, heating and air conditioning systems, electrical system, and life safety features. Part A asks about the condition of the various systems/features in the school’s permanent buildings and part B asks about the condition of the same systems/features in the school’s portable (temporary) buildings. Question 2 is a modified version of an item that was included in the 1999 survey.

Question 3 asks about the condition of various outdoor features at the school: school parking lots and roadways, bus lanes and drop-off areas, sidewalks and walkways, outdoor play areas/playgrounds, outdoor athletic facilities, covered walkways, and fencing. The question includes a 4-point rating scale (excellent, good, fair, poor) and a “school does not have feature” option.

Question 4 asks for an overall rating of the condition of the permanent and portable (temporary) onsite buildings at the sampled school. The question includes a 4-point rating scale (excellent, good, fair, poor) and a “school does not have building type” option. This is a modified version of an item that was included in the 1999 survey.

Question 5 asks for the best estimate of the total cost of all repairs/renovations/modernizations required to put the school’s onsite buildings in good overall condition. If the school’s onsite buildings are already in good or excellent overall condition, respondents are instructed to enter zero. This item was included in the 1999 survey.

Question 6 asks about the sources on which the cost estimate given in question 5 is based. This item was included in the 1999 survey.

Question 7 asks how satisfactory various environmental factors are in the school’s onsite buildings. Environmental factors included are artificial and natural lighting, heating, air conditioning, ventilation, indoor air quality, water quality, and acoustics or noise control. Satisfaction is rated separately for permanent and portable (temporary) buildings. This is a modified version of an item that was included in the 1999 survey.

Question 8 asks in what year the school’s main instructional building was constructed, and question 9 asks in what year the last major renovation of the main instructional building took place. Both items were included in the 1999 survey. Question 10 asks in what year the last major building replacement or addition was made to the school. These items provide information about the functional age of the school.

Question 11 asks whether any major repair/renovation/modernization work is currently being performed at the school. Question 12 asks whether any major repair/renovation/modernization work for the school is currently under contract, but not yet begun. These items provide information about current and future work to improve school conditions.

Question 13 asks which kinds of construction projects, if any, are planned for the school in the next 2 years. The construction projects include building new permanent buildings or permanent additions to buildings, and major repairs, renovations, or modernization of existing permanent buildings. This is a modified version of an item that was included in the 1999 survey.

Question 14 lists the same 17 building systems/features used in Question 2, and asks which, if any, have major repairs, renovations, or replacements planned for the next 2 years. If major repairs, renovations or replacements are planned, part B asks for the main reason for this work.

Question 15 asks if there is a written long-range educational facilities plan for the school. This item was included in the 1999 survey.

Question 16 asks if there is a facilities plan that specifies various actions for the school: specific ways to improve energy efficiency, specific ways to improve environmental conditions, and creating green schools.

Question 17 asks about the use of qualified professionals within the last 5 years to perform inspection of the condition of the physical features of the facility, evaluation of energy use, and evaluation of indoor environmental hazards.

Question 18 asks about actions undertaken within the last 5 years to improve energy efficiency at the school. Actions included are replacing lighting fixtures, lighting ballasts, or bulbs; installing motion-sensors for lighting; upgrading insulation, outer walls, and/or siding; replacing windows and/or doors; installing or upgrading reflective roof coating; installing more efficient HVAC systems; and installing or upgrading energy management system.

Question 19 asks whether there are significant problems with the facilities at the school that are not covered in this survey. If the response is “yes,” space is provided to describe the problems.

Timeline

Pretest activities are expected to begin as soon as approval is received from OMB. It is anticipated that recruitment, debriefing, write up of the memorandum summarizing the results, and survey revisions will take approximately 6 weeks for each pretest round.

Attachment 1: FRSS 105 Pretest Call Recruitment Script

FRSS 105

FRSS 105 Condition of Public School Facilities

Pretest Call Recruitment Script

Hello, my name is __________________.

I am calling from Westat on behalf of the U.S. Department of Education regarding a survey on the condition of public school facilities. We would like your help in identifying the district-level person who is most familiar with school facilities in your district.

Who at your district is most knowledgeable about your school facilities?

(RECORD CONTATCT INFORMATION)

May I please speak to that person?

CONNECTED TO DISTRICT–LEVEL PERSON MOST FAMILIAR WITH SCHOOL FACILITIES IN THE DISTRICT

Hello, my name is __________________.

I’m calling from Westat on behalf of the U.S. Department of Education regarding a survey on the condition of public school facilities. We would like your help in reviewing our draft questionnaire to ensure that it is clear and easy to complete. Specifically, we would like you to complete a questionnaire and then obtain your comments about the survey by telephone. This is a short questionnaire which should take about 30 minutes to complete. [The call will take about 30 minutes.]

Your input, while voluntary, will be essential in developing a questionnaire that is relevant. All information you will provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable from for any other purpose unless otherwise compelled by law (Education Sciences Reform Act of 2002, 20 U.S.C. 9573.)

1. May I have your email address to send you the survey materials?

2. We ask that you complete the questionnaire and fax it back to us before you talk to the survey manager. When would be a good time for the survey manager, Debbie Alexander, to call you to discuss the survey and obtain your comments? How about [SUGGEST A TIME]. [Just to be sure, you are in the [Eastern, Central, Mountain, Pacific] time zone?]

3. What is the best telephone number for the survey manager to reach you on?

Thank you. Your insights will be very helpful.

Attachment 2: FRSS 105 Pretest Cover Letter

August 2012

Dear Pretest Participant:

Thank you for agreeing to participate in the preset of the survey on the condition of public school facilities. Westat, a research company located in Rockville, Maryland, is conducting this survey for the National Center for Education Statistics (NCES), U. S. Department of Education. The survey is part of the Fast Response Survey System (FRSS), a system charged with collecting information on important and emerging issues related to education. The survey is designed to be completed by district personnel most knowledgeable about school facilities in their district. When the survey is conducted in January 2013, individual schools will be sampled, but the questionnaire will be sent to the district in which the school is located. The purpose of the survey is to provide nationally representative data about the current condition of facilities in U.S. public schools.

We are currently conducting the pretest of the survey. Your input, while voluntary, will be essential in developing a survey that is relevant, clear, and not overly burdensome to respondents. Your participation is very important because your comments will improve the survey before the actual data collection begins. Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose unless otherwise compelled by law (Education Sciences Reform Act of 2002, 20 U.S.C. 9573.)

We ask that you (1) complete the questionnaire and fax it to Westat; (2) keep track of the time you spend filling out the questionnaire; (3) write down any comments about the questionnaire; and (4) discuss your comments with me by telephone at the time scheduled. Please keep the following questions in mind as you complete the questionnaire:

1. Are the instructions and definitions clear and helpful to you?

2. Are the survey questions clear and easy to interpret?

3. Would you have access to the information necessary for answering these questions?

Please fax the completed questionnaire to me. My toll-free fax number is 1-800-254-0984. My colleague and I will call you at the scheduled time to get your feedback on the questionnaire and discuss any comments or suggestion you may have about the study. In the meantime, feel free to call me at Westat’s toll-free number, 800-937-8281, ext. 2088, if you have any questions. You may also reach me by email at [email protected].

Again, thank you for your much needed assistance!

Sincerely,

Debbie Alexander

Westat Survey Manager