NAEP 2012 Charter School Pilot Questionnaire Study

National Assessment of Educational Progress

Volume II

Appendix A-C:

Letter to Pilot Charter Schools

Pilot Charter School Questionnaire

Phone Script and Question by Question Explanations

Request for Clearance for 2012 Charter School Pilot Questionnaire Study

OMB# 1850-0803 v.52

7/1/11

Appendix A: Letter to Pilot Charter Schools

January 2012

Dear [SCHOOL ADMINISTRATOR]:

On behalf of the National Center for Educational Statistics (NCES), part of the U.S. Department of Education, the National Assessment of Educational Progress, also known as NAEP, has developed new survey questions for charter school administrators. Before giving the new survey to all charter school administrators whose students participate in a NAEP assessment, NCES is testing the survey questions to evaluate the quality of the items. Your responses and feedback will help NCES improve the survey to collect the most valid and reliable information from educators like you.

This study involves collecting responses for 9 questions about your charter school. To facilitate the collection of these responses, a copy of the questions is enclosed. Within the next few weeks, a NAEP representative will be contacting you to collect the responses to these questions over the phone.

If you have any questions about the study or its procedures, please contact me at __________________.

Thank you in advance for your help on this very important study!

Sincerely,

NAEP Study Manager

Appendix B: Pilot Charter School Questionnaire

OMB Information on Questionnaire Cover Page

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0803. The time required to complete this information collection is estimated to average 30 minutes including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, D.C. 20202-4537. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: NAEP/NCES, U.S. Department of Education, 400 Maryland Avenue, S.W., Washington, DC 20202.

This is a project of the National Center for Education Statistics (NCES), part of the Institute of Education Sciences, within the U.S. Department of Education.

Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

OMB No. 1850-0803 Approval Expires 09/30/2013

Appendix C: Phone Script and QxQs for Pilot Charter School Questionnaire

Phone Script

Script language is in italics. Interviewer instructions are in normal text within text boxes. Informational text for the interviewer is in shaded text within text boxes.

1. Survey Introduction

Good morning/afternoon. May I please speak to <name>?

Good morning/afternoon. My name is <name> from Westat calling on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. I am calling at this time to conduct the pilot test of the NAEP charter school questionnaire. The survey will last approximately 15 minutes.

Is now a good time to conduct the survey?

If not, reschedule the interview for another time.

The National Assessment of Educational Progress, also known as NAEP, has developed new survey questions for charter school administrators. Before giving the new survey to all charter school administrators whose students participate in a NAEP assessment, NCES is testing the survey to evaluate the quality of the items. Your responses and feedback will help NCES improve the survey to collect the most valid and reliable information from educators like you.

Every effort should be made to recreate the conditions under which a respondent will conduct the survey as part of the operational assessment. Therefore, it is preferable that the respondent have the questionnaire in front of them. It is acceptable if the respondent has completed the questionnaire in advance. Moreover, for the safety of the respondent and to best recreate self-administration conditions, do not conduct an interview with a respondent who is driving or otherwise engaged in an activity that would prevent them from safely reviewing a print or electronic version of the questionnaire.

Did you receive the pilot charter school questionnaire in advance of this call?

Record response: (A) Yes—continue to next question in script, (B) No—Ask to e-mail the questionnaire to the respondent. Arrange to conduct the interview after respondent confirms receipt or if the respondent prefers, arrange a call back to conduct the interview.

Did you read through the pilot charter school questionnaire in advance of this call?

Record response: (A) Yes, (B) No

The previous question is being asked to collect data on whether respondents who review the questionnaire before the interview respond differently in the phone interview than those who received the questionnaire, but did not review it.

Do you have the pilot charter school questionnaire available to read while we conduct the pilot now?

Record response: (A) Yes—continue to “Script for Respondent with Questionnaire”, (B) No- Ask to e-mail the questionnaire to the respondent. Arrange to conduct the interview after respondent confirms receipt or if the respondent prefers, arrange a call back to conduct the interview.

2. Script for Respondent with Questionnaire

If respondent has the questionnaire available, then read question numbers and questions solely to structure the interview, but do not read any definitions or response choices.

Because you have the questionnaire available, I am only going to read the questions to you. Please read any definitions included with the question and the response choices. After you have reviewed the response choices, please give me a response to the question based on the answer choices provided. Let’s begin; number 1; is your school a public charter school?

Record response

Proceed to next appropriate question based on response and skip pattern instruction, where applicable. Read question number and question; record response; and continue this process through end of survey.

3. Survey Closing Script

You have finished the survey. If you don’t mind, we would appreciate any feedback that you have on the questionnaire. In particular, please describe any difficulty you encountered in understanding or answering any of the questions.

Record response

On behalf of the National Center for Education Statistics, I would like to thank you for your participation in our pilot of the charter school questionnaire. Your input, combined with input from other school administrators like you will help improve the survey questionnaires for the National Assessment of Educational Progress. Thank you for your time.

Pilot Charter School QxQs Introduction

QxQs are descriptions of the purpose of each question in the questionnaire along with definitions for potentially unfamiliar words or terms. QxQs are provided to assist administrators in answering respondent questions about questions in the questionnaire. Under standard conditions, QxQs are only provided for student questionnaires because students are in an environment with an available administrator who can assist them in selecting the most appropriate response when they ask for assistance. QxQs are typically not provided to teachers or school administrators because they do not have direct administrator assistance when they are completing the questionnaire. In addition, teachers and school administrators have the sophistication and familiarity with survey questionnaires to know that they should answer the questions to the best of their ability based on their perceived meaning of a word, question, or instruction. Nonetheless, if a respondent asks the interviewer a question about a question, QxQs have been provided for this pilot charter school questionnaire.

When conducting the interview, every effort should be made to recreate the conditions under which a respondent will conduct the survey as part of the operational assessment. Under the operational assessment, school administrators will not have QxQs. Therefore, interviewers should not offer a QxQ explanation unless a respondent explicitly asks for assistance to understand a word, term, or meaning associated with a question.

Pilot Charter School QxQs

To assist you in answering respondent questions, each question in the questionnaire section is repeated below, with an explanation of what information the question is attempting to obtain from the educators.

1. Is your school a public charter school?

(A charter school is a public school that, in accordance with an enabling state statute, has been granted a charter exempting it from selected state or local rules and regulations. A charter school may be a newly created school, or it may previously have been a public or private school.)

If the respondent says “no,” ask the respondent how he or she would describe their school type (e.g., regular public, public magnet, independent, parochial, etc.). If the school type is not a public charter school, end the interview and thank the respondent for their time. Then, speak with a supervisor to ascertain why the respondent’s school was sampled for the pilot study.

2. In which year did your school start providing instruction as a charter school?

The intent of the question is to learn the first year that the school began conducting classes as a charter school. This is not necessarily the first year the school was established because some schools were originally regular public, public magnet, or private schools before becoming charter schools.

3. Who granted your school’s current charter?

The intent of the question is to learn who granted the school’s charter. The "charter" establishing a charter school is a performance contract detailing the school's mission, program, goals, students served, methods of assessment, and ways to measure success. The length of time for which charters are granted varies, but most are granted for 3-5 years. At the end of the term, the entity granting the charter may renew the school's contract. Each state establishes their own law regarding the legal definition of a charter and which entities (e.g., local school district or state board of education) can grant a charter.

1. School district: A geographic district, the public schools of which are administered together.

2. State board of education: An elected or appointed body at the local or state level that supervises a given school system or systems.

3. Postsecondary institution: College, University or Trade/Vocational school.

4. State charter-granting agency: An agency or board that has been authorized by the state to accept or reject charter applications.

5. City or state public charter school board: A board of appointed or elected officials authorized by the state to monitor charter schools and accept or reject charter applications.

4. What is the legal status of your school?

The intent of the question is to learn whether or not the school is legally part of the public school district. Public school districts are by definition local education agencies. Local educational agency (LEA) can refer to a public school district, or in rural areas a body that oversees multiple schools. The responsibilities of an LEA may include operating the public school system, distributing grant money to school projects, and contracting for educational services. An LEA is recognized in a state as an administrative agency responsible for the public elementary schools and/or secondary schools in its jurisdiction.

1. Officially part of the school district or local education agency (LEA): This is where the charter school is considered part of the local school district, which is almost always the case when the school district grants the charter. The school is accountable and reports data, etc. to the local school district.

2. Independent from the school district or local education agency (LEA): This is where the charter school is independent of and not accountable to the school district. In this situation, the school is accountable and reports data, etc. to the state board of education or state charter-granting agency. This is often the case for cyber charters.

3. A separate local education agency (LEA) as stipulated by state law: This is where the charter school is accountable and reports data, etc. to a local education agency that is different from the school district where the charter is physically located. This is often the case for charter schools that are granted by postsecondary institutions or granted on appeal to the state after being rejected by a local school district.

5. Is this school operated by a company or organization that also operates other charter schools?

The intent of the question is to learn whether or not the school is independently run or is a “subsidiary” or “franchise” of schools that are operated by a single organization, often referred to as a public charter management organization.

6. What is the main focus of your mission as defined in your school’s charter? Fill in only one oval.

The intent of the question is to identify the primary purpose, goal, or mission as defined in the school’s charter. There is undoubtedly overlap among the response choices because many schools have multiple missions; for that reason, some respondents will be reluctant to choose only one as their main focus. Nonetheless, respondents should narrow it down to one main focus.

1. Developing core skills and meeting specified academic standards: Primary mission of school is to deliver educational instruction to address specific content or academic standards (regardless of who set the standard); for example, a charter that focuses content primarily (and almost exclusively) on reading and math.

2. Serving a specific student population: Primary mission of school is to serving the education needs of students that share a specific background or characteristic; for example, a charter school serving students with autism spectrum disorders.

3. Implementing a specific learning approach: Primary mission of school is to implement a specific learning approach such as an individualized (e.g., self-paced), or distance learning approach.

4. Following a specific curriculum: Primary mission of school is to instruct using a specific curriculum (e.g., STEM-focused curriculum).

7. Which one of the following best describes your charter school’s primary focus in terms of program content?

The intent of the question is to identify the primary focus of the instructional content provided in the school. There is undoubtedly overlap among the response choices because many schools have several goals with regard to program content; for that reason, some respondents will be reluctant to choose only one as their main focus. Nonetheless, respondents should narrow it down to one primary focus. An explicit instruction to fill in only one oval (as seen in the previous question) was not included in this question in order to maintain trend with how the question was presented in a previous operational administration.

The response choices are full explanatory sentences, thus no further information should be required for understanding.

8. Does your school provide a written contract for parents?

The intent of the question is to identify whether the school has a written contract that the school asks their students’ parents to sign. The contract can include, but is not limited to, describing the rules and regulations of the school, expected duties and commitments of parents, and other agreements between the school, parent, and child.

1. Yes, and parents are required to abide by it. : School requires a parent’s signature before a child is admitted and can attend the school.

2. Yes, but signing it is voluntary. : Student attendance is not dependent upon a parent signing the contract.

C. No: School has no contract for parents. > End the interview with the Survey Closing Script.

9. Are the following elements addressed in your charter-parent contract? Fill in one oval in each row.

The intent of the question is to identify what is in the written contract that the charter school asks parents to sign. The respondent is expected to select Yes or No for each sub-item.

Most of the response choices are self-explanatory, but there are a few that might be less understood and for those additional information has been provided.

2. Home learning environment: Expectations for the quality of the home with regards to learning. These expectations can include, but are not limited to, limitations on television and video games or maintaining a public library card for the child.

E. Parent volunteering: Amount of expected parental participation in helping out in the classroom or in the school.

H. Student promotion policy: Level of academic performance or passing requirements necessary for student to earn advancement to the next academic grade.

> End the interview with the Survey Closing Script.