Responses to OMB Wave 2 (9-7)

Responses to OMB FOLLOW-UP QUESTIONS/COMMENTS (New information is in red)

4^th grade Science questionnaire, original passback question 2

1. The answer was specific to one suggestion but we would still like more global answer to the question about the degree of pretesting of the new items on this and the other questionnaires in the package.  We ask in part because we found "nits" such as stray instructions or skip patterns that might not be quite accurate and in part for more substantive reasons.

Original question and response: OMB—Please clarify the status of pretesting the new items.  For example, #18, do 4^th graders know what "hands on" means?  NCES— In the 2005 science student background questions section, the phrase “hands-on activities or projects” was used with grade 4 students.  However, to be consistent with the wording used in questions 1-7 of the pilot science student background questions section and to avoid any confusion with the term “hands-on”, we propose to reword questions 18 and 19 by replacing “hands-on activities” with “activities or projects.”

 

18.        In school this year, how often do you do activities or projects in science?

            A          Never or hardly ever

            B          Once or twice a month

            C          Once or twice a week

            D          Almost every day

 

19.        In school this year, how often do you talk about measurements and results from your science activities or projects?

            A          Never or hardly ever

            B          Once or twice a month

            C          Once or twice a week

            D          Almost every day

Depending on the volume of items developed and a given schedule, items are pretested by:

1. conducting cog labs to test select items before piloting,

2. conducting cog labs to test items as they appear in background questionnaires simultaneously with piloting and/or

3. after pilot testing conducting cog labs to test select items to finalize operational background questionnaires.

As part of the initial Science background questionnaire item development, we conducted cognitive labs to test select student, grade 4 science items (e.g., items that used the term “hands-on”).  We plan on conducting cog labs to test all student background questionnaires later this year.  Together with the pilot data and information garnered from the cog labs, we will finalize the items for the 2009 operational administration.  Through both processes, we expect to resolve any “nits” you have identified.

School questionnaire Grade 4, Part 1.

1. Missing "skip" instruction at end of item 3.

2.       Do all students in your school follow the same calendar? (VB556165) [4/8/12]

A    Yes  Go to Question 3

B    No  Skip to Question 4

3.       Please indicate the number of hours of instruction that fourth-grade students in your school completed as of February 1, 2008. (VB556166) [4]

,

hours of instruction as of February 1, 2008

4.       For each group of students following a separate calendar, please indicate the number of hours of instruction that fourth-grade students in your school completed as of February 1, 2008. (VB556168) [4] 

First group:

,

hours of instruction as of February 1, 2008 [4]

(VB556169)

Second group:

,

hours of instruction as of February 1, 2008 [4]

(VB556170)

Third group:

,

hours of instruction as of February 1, 2008 [4]

(VB556171)

We recommend adding a skip pattern to prevent potential problems.  Given the format of this item, we recommend adding the skip instruction to the stem: “Please indicate the number of hours of instruction that fourth-grade students in your school completed as of February 1, 2008. Fill in the blocks below and then skip to Question 5.” We will test this item later this year and make any necessary changes.

2. Item 22, did your pretesting assure you that you don't need another category of response?  I ask because, anecdotally, none of these seems to describe the method used at my kids' school.

22.     Which of the following describes how your school assigns fourth-grade teachers to classes? (NEW) [4]

A   Requests are granted based on teacher seniority.

B   Class assignments are rotated.

C   The Principal/School Administrator decides which teacher fits best with a particular class or group of students.

This item was developed according to input we received from experts; as part of NAEP BQ item development, NCES convenes experts, most whom participated in the original assessment framework committees, to identify issues and then review items using their informed judgment.  We will test this item later this year and make any necessary changes.

3. Item 30, what is the difference between b and c?  Is c supposed to be school or intra-district based?  Maybe clarify.

30.   Which one statement best describes how your school develops fourth-grade curricula for the following subjects? Fill in one oval for mathematics, one oval for reading, and one oval for science.

b.     The District develops a curriculum that each school adopts.

c.     A group of curriculum specialists develops the curriculum.

d.     A curriculum committee composed primarily of teachers develops the curriculum.

Item c is school-based.  We recommend rewording item c to read “A group of curriculum specialists develops your school curriculum.”  And then to be consistent, we also recommend rewording item d to read “A curriculum committee composed primarily of teachers develops your school curriculum.”  We will test this item later this year and make any necessary changes.

School questionnaire Grade 4, Part V

1. Item 18, Does NCES have evidence that schools have an objective information source for responding to these subquestions?

18.     About how many parents indicate that they chose to send their children to your school for the following reasons? Fill in one oval on each line. (VC104952) [4/8/12]

	None	Less than half	About half	About three-quarters	All or almost all	Did not ask
a.     Their child was performing poorly in another school.	A	B	C	D	E	F	(NEW)
b.     They wanted a more rigorous curriculum for their child.	A	B	C	D	E	F	(NEW)
c.     They wanted the particular curricular focus this school has.	A	B	C	D	E	F	(NEW)
d.     They wanted a safer environment for their child.	A	B	C	D	E	F	(NEW)
e.     They were seeking greater parental involvement in their child’s education.	A	B	C	D	E	F	(NEW)
f.      They wanted smaller class sizes.	A	B	C	D	E	F	(NEW)
g.     Other reason	A	B	C	D	E	F	(NEW)

Since some schools may not systematically collect this information, we recommend making two changes:

(1) Add the following filter question: “Have you asked parents why they chose to send their children to your school this year?  No You have finished the survey. Thank you for your time.; Yes, we have asked some parents about this.  Go to Question X; Yes, we have asked all or most of the parents.  Go to Question X”

(2) Edit the stem of #18 to read “Of the parents you asked, about how many indicated that they chose to send their children to your school for the following reasons?”