Questions and Answers

November 3, 2010

Question from OMB re: changes to the SCS:11

Please tell us how the new response categories to Question 16C were developed and tested or from what established survey they come. Please err on the side of more detail rather than less in responding.

Response

The content of Question 16C is based on the work of social bonding theorists (Hirschi 1969; Simons-Morton et al. 1999; Pyper et al. 1987) who suggest that bonding is a potential mediator of problem behavior. Specifically, when youth are bonded or positively associated with their school or other positive institutions they are more likely to engage in prosocial behaviors, whereas one would expect poorly bonded students to be frequent victims of crime, bullying, and other negative behaviors.

The particular item proposed in 16C is taken from the California Healthy Kids Survey. It has been administered as a “Caring Adult Relationships” scaled measure since 2005-06 in the state of California. The item was suggested for inclusion in the School Crime Supplement 2011 (SCS:11) by ED’s Office of Safe and Drug Free Schools. OSDFS has proposed an identical item on its Safe and Supportive Schools (SSS) Survey, to be administered to SSS grantees in 2011. OSDFS hopes to use the SCS:11 national data as “benchmark” against which they can measure their grantees responses to this particular item.

Further, because the item was “new to the SCS,” despite the fact that it has been fielded in other surveys, it was vetted through cognitive lab, which was administered by Census in June and July of 2010. The 15 cognitive lab respondents were evenly divided by gender (8 male, 7 female) and were on the older side of the age range (6 aged 12-15 and 9 aged 16 or 17). Respondents were evenly divided between White and Black, but did not include respondents of other races. Six respondents attended public schools, 1 attended a public charter school, and 8 attended private schools. Census interviewed students who completed the last school year in the Washington DC metropolitan area. Results from cognitive lab are pasted here:

Results and Recommendations

Question 16c. is a new question that replaces one that asked about a more restricted set of characteristics about adults at school.

For the most part, respondents did not have difficulty with these new questions. The majority of respondents answered “agree” or “strongly agree” to all six sub-items. There appeared to be little hesitation or confusion on the part of respondents in deciding how to answer. Follow-up probes revealed that nearly all of the respondents had been thinking about adults affiliated with the school when answering these questions, ranging from athletic coaches to guidance counselors to academic instructors. Many respondents reported thinking about more than one adult at school.

Although the majority of respondents did not have difficulty with this question, there were two exceptions. One respondent hesitated and had difficulty deciding on her answers throughout the series, tentatively answering “agree” to all six sub-items. Upon probing, the respondent said that she really did not think “anyone at school, like any teachers think I’ll be a success. I don’t want to say they think I’ll fail but I don’t know if they think I’ll be a success. I know my coaches do, but they’re not at school.” Additional discussion revealed that the coaches to whom the respondent was referring were adult volunteers that coached the school’s track team, of which this respondent was a team member, but they were not official school employees. The respondent, therefore, did not include them in her thoughts as she answered this question because she did not consider them to be adults “at school.” The issues with this participant’s response were twofold. First, she answered “Agree” to all items in the series when thinking about the teachers at her school when she should have answered “Disagree” according to the probe discussion. Second, she did not include the adult volunteers that coach her school-sponsored track team in her thinking as she answered these questions.

Another respondent answered “strongly agree” with less hesitation than the respondent above, but it was revealed in a follow-up probe that he had been thinking of adults both related and unrelated to the school when answering these questions. He said he was thinking about his mother, his recently-deceased grandfather, and his homeroom teacher. Of the 15 respondents, however, he was the only one to report thinking about a relative or personal friend while answering these questions.

With the exception of the two problematic responses described above, participants did not have difficulty answering Q16c.

Q16c. Recommendation from Census: We do not have any recommendations for changes to this question.

SPONSOR’S (NCES) FEEDBACK: The sponsor decided to add a lead-in to this question that matches the prior question. “Thinking about the ADULTS at your school, including teachers, would you strongly agree, agree, disagree, or strongly disagree with the following…” Also, the sub-parts were changed from the first person (i.e., me) to the second person (i.e., you) in response to input from Field Representatives who felt uncomfortable asking the questions when it sounded like they were referring to themselves.

In sum, NCES and BJS agree this is an established, well-vetted item that showed no significant problems during cognitive lab. The item will contribute a concept to the SCS (social bonding/caring adult relationships/protective factors) that is attractive to researchers, policy-makers, and the public.

References

California Healthy Kids Survey. Module A: Core, High School Questionnaire. California Healthy Kids Survey, Version H12 – Fall 2009-2010. WestEd and the California Department of Education. See: http://chks.wested.org/

Hirschi, T. (1969). Causes of Delinquency. Berkeley: The University of California Press.

Pyper, J. R., Freiberg, H. J., Ginsburg, M., et al. (1987) Instrument to measure school climate. In Barber, L. W. (ed.), School Climate, Bloomington Center on Evaluation. Phi Delta Kappa, pp. 87–96.

Simons-Morton, B.G., Crump, A.D., Haynie, D.L., et al. (1999) Student-school bonding and adolescent problem behavior. Health Education Research, 14 (1) 99-107.

2