Research Support Services, Inc.
February 9, 2015
Introduction
This report summarizes findings of the cognitive testing conducted by Research Support Services Inc. for the High School Longitudinal Study of 2009 (HSLS:09) Second Follow-Up field test survey instrument. Thirty cognitive interviews were conducted in Illinois, in the Chicago Standard Metropolitan Statistical Area from December 6, 2014 through February 3, 2015.
The following table displays the respondents’ characteristics:
ID |
High School Credential |
Enrollment and Employment Status |
Postsecondary Program/Status |
Sex |
LGBTQ* |
Race/ Ethnicity |
004 |
Diploma |
Working/Enrolled |
Currently in 4-year program |
Male |
Yes |
White |
007 |
None |
Not working |
Never enrolled |
Male |
Yes |
African American |
008 |
None |
Not working |
Never enrolled |
Female |
No |
African American |
009 |
None |
Not working |
Never enrolled |
Male |
No |
African American |
012 |
Diploma |
Working/Enrolled |
Currently in 2-year program |
Female |
No |
Hispanic |
100 |
Diploma |
Working/Not Enrolled |
Never enrolled |
Male |
No |
African American |
101 |
Diploma |
Working/Enrolled |
Currently enrolled in 4-year program/ Completed 2-year program |
Male |
No |
Asian |
104 |
Diploma |
Working/Enrolled |
Currently in 2-year program |
Male |
No |
Hispanic |
105 |
Diploma |
Not working |
Currently in 2-year program |
Female |
No |
Hispanic |
109 |
Diploma |
Not working |
Currently in 4-year program |
Male |
No |
Biracial |
111 |
Diploma |
Not working |
Left 2-year program |
Female |
No |
African American |
112 |
Diploma |
Not working |
Left 2-year program |
Male |
No |
African American |
114 |
Diploma
|
Working/Not Enrolled |
Left 2-year program |
Male |
No |
African American |
122 |
Diploma |
Working/Not enrolled |
Completed 4-year program |
Male |
No |
White |
123 |
Diploma |
Working/Not enrolled |
Completed 4-year program |
Female |
No |
Hispanic/ White/ |
125 |
None |
Not working |
Never enrolled |
Male |
No |
African American |
127 |
None |
Not working |
Never enrolled |
Female |
Yes |
African American |
129 |
Diploma |
Working/enrolled |
Currently in 4-year program |
Female |
No |
Biracial |
133 |
Diploma |
Working/not enrolled |
Left vocational program |
Female |
Yes |
African American |
148 |
Diploma |
Working/not enrolled |
Completed 4-year program |
Male |
No |
White |
151 |
Diploma |
Not working |
Left vocational program |
Male |
No |
African American |
160 |
Diploma |
Not working |
Currently in 4-year program |
Male |
Yes |
White |
166 |
None |
Not working |
Never enrolled |
Female |
Yes |
African American |
169 |
Diploma |
Working/enrolled |
Currently in 4-year program |
Male |
Yes |
Asian |
173 |
Diploma |
Not working |
Currently in 4-year program |
Male |
Yes |
Asian |
301 |
None |
Not working |
Never enrolled |
Female |
Yes |
Hispanic/ Native American |
403 |
GED |
Not working |
Never enrolled |
Male |
Yes |
African American |
406 |
Diploma |
Working/enrolled |
Currently in 4-year program / Completed vocational program |
Female |
No |
African American |
508 |
Diploma |
Working/enrolled |
Currently in 2-year program / Completed vocational program |
Male |
No |
Hispanic |
509 |
Diploma |
Working/enrolled |
Currently in vocational program/ Completed 2-year program |
Female |
No |
Asian |
*LGBTQ: Lesbian, Gay, Bisexual, Transgender, Questioning/Queer
Participants were recruited through online postings and flyers targeted at various postsecondary institutions and organizations. Flyers (shown on Appendix D) were handed out to students on five college campuses (including Chicago community colleges), and were posted in a variety of community venues, including job search centers, and community organizations serving youth --including those specifically serving lesbian, gay, bisexual, transgender, and LGBTQ youth.
Interested candidates responded to a telephone screener (see Appendix E). Of the thirty total participants, ten identified as LGBTQ and twenty as straight or heterosexual. Twenty-one respondents were currently or previously enrolled in a postsecondary program, while nine had never enrolled in a postsecondary program. Twenty-two respondents had received their high school diploma, seven had no high school credentials, and one had received a GED. Fifteen respondents were not working as of their interview date, nine were working while enrolled in a postsecondary program and six were working and not enrolled in any program. Eighteen of the thirty respondents were male, twelve female and twenty-six belong to racial/ethnic minority groups.
Interviews were conducted in an interview room in a Chicago area hotel. For some of the interviews, RTI or NCES project staff listened in real time by telephone. Each interview began with an introduction and administration of a consent form (shown on Appendix A). Then the respondent was asked the questions verbally just as a CATI interviewer would, but with concurrent probing, that is, after each question was asked, follow-up cognitive probes were administered to shed light on the mental process the respondent followed to provide an answer. The Interview Protocol appears on Appendix B. Showcards (see Appendix C) were used to have the respondent select one or multiple answers from a list.
A paper questionnaire was used for the cognitive tests because the instrument had not been programmed yet. For field test administration, the questions will be in a programmed instrument that can be either self-administered online or interviewer-administered on CATI (by phone).
Findings from Cognitive Testing
Do not include any high school completion programs for adults that you may have attended. If you are enrolled in high school, but are out for a break, illness, injury, or vacation, please consider yourself as currently attending high school. |
This question was administered to all 30 respondents. The full question including the respondent instructions felt too long to read. Some respondents forgot that the question was asking for month and year by the time they had to answer.
Most respondents were able to easily recall last month and year attending high school and had a clear understanding of traditional high school, alternative high school and high school completion programs for adults. Three respondents had attended an alternative high school; all others attended traditional high schools. Traditional high school was generally perceived as “regular” four-year school, with extra-curricular activities.
101: “Normal school where you go from 9th grade to 12 grade…normal classes, English, math…”
105: “A regular high school for teenagers.”
111: “It’s a regular four year high school where you accumulate hours and graduate."
125: “Your regular four years, prom, graduation, homecoming, high school reunions and stuff like that.”
Several respondents associated ‘high school completion programs for adults’ with the GED and distinguished from alternative high school based on student age.
301: “I think of the alternative as for (students) that still could go back to high school but doesn’t choose to or can’t. And then the high school completion program is for the ones that’s over age who can’t go back to high school.”
112: “You dropped out of high school, you’re an adult and now you want to get your GED.” This respondent suggested replacing “high school completion program for adults” with “GED program” to be more clear and further distinguish from “alternative high school.”
Some respondents thought of alternative high school and high school completion programs for adults as the same.
Various interpretations of 'alternative high school' were elicited. For some it is a separate building or school within the same high school where problem students are sent. For others it was a completely separate, unrelated school for problem students. Others were not sure of meaning. Some respondents thought of ’alternative high school’ as non-traditional in terms of curriculum or administration. One respondent (104) thought of vocational, technical and disciplinary schools as examples of alternative high school.
109: Alternative high school: “Maybe a school that doesn’t use a grade system…or a program for troubled kids”
a) Yes ---> ASK 2A b) No ---> GO TO PROBES
2A) How many times have you taken one of these tests?
# OF TIMES: _________
|
Only one respondent (148) was familiar with all three tests, while most respondents were familiar with the GED but not the HiSET or TASC. Three students had not heard of any, not even the GED.
Of all 30, just one respondent had taken the GED. He did not consider the test he took as separate tests (i.e., one per subject). Another one (151) was required to take both HiSET and TASC while attending an alternative high school. No other respondents had taken either test, or any other high school equivalency credential.
The concept of High School equivalency credentials was not entirely familiar to several of the respondents. They gave the ACT/SAT as examples of other high school equivalency credentials.
Questions in this interview also use the phrase "schools that provide occupational training" to refer to other types of schools after high school, sometimes called technical institutes or trade schools. These schools usually offer programs that take less than 2 years to complete. Examples include culinary institutes and cosmetology schools.
Have you ever…
community or junior college? Yes No
or school?
|
As with Q1, this item proved too long to read. Some respondents had a glazed look while the interviewer read it.
Only three respondents understood well the distinction between for-profit and non-profit colleges or schools. For any state college/university they applied to, they clearly defined those as public 4-year colleges, but they generally thought that private universities are for-profit.
Respondents were generally more familiar with the terms “private” and “public” to distinguish different types of colleges or schools.
007: For profit college or school: “That sounds like a business. That’s pretty much what it sounds like. It’s not really geared so much like toward the personal investment of the students more so about getting as much profit as they can from however many students get enrolled. That’s what that sounds like a business.”
012: For profit college or school: "I have no idea! I just heard public, private, 4 years, two years…I’m not sure about the nonprofit or for-profit so… I don’t know if private means nonprofit or something like that.”
104: “You work for them, something like that…where you can make money for that school.”
105: "I think it’s confusing when it says for profit...I think one would be…for-profit would be more expensive than the non for profit. So it’s kind of confusing on what kind of college it’s talking about...I think a for-profit school probably more a private school.”
109: “I haven’t really thought about it…because I haven’t heard about non-profit schools.”
114: “For-profit? You have to pay."
111: “I know there’s public and private... All I know is more expensive, private and smaller classes sometimes, all depends what private school. And a public school is more bigger, more for your money I think…I’ve never heard of for-profit.”
123: “I don’t know what for-profit means …I mean, I went to a private school but I don’t know if this was for-profit or not ...I know that because state colleges are public but I don’t know what is ….for-profit and nonprofit.”
125: “Profit meaning get money.” The respondent gave Harvard as an example of “for-profit” because it is expensive to attend.
About twice as many respondents thought of Community and Junior College as the same, than as different. Several others thought of community colleges as “local” or “for the community.”
007: “I guess when I hear community college I think they are probably more local, probably easier to get into… with the city colleges pretty much if you are local, then it’s easier to get in. Junior college, I don’t know too much about it what I think about it is like if you are still in high school and you are trying to branch into college, maybe it’s a place where you can obtain certain college courses and credits while you are still doing high school…”
008: “Community (colleges) they are more open to the community for different people just to come in and like work and do different type of work towards maybe a certain type of degree, and maybe a junior college means more like for younger kids… like around the age for college…”
012: "I think they are different, I don’t know much about Junior College but I feel like they are… a little bit less intensive that the community colleges…like the community colleges…can provide grades, junior college I think is more…technical."
122: “I think (Junior College) is possibly more a steppingstone … than the community college necessarily has to be… I believe community college is where you can just go and get your Bachelor’s degree.”
123: “Community college…offers all types of degrees for people who don’t want to go too far away. Or…probably have a job around here and want to stay for the job…and would be easy for them versus going to school in a different state. Junior college is more like a private…”
403: “Community college…is for the community …and junior college is like a private school.”
Respondents did not offer other terms to refer to the institutions.
diploma program at a school that provides occupational training. Certificates or diplomas usually take 2 years or less to complete, often leading to a license, such as cosmetology Yes No
2-year degree) program Yes No
4-year degree) program Yes No
|
Overall, Q4 seemed to be more clearly understood by the majority of respondents than Q3, perhaps needing some simplification of option 4a for clarity. Most preferred Q4 to Q3, primarily because of uncertainty about the different types of schools as listed in Q3 and described above.
The remaining respondents indicated that 4a) was too long, “wordy” or “complicated” and chose Q3 as easier despite the fact that they did not understand the distinction between for-profit and non-profit. This may have been due to Q3 appearing more abbreviated on the provided show card.
Some respondents noted that neither Q3 nor Q4 had a good option for attending a vocational, technical school or trade school, or for those who had taken courses not towards a specific degree. One respondent who takes classes in a 2-year college intending to transfer the credits is not enrolled in a 2-year program because he is not intending to get an Associate’s degree; he had a hard time saying yes to option b because of the word 'program'. Another respondent was confused by ’applied to or registered’ in Q4. This person had received a vocational education certification but did not find Q4a to be a clear response option for his situation.
Ten respondents found Q3 easier than Q4. Several had never enrolled in any postsecondary program.
007: “I think question 3. Because it’s not…I think it’s pretty basic. It’s more basic than question 4. I think question 3 pretty much gets to the point about have you applied to this type of college or that type of college.”
112: "I think these are much easier to understand [Q3]…because there’s less wording, less words and gets right to the point instead of all these extra words (reads 1st sentence of Q4a)…you’re throwing a bunch of big words at me, it’s gonna take a while for me to process in my brain what exactly you’re asking as opposed to this one (reads Q3a) you know, much more simpler and right to the point." [Note: This respondent chose Q3 as easier even though had "no idea" about for- profit/nonprofit.]
122: Because I knew the distinction (between for-profit and nonprofit), Q3 was easy to understand but if I were unsure about that, then Q4 would be easier.
509: “I want to say question 3 was easier… I think it’s just because I can easily tell you, like a two year community or junior college, but this one gives you more, question 4 has more options. I think it’s just because it’s shorter too (Q3).”
Eighteen respondents found Q4 easier than Q3. In particular, respondents who have applied to colleges generally had an easy time with this item.
012: “Question 3 was harder for the nonprofit and the for-profit questions.”
101: “Question 3 harder to answer because…I’m not 100% sure about on the public or private... profit… non-profit…”
105: Question 4 easier: "It’s less words and tells you what the Associate degree is and Bachelor's degree is and then it tells you what a usually two-year degree program and then a four-year degree program ...I think the answer 4a goes more into depth on what the cosmetology or what a certificate can be than Q3.”
109: (Q4 easier): “Because it is something that you are more exposed to day by day. I was kind of thrown off by the profit and non-profit, I never heard that.”
123: “Question 4 is the better way of presenting different types of colleges and schools... makes more sense, people usually know what they are applying for.”
160: “Q3 would be better if it can include some examples, for me was easy…but may be confusing for other people, I think a lot of people don’t know the differences between private nonprofit, private for-profit. Q4 is more clear but a little wordy.”
How many of your close friends or family members permanently left a postsecondary education program before completing their degree, certificate or diploma?
|
In general, respondents were not familiar with the term ’postsecondary education’ and needed Q5 to be repeated. When clarified, most respondents had no difficulty in providing an appropriate response. It appeared that the definition of postsecondary was useful. The difficulties observed may be due to the length or dense phrasing of the question when administered verbally, and perhaps less of an issue in the self-administered mode.
Some respondents were unsure who to include as ‘close friends’ or if they should include more extended family members (i.e.: cousins). Probing showed uncertainty as to who might go back to school or not. This was particularly true when thinking of peers. With family it was easier. With friends or schoolmates, it was hard for them to tell if someone that is no longer going has really left the program for good.
Some respondents, particularly those who estimated, were vague about how they figured their answer. If someone focused more on family members than friends, they seemed to answer with more precision. For friends, while some thought of up to a handful, others included more than 20 people in their thinking to provide a response.
FILTER: As things stand now, what is the job or occupation that you expect or plan to have at age 30? _____________________________
OR CIRCLE DK-----------> DK CONTINUE ANYWAY.
Would you have answered this question the same way when you were in high school?
YES ---> SKIP TO Q7 NO ----> How would you have answered it then?
RECORD VERBATIM: ___________________________________ ______________________________________________________ ______________________________________________________
6A. Now tell me if you changed your mind about your plans for a job or occupation for any of the following reasons.
employer encouraged it Yes No
|
No particular difficulties were observed in the administration of Q6 and Q6a. Even though the verbatim responses did not seem to interviewers to fit exactly in the choices in Q.6A, respondents indicated they could choose one main reason from the list.
Most respondents indicated that they would have responded differently if asked the same question in high school and were able to clearly articulate reasons for the change. Some respondents mentioned that their future plans while in high school were “unrealistic” or they have since set another short-term career goal with hopes of achieving their original goal later in life.
123: [Plans to become a music teacher.] In high school, “I would have said singing at the Met but now that I’m graduated I realized that...is not very realistic dream…I need a real job.”
151: “I've had a lot of growing up to do and I have to think more realistically. Sometimes being an entertainer is not the route to take because sometimes it takes money to make money.”
Several respondents suggested adding an additional option for “a change in life situation/circumstance” or simply “life experience” as a reason for changing their mind about a job or occupation. Another one suggested adding “skills compatibility”.
009: Mother suffered an illness which influenced his change in plans.
112: Was in an accident and had to change his plans as a result.
111: Changed plans due to difficulty in passing entrance exams for nursing.
122: Indicated that his “experiences in college opened up other passions.”
101: Indicated that attending conferences and career fairs influenced his decision.
Two respondents (112 & 509) did not understand the meaning of ‘prestige’ in option f).
7A) Salary may be only one part of why people choose a job. Compared to salary, how important is the ability to have work-life balance to you: less important, as important, or more important? |
7B) Salary may be only one part of why people choose a job. Compared to salary, how important is the ability to work or collaborate with others to you: less important, as important, or more important? |
Some respondents were confused as to whether the questions asked if salary is more important than work-life balance/ ability to work or collaborate with others, or if work-life balance/ability to work or collaborate with others is more important than salary. It may be useful to offer response options that are clear and unambiguous such as:
Work-life balance is more important than salary
Work-life balance is as important as salary
Work-life balance is less important than salary
Some respondents had trouble seeing a difference between the two questions. This was less pronounced in the students who have been to college.
In Q. 7a, the vast majority of respondents found the added definition of “work-life balance” helpful to clarify the meaning of the question. Several respondents did not have a clear idea of the meaning of “work-life balance” and were able to provide a thoughtful response after the definition was provided. Some had initially interpreted 'work-life balance' as “worklife balance”.
Most respondents who had an initial understanding of ‘work-life balance’ felt that the definition would be useful and add clarification for others. Only three respondents did not find the added definition to be helpful.
122: “I think this (definition) is only one version for what work-life balance could be. I think depending on what you do for work , you can work all the time and also be balanced because you are doing something that fits in the activities that you really love …that can be a different kind of balance”
In one case alone, (105) the respondent misunderstood the question to mean “if you prefer getting paid salary or hourly," even after hearing the definition. “I think the definition is too long. Maybe a different word... I think the term itself is confusing."
In Q. 7b, all respondents were clear on the meaning of ‘to work or collaborate with others.’
In both questions, respondents gave thoughtful answers about why they would consider salary more, less or as important as work-life balance or team work.
(IF R HAS ATTENDED A POSTSECONDARY INSTITUTION, ASK: Regardless of your major or field of study…)
1=Strongly agree 2=Agree 3=Disagree 4=Strongly disagree
|
Respondents understood Q8 overall as intended and considered personal strengths and interests in answering and describing each “type of person.” Several tended to justify their answers in terms of how well they had done in high school or college in the different subjects (math and science, especially).
Nearly half of the respondents were unsure about the meaning of the term 'engineering' or misunderstood it.
009: Thought of engineering as “just being able to work with your hands.”
105: "Someone that has to do with fixing engines, different types of things that need to be fixed.”
125: “Something like a mechanic?”
151: “It could be different types of things. I think of automobiles first, but you can also be wood shop in engineering, anything. As long as you are using your hands.”
166: “Do it have to do with like janitorial and stuff? I don’t like that.”
301: “For me an engineering person is people that work on the buildings…clean stuff up, fix things.”
508: "People that get dirty...get their hands dirty…building stuff, make car parts."
For some respondents, the distinction between engineering and computers/technology was not clear. For a few others, being a computer person was defined as knowing how to use a computer well.
Questions 9, 10 and 11 ask about sexual orientation and gender and were answered without discomfort by all respondents. A few did a bit of a double-take when they first heard question 9 simply because of the change in topic, but no one appeared to have other issues.
Ten of the 30 respondents self-identified in screening as gay, lesbian, bisexual or transgender. The only one respondent screened as transgender changed his self-definition at the interview and explained he considers himself “demiheterosexual”. This reflected more of a political view about the non-binary nature of sexuality and gender than anything else.
Now let me ask you about yourself.
1=Lesbian or gay 2=Straight, that is, not gay 3=Bisexual 4=Something else (please specify) 5=Don't Know
Now look at this other way of asking the question. SHOWCARD. How would you answer if it was phrased like this?
2nd showcard: 1=Homosexual 2=Heterosexual 3=Bisexual 4=Something else (please specify) 5=Don't Know |
No respondents had difficulty selecting a response to Q9. However, there was a broad range of opinions among LGBTQ and non-LGBTQ respondents alike, about which list of response options was preferred, or easier to understand.
Some respondents thought 'straight' and 'not gay' were the same, so the 'not gay' is unnecessary.
122: “I do not need synonyms…when someone is asking me about my sexuality I wouldn’t say I’m not gay, I would say I’m straight.”
Others noted that 'not gay' is different than 'straight' and could include other categories such as bisexual—someone identifying as bisexual is not gay and not straight.
Among LGBTQ respondents, these reactions included:
004: [Respondent identifies as “demiheterosexual.”] “Straight… not gay excludes…bisexuality or pansexuality or (the) possibly infinite number of sexual orientations.”
007: “Straight and not gay don’t really mean the same to me, and that’s just because in my personal experience I know that some people don’t really conform to labels as far as straight, gay, bisexual, different things of that nature. I think that just by stating straight or heterosexual, I think that that just by itself is pretty flat out, okay, that is what it is. Straight is pretty self-explanatory.”
169: “You can be bisexual and not gay, you can be other sexualities that people identify with…not straight but also not gay.”
Heterosexual respondents’ comments included:
105: “If it says straight, that is not gay, and then there is a gay option at the top, it would be kind of confusing. I don’t think gay and straight are the same. It could be... gay that is not straight I think that would be more towards like bisexual.”
111: [Referring to “straight, that is, not gay” option]: “doesn’t say “not lesbian” either, someone could think being “not gay” would just refer to a man.” (Someone could identify as “not gay” and “lesbian”).
One LGBTQ respondent found the ‘straight, that is, not gay’ option to be offensive as emphasizing ‘not gay’:
133: “I don’t like straight, that is, not gay… I don’t like that …just put straight because it is… like…kind of homophobic."
Some respondents did not have a clear understanding of the terms ‘homosexual’ and ‘heterosexual’ or considered ‘straight/gay’ to be more commonly understood terms. These were heterosexual respondents:
008: Not sure of meaning of homosexual/heterosexual: “I mean I would just put…I’m straight but…I’d just put I’m “something else” which I’m straight.”
112: “A lot of people don’t know what heterosexual is, a lot of people don’t know the definition of that.”
Some LGBTQ respondents worried that others may not be familiar with the terms homo- and heterosexual:
173: "For me they mean the same thing but, the word lesbian and gay I think has more connotations, whether it's positive or negative...Whereas these, homosexual, heterosexual it's just very, like more scientific terms I think that people wouldn't use on a daily basis...these words (lesbian/gay, straight) are more commonly used, to identify themselves. I think these are better.”
301: “For me I know what all this means, I know what homosexual is or heterosexual, bisexual, I know what those terms mean, but if it was for somebody else who don’t know what those terms mean, I think this would be more descriptive for them ….A lot of people don’t know what homosexual or heterosexual is.”
Some respondents, all heterosexual, felt that “homosexual/heterosexual” are presumably more proper, less offensive terms than “straight/gay” to those who identify as LGBTQ.
012: "Heterosexual…I think it is… Less offensive."
125: “These are better ways to ask, homosexual, heterosexual, it’s (more) proper than saying gay…heterosexual saying straight…These are the more appropriate terms to use….if you don’t know what heterosexual mean or homosexual then I don’t know what to tell you. It’s common sense though.
151: “I feel that some people might get offended, but I don't know how sensitive people are, but heterosexual seems more appropriate than just say straight.”
1=Male 2=Female
|
All respondents understood clearly what is meant by ‘sex assigned at birth.’ Perhaps thanks to the clarifying second part of the question 'that is, what did the doctor put on your birth certificate', none had difficulty selecting a response between the two options. They explained that based on anatomy (using words as genitals, organs, genitalia, penis/vagina) the doctor indicates at birth the baby's sex.
Some respondents recognized the distinction this question suggests between biological sex and gender, while others thought of biological sex and gender as the same. Some respondents considered intersex individuals when asked what was meant by ‘sex assigned at birth.’
112: “A lot of people are assigned a sex at birth but then get a little older and identify themselves as the opposite of what they are, so that phrase is being very sensitive, hyper-sensitive to how they may feel.”
008: “I guess they want to know like at birth what were you, cause you could have been born a guy and then transform yourself…as you got older into a woman.”
100: “What is your sexuality, what are your preferences, what are you particularly like, what are you attracted to…(question is) asking about your gender.”
169: (Question is asking about) “Biological sex. Sex assigned versus...gender. So gender identity is something that you develop on your own. Whereas, biological sex is... assigned according to your physiology.”
173: “It's what doctor determines after seeing your genitalia but I know that for some people with ambiguous genitalia, just like intersex, doctors used to just assign one or the other or do some surgery to make it one or the other whereas these days has been ongoing opinions that doctors shouldn’t do that and let intersex people grow …the way they are”
HAND SHOWCARD.
|
No respondents had difficulty selecting a response to Q11. Only one respondent self-defined as transgender, and it was someone who expressed uncertainty about his sex and gender, and was inconsistent between screening and interview.
Most were familiar with the definition of gender and found it helpful to include.
007: “I know the way society is set up now transgenders are really just now coming out and speaking about what it means to be transgender so I think for those who aren’t really accustomed to what that might be that’s a pretty good definition because some people think, “well you’re a man, you should stay a man, if you’re a woman you should stay a woman” but that kind of helps people to see it’s not an outward thing, it’s an internal thing, so I think that definition definitely suits that." [an LGBTQ respondent]
105: "I think that’s new because mostly when they ask gender they ask whether if its female or male...Yeah, it’s useful for the question. It’s telling you how you feel inside not what you are, so I guess if he was a male and he’s more like a transgender he know that he can, transgender would be an option."
406: "Originally, before I took a course of sexual identity, I thought that transgender means to literally get a sex change, but it was not. That is, it is basically what you said,
Someone who identifies different than they were physically born."
151: “When I was first talking about it, I did say gender, which now I think I should have said sex instead of gender. Like you said gender could be a different feeling than what you really are. And some people are not quite sure of what their gender is. If you add that definition with the rest of the question, it would help out a lot.”
173: “It's something I'm very familiar with. I know a lot of transgender people and it's not a myth, it's real and especially these days, trans people are gaining more voice and they're getting more exposure so it's just something that I knew.” [an LGBTQ respondent]
Some respondents understood the gender definition but disagreed. They thought of gender as the same as biological sex (male/female).
008: “I don’t believe in that, but…everybody has their own opinion…I mean cause I look at it like…sex and gender is…the same…I don’t like that definition…it doesn’t make sense…to me.”
111: "I don’t believe in that. I believe what you are born with, that’s who you are, and everything is learned...I think whatever the doctor say you are when you come out, that’s who you are."
112: "I feel like your gender is your gender, if you were born with a penis then your gender is male of course, but how you feel inside is totally different from your gender. I feel like your gender is biological... I don’t want to change the definition that’s on there or like the scientific definition but that’s just my opinion…that your identity is how you feel inside, you know which could be transgender, but your gender is biological."
Only one respondent (004) considered more than one response option. He chose 'male' but considered selecting ‘transgender, do not identify as exclusively male or female.’ This respondent identifies as “demi-heterosexual” and would have selected “gender non-conforming” if this were an available response option.
Several respondents took issue with response option e ('Transgender, do not identify as exclusively as male or female.’) Perhaps this response option should be replaced with a “gender non-conforming” or “something else” option.
007: "I think for this one though (option e), they should have one called gender non-conforming… But I think the other ones are legit to show that they are different technical terms for male-to-female, female-to-male."
122: “I would maybe not put the word transgender there, I would maybe just have do not identify as exclusively female or male, because you might identify as something else but you don’t identify as transgender.”
123: “The one that doesn’t really make sense here is ‘transgender, do not identify as exclusively female or male’, well you just can say androgynous. I see that transgenders identify as the opposite that they were born as, but androgynous would not necessarily need to be transgender.”
169: “Not everyone identifies as transgender…add “something else” in the list as one more option.”
|
Respondents still living at the parental home generally answered yes to d., e., and j. since the parents take care of room and board, but some were not sure:
012: Unsure if should include options f) & j) “the house and the food because I live with them (parents).”
109: Option j) “I was confused because they didn’t gave me the money for food, just gave me the food.”
508: Unsure about e) & f) Lives with parents who own home.
Some respondents were unsure about certain response options. For instance, 009 wondered if sport fees would be included under 'fees' in option g. Respondents had a number of suggestions for additional response options to include: medical/dental expenses, travel expenses, help with debt, cable/internet, clothing, car payment or gas, renters insurance, storage, and help supporting (respondent’s) children.
Option g functioned as double barreled to one respondent who made a distinction between tuition on one hand and fees on the other.
Some respondents tried to think back to the last 12 months and were not completely sure if something fell within that period or not.
Have you held a paid position since you last attended high school?
IF YES: Please answer the following questions about your current or most recent job.
IF NOT: INTERVIEWER, SKIP TO INSTRUCTION BEFORE Q16.
1=A career position 2=A way to explore a career option 3=A way to save money for school 4=A way to pay the bills 5=A way to earn spending money
|
Respondents had no difficulty responding to Q13 and choosing more than one response option as applicable. Generally, respondents had a clear understanding of the distinction between ‘a career position’ (long-term, more commitment) and ‘a way to explore a career option’ (short-term, “trying it out”, internship).
012: Career position: "Something that you find to make your career in the long term and it is in the field that you wanted "explore career option" is something that someone does based on an interest they have"
009: Career position: “Something that you do in the long term. Explore career option: Maybe like… internship… probably something that you are deciding if you want to do or not.”
105: “A career position would be something more permanent, something that you’re sure of doing and exploring would be something that you’re still unsure and trying to experience other careers.”
111: “I think explore doesn’t sound like it has too much of a commitment to it, so that’s why I said just learn. Maybe a career it has a commitment."
112: "A career position is sort of like me saying I want to be a doctor, that’s the position that’s title I have that is doctor, but a career option is you know, the hospital. The career option is the hospital, the position is the doctor. That’s how they’re different to me."
Some respondents had difficulty understanding the difference between response choices 1 and 2. This appeared more common among respondents without work experience.
301: Thought of ‘career position’ and ‘way to explore career option’ as the same, just different words.
Some respondents had suggestions for additional response options:
112: “Maybe a hobby or something you love to do."
114: “Forced to work because someone, like a family member tells you you have to.”
122: “Doing the job because it offers good learning experiences.”
151: “…a way to save money for a new house” (or to save money for something other than school as in option 3).
IS R. CURRENTLY WORKING? YES --- ASK Q14 NO ---- SKIP TO NEXT Q.
1=Strongly agree 2=Agree 3 = Neither agree nor disagree 4=Disagree 5=Strongly disagree
|
These 3 items were only asked of respondents who were currently working. The questions worked well. Respondents gave thoughtful explanations as to why they answered as they did.
1=Strongly agree 2=Agree 3 = Neither agree nor disagree 4=Disagree 5=Strongly disagree
|
We asked this question of those working now but also of those not working now. Those not currently working in some cases had difficulty with the concept of current line of work, whereas when the respondent had a current job these questions worked very well.
Most respondents answered 1, 2, 4, or 5. Of the 23 respondents who were asked Q15, 13 did not see a difference between ‘current job’ and ‘current line of work.’ Overall, this distinction was not clear.
Some respondents did not understand the meaning of ‘current line of work:’
166: Thought of ‘current line of work’ as work history: “I think it’s like every job I worked before. That’s all I could think.”
Is your current or most recent job related to that job or occupation? Would you say it is...
1=Closely related 2=Somewhat related 3=Not at all related
|
All respondents were clear on the meaning of ‘related’ in the context of this question. No difficulties in interpretation were observed although some were not articulate in defining the meaning. They often gave circular answers to what 'related' means, yet they clearly understood the concept based on their explanation as to why they chose the answer they chose.
Respondents reported being sure of their answers.
Have you ever been enrolled in a 4-year college, 2-year community college or junior college, or a school that provides occupational training?
IF YES: CONTINUE WITH Q17.
IF NOT: GO TO END OF INTERVIEW BOX ON LAST PAGE.
a) Yes b) No
|
A couple of respondents who had taken postsecondary courses did not think of themselves as having been ‘enrolled’ in a program. This interpretation issue resulted in answering No to the filter preceding Q17. These respondents, following the skip pattern, ended the survey prematurely. This was the case with one respondent (105) who did not consider having been enrolled: "They were just classes for my Associate's degree." Another (133) had taken postsecondary courses but did not consider herself as having been ‘enrolled.’
As the question was first read to the respondents, some looked at the interviewer as though expecting further clarification. Probing to establish what respondents understood as 'help' revealed that many thought of tutoring as an example, but had not considered other options such as office hours, study groups or other types of help.
104: Was thinking of “financial help” and would add “financial” as another type of help.
114: Thought of help as learning about the college application process, loans and scholarships.
122: Suggested including “Mentor sessions, writing correspondence, emails to the professors because sometimes you can’t sit-down with them face to face.”
509: Considered “advising” as another type of help: “I am part of (an advising) group…and they have helped me decide what schools I want to go to after a two-year and they have helped me with financial aid and getting scholarships.”
After the definition was offered (“Help could include participating in a study group, going to office hours, or receiving tutoring”), one respondent changed “No” to “Yes”, because he had not considered office hours. Another changed because he had not considered study groups. No one was able to come up with additional types of help beyond those in the definition.
Have you ever been enrolled in an Associate’s degree program?
a) Yes --> ASK Q18 b) No --> SKIP TO Q19
1=Plan to earn an Associate’s degree, 2=Plan to transfer to a Bachelor’s degree program, 3=Plan to do both, or 4=Just want to take courses or earn credits?
|
Eight respondents answered “Yes” to having ever been enrolled in an Associate’s degree program. Some did not realize that the question intended to capture a single response, and they chose options 2 and 4.
105: Was not sure if he could select more than one response option.
104: Chose response option 4 only but indicated that he plans to transfer to a four-year program (Both options 2 & 4 applied to this respondent).
111: Combined options 2 & 4: “Maybe take the classes you need to transfer?” Ultimately selected option 3.
a) Yes b) No
19A) IF NO AT Q19, SKIP TO Q20
ASK ONLY IF YES AT Q19 Why did you transfer the most recent time?
19B) Now tell me if you transferred for any of the following reasons.
study not offered by your original institution Yes No
scheduling not being convenient Yes No
|
Only six respondents answered “Yes” to Q19. Most respondents were clear on the meaning of ‘transfer’ but some were not sure whether transferring necessitates transfer of credits or simply changing institutions. The list of reasons on Q19B appeared comprehensive. Respondents were able to choose one main reason.
Some respondents were again confused by the term ‘postsecondary.’ It may be helpful to remind respondents of the definition several times throughout the survey as the term is used.
112: Responded “No” to 19a: "That’s the thing, I don’t know if I transferred. I stopped going to one and then I started going to—I didn’t get my credits transferred though. So, that’s what I think transfer means, getting your credits transferred to the new school you go to."
508: Responded “No” to 19a, then changed response to “Yes.” Needed clarification on meaning of “postsecondary” First responded that transfer means transferring all credits then changed to transfer can be “just going from one school to another”. The respondent did not actually transfer--completed a vocational program, completed an associates program and is now taking classes... has been enrolled in different programs but did not transfer.
111: Suggested adding additional option of “general change in life” as reason for transferring. (Perceived as different than option h) “Someone’s living situation can change too, so just change in life in general I think."
What (is your current/was your) major or field of study?
RECORD VERBATIM: ________________________________________ Why did you choose it?
RECORD VERBATIM: ________________________________________
|
No significant interpretation problems were observed in Q20. Two respondents did not understand the meaning of “prestigious occupations” in option h. The definitions respondents offered for ‘work lifestyle’ were reasonable.
The list of reasons appeared comprehensive. Respondents were able to choose one main reason. Only one respondent had difficulty selecting one main reason, and felt he would need to select more than one option.
Ten respondents preferred the term “major,” three preferred “field of study” and five expressed no preference between the two. The majority of respondents preferring the term “major” had been enrolled in four-year, Bachelor’s degree programs, while those preferring the term “field of study” had been enrolled in two-year programs.
173: Mentioned “concentration” as an alternative to “major or “field of study”
In general, respondents had varied interpretations of the term ‘work lifestyle.’ Some thought of lifestyle at work or lifestyle in general. Others thought of ‘work lifestyle’ as very similar to, if not the same as ‘work-life balance’ as asked in Q7a:
114: “The way your work and what you do outside of work, how it blends.”
105: Thought of “work lifestyle” as “Something that I would…enjoy to do for the rest of my life.”
111: “I think about lifestyle, I think about I guess money, that’s about it.”
112: “I guess the lifestyle you’ll be living that’s associated with that job or that work.”
160: “Being able to have personal time but also being able to do something that you enjoy, I think that is an important thing…like I said not necessarily live to work but if I have a job that I enjoy that just makes life that much better.”
Before you entered postsecondary education, what major or field of study were you considering.
RECORD VERBATIM: _______________________________
IF SAME AS MAJOR IN Q20, SKIP TO Q22
IF DIFFERENT FROM MAJOR IN Q20: Why did you change your major or field of study?
RECORD VERBATIM: _______________________________
|
The filter for Q21 is very similar to the filter preceding Q6. Several respondents remarked that this question was repetitive or that they felt they had responded to this question already.
No respondents had difficulty providing a response to Q21. Option a) ‘Change of interests’ was most commonly selected as respondents’ main reason for changing major or field of study. Two respondents were not able to choose one main reason.
101: Suggested adding “attending conferences or job fairs” as in Q6.
111: Suggested adding “change in life” as in Q6.
The question was tested with a very small number of respondents due to the skip instruction preceding it. For students who are not far along in school, they often have not yet declared a major, much less changed majors.
What is the highest undergraduate degree, certificate or diploma you expect to ever complete?
1=None -- SKIP TO Q23
2=Certificate or diploma from a school that provides occupational training 3=Associate's degree 4=Bachelor’s degree
$______________
|
Respondents were not too confident about their answers. Most respondents provided an estimate for total amount borrowed, calculating expected amount to be borrowed per year. Figuring the answer followed various strategies. The answers appeared to be estimates. In some cases they seemed to be somewhat wild estimates and in others more grounded in information the respondent had about cost of a year of tuition or tuition plus room and board.
Those who had completed school were more confident in the total amount. If students had not taken out loans, or had received financial aid or scholarships, they mentioned this as well.
004: “$20,000 is the amount of the loan that I take out each year, multiplied by four years.”
123: Completed Bachelor’s degree and knew exact amount to the dollar based on her federal loan statement.
173: Estimated the amount his parents borrowed for his education, “My parents would owe it on my behalf.”
509: “Right now I have financial aid, so I don’t have to pay for anything and right now I am not sure what four-year college I am going to go to and depending on how much financial aid I am going to get for those schools, I am not really sure. I know I will have to get loans, I don’t know how much.”
Several respondents were unsure of the meaning of the term ‘undergraduate’, including some who have attended college:
105: Undergraduate: "Something that I haven’t graduated school from. So if I’m an undergraduate from college I’d probably be in high school."
112: "Like in my lifetime? Masters." When asked, “What does “undergraduate” mean to you?,” the respondent answered, "No idea."
151: “I actually never understood what that mean. I feel that is somebody that is in college...but, I never really knew what that was.”
In some cases, upon hearing 'highest' the respondents answered with the highest degree they would like to complete, without attention to 'undergraduate'. For example, one answered MBA and another said PhD.
100: First responded “PhD” then changed answer to “Bachelor’s” after question repeated.
101: First responded “MBA” then “Bachelor’s” after question repeated.
a) a basic or developmental algebra course?
Yes No
b) Since you started your postsecondary education, have you taken a basic or developmental mathematics course?
Yes No
PROBE: Would the following definition have been unhelpful or helpful?
“Basic or developmental courses are used to strengthen your basic math skills. Students are usually assigned to these courses on the basis of a placement test taken before the school year begins.”
|
'Basic or developmental' was interpreted as introductory. All respondents found the supplemental definition of 'basic or developmental' to be helpful: .
Most respondents had a clear understanding of Q23a & Q23b without the added definition, but still felt it would be useful to include. In the definition, what appeared most useful in conveying the idea of a course to bring someone to college level was the mention of the placement test.
004: Definition “very helpful…would clarify precisely what you are getting at.”
105: “It tells you in depth of what it means in the question and they probably would get confused on what it means…it tells you there that it would be something you took before the school year.”
111: “I think basic would sound better, I know developmental but not really with that question, developmental math, never heard that… I took one in college but when I was in college they called it remedial, so I guess that’s the same thing.”
Some respondents did not distinguish between math and algebra and answered yes to both but referring to the same course. Thus, there is a risk of over reporting.
|
Most respondents were able to select from the list of course titles, even when titles of courses taken did not match exactly. However, we cannot establish whether or not they matched them correctly at all.
Some respondents who have taken more advanced math courses were not clear as to whether they should select 'other' or 'none.'
004: Selected 'other' option for 'Elementary Statistics.'
012: Selected 'none.' “I took Medical math, and Chemistry does have some math in it with Stoichiometry…”
123: Mentioned she had taken “higher up math courses.” Selected g) “I mean Calc, but Calc 2, not just Calc 1.” Thought “stats” might fall under Calculus option, or should be added.
129: Selected 'other' option for Business Statistics course.
173: "I took g) and h) Linear Algebra and Statistics. ... Multi-variable Calculus is more advanced, but it’s still Calculus-based course, so I put under g). Linear Algebra doesn't belong in any... Linear Algebra and Calculus, I don't consider them basic math, but these from a) to e) at least would be considered basic. So if the question concerns only basic math then I think this list is fine, but if it's a question about just math in college in general then it's not comprehensive."
Several respondents felt that there should be additional options added to the list.
105: “I think they should add more options…Trig and geometry is not on there, Stats, I think they should have more math options." Respondent would have selected "other” for geometry, trigonometry and statistics.
112: Selected ‘other’ for “Computer Science technology” and felt that “trigonometry and geometry” should be added.
160: Did not take any math courses but felt that “logic” should be added to the list.
508: Respondent did not know meaning of ‘arithmetic’ thought there should be an “Intro to Math” option.
1=Yes 2=No 3=Don’t know
Please look at this definition. SHOWCARD
“Universities generally have major non-specific requirements for degree attainment known as ‘General Education’ (or ‘Gen Ed’) or ‘Core’ requirements. Additionally, specific majors generally publish their requirements for completion. If this math course would appear on either of these lists, you should answer ‘Yes.’”
|
This question appeared to work well. Everyone picked Yes or No and justified their answers reasonably. Most respondents interpreted Q25 as asking if the course was a general education requirement OR a degree requirement, not as a Yes/No question. This may not be an issue when the survey is self-administered.
The supplemental definition was generally helpful. Most respondents understood the meaning of and distinction between ‘General Education’ and ‘Core’ requirements. Definition helped to clarify for respondents who were unsure. Some respondents found the definition to be too “wordy.”
105: “I think that helps you know the difference between the general classes and what is required for your degree….It tells you general education and Core, or Gen Eds, which is a common word used, it says that specific majors require certain things for completion…I think it goes more in depth about what certain things are."
112: “It (definition) doesn’t change my understanding…I still think this description is too wordy but if someone had a big enough vocabulary I think they could understand it…I think it’s not necessary, too wordy, if this was much simpler I think I would say yes it’s necessary."
122: “I think it’s a little wordy... but, could be helpful, I think the more clear definition you have the better it helps people to understand.”
160: Felt definition would be helpful for others who may be “not familiar with the terminology.”
173 & 169: Both students mentioned that the University they attend uses the term “Distribution Requirements” rather than “General Education” or “Core” requirements.
509: Found definition confusing: “I think it was the second part that confused me more. The “Additionally, specific majors generally publish their requirements for completion.” It was just because I already knew what general education meant.”
Generally respondents could distinguish between requirements of their major and requirements of the program aside from those specific to the major.
courses? Please select the closest course titles: SHOWCARD
|
Most respondents were able to select from the available options. Generally, course titles taken were more varied and specific than those listed. Some respondents were able to find an appropriate option on the list, while others had more difficulty.
As in Q24, some respondents who have taken more advanced courses were not sure if they should select ‘other’ or “none” -- not actually offered as a choice -- as the list consists of more basic English and Language Arts courses.
004: Course titles do not match well. “I did not take these courses …There are several different courses that can be taken, but they are different from what I can find in this list… this is not very broad.”
148: Selected ‘other’ for Writing for Copywriting and Screenwriting.
104: Selected option f) College Composition as closest option for “Rhetoric 101”
109: Course titles match “except ESL has nothing to do with it.”
111: Confused by option d) Foundations of Writing, Foundations of English, or Foundations of Reading. “I’m kind of clear but I’ve never seen…Foundations of Writing, Foundations of English, never saw that before…”
160: Was not sure if should include “Intro to European Literature.”
406: “I feel like sometimes Philosophy might get thrown into this category so I was unsure whether or not to fit in under Introduction to Literature. In my school, English and Philosophy are closely related.”
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Diego |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |