Research Support Services
September 27, 2010
This report summarizes findings of the cognitive testing conducted by Research Support Services for the ELS:2002/12 study. The cognitive interviews were conducted in Evanston, Illinois, between August 18 and September 21, 2010. Eleven interviews were conducted with Form A, 9 with Form B, and 5 each with the two versions of Form C.
The following table displays respondent characteristics.
Respondent characteristics
|
Age |
Gender |
Race-Ethnicity |
Form Tested |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
16 |
25 |
Female |
White |
Form B |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
17 |
25 |
Male |
Black or Afr Amer |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
18 |
26 |
Female |
White |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
19 |
25 |
Male |
White |
Form C – v1 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
20 |
27 |
Male |
White |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
21 |
26 |
Male |
Black or Afr Amer |
Form C – v2 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
22 |
24 |
Male |
White |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
23 |
26 |
Male |
White |
Form C – v1 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
24 |
27 |
Male |
Hispanic (wrote in for race) |
Form B |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
25 |
25 |
Female |
White |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
26 |
28 |
Male |
White |
Form C – v2 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
27 |
27 |
Female |
White |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
28 |
24 |
Female |
White |
Form A |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
29 |
25 |
Female |
Hispanic (wrote in for race) |
Form C – v2 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
30 |
27 |
Male |
Hispanic White |
Form C – v1 |
An advertisement asking for interested candidates for the interviews was posted on Craigslist on the Internet. Callers were screened using the screener in Appendix 1. The respondents interviewed had the following age distribution: age 24 (3 respondents), age 25 (9 respondents), age 26 (6 respondents), age 27 (9 respondents), and age 28 (3). Fifteen were women and 15 were men. As for race/ethnicity, one respondent self-identified as Asian, six as African-American, seven as Hispanic, and 16 as White.
Findings from Testing Form A1
Please answer the following questions thinking about your most recent job in the past 12 months. Circle one answer for each question, indicating how strongly you agree or disagree with each statement.
Respondents had some difficulty with the verb tense of the questions. These are in present tense, but the respondents answering this form were all currently not working. They had all had a job within the past 12 months, and were answering about that most recent job. Several recommended changing the verb tense for the questions.
The jobs they were thinking of were:
#2: Roofer – had shorter jobs later but decided not to report on them. However, kept them in mind for some questions
#4: (did not report on most recent: outside sales for security company) previous job: substitute school teacher – Flip flopped between both jobs in responses and sometimes averaged answers.
#9: HR generalist
#14: telephone operator at incoming call center answering questions of filling out FAFSA forms
#17: nursing assistant
#18: Graphic designer at printing company
#20: Accountant on location at a national park
#22: Car salesman
#25: executive administrator of operations
#27: Clinical assistant at Planned Parenthood
#28: Staffing agency administrative assistant
|
Questions about Your Most Recent Job
I’m confident that I can perform my job at a very high level of skill
All Rs were able to answer this question. They circled Agree or Strongly Agree. Respondents had different interpretations for ‘very high level of skill’ during probing. However, when first answering, they were answering about doing their job well, successfully, competently.#2: High level of skill I would refer to as efficiency. Cause certain jobs require a little bit more know-how, a little bit more knowledge, as to what you're doing.
#4: a high level of proficiency, the ability to do the job.
#9: that you’re excelling at your work, as opposed to just being average or good.
#14: It means that I’m very good at my job.
#17: Do a bit more than just like just getting by, just like average. So actually being a little bit more doing more, without being asked.
#18: For R this meant the ability to do more than one task at a time. To do multiple tasks efficiently and creatively.
#20: I think just being able to do every aspect of the job by yourself.
#22: Well, I was thinking mainly about my ability to serve the customer and sell a car for my dealership, which is a pretty independent process, but if you use your, you know, your management the right way, and the tools that you have, it’s pretty—it can be a simple task.
#25: I am confident that I can perform my job at a very high level of skill, that you could perform your job successfully, and could apply your knowledge to that job, and be successful in that career.
#27: Well I guess high level of skill to me would be like higher education, like having a graduate degree or a Masters.
#28: When I think of high level, I think experience. I think the amount of work you’re capable of doing. I mean it all depends on what the job is. …I thought about knowledge more, like certain computer programs, like how well do you know Excel, that you know how to use every function, like that kind of thing, you know, have the skills.
|
I’m certain that I can solve big problems that occur at work
Generally the question worked well. Respondents varied in their answers. Half answered Strongly Agree, four Agree, and two Neither Agree nor Disagree.In their narratives, Rs varied in what they considered ‘a big problem’, and that was very much tied to the specifics of the job. A common theme was that of the level of independence they had at work to solve the problems themselves or to have to get someone who could solve them. Some felt less confident because they had to ask help, but others answered Strongly Agree, because they knew what steps to take for someone else to solve the problem.Some focused on more common, every day problems (students fight in school, problems with someone’s paycheck not being correct), whereas others thought of really BIG problems (a fire at work; a patient having cardiac arrest). |
I’m confident that I can reach the goals I set for myself at work
Two answered Agree, nine answered Strongly AgreeMost Rs claimed to actually set goals for themselves at work, some more short term, like daily goals, and others more long term. Some Rs in jobs were goals were set by others for them, focused on that, and did not ‘hear’ the part about ‘goals I set for myself at work’. (car sales, outside sales, shift plan for nurse). Other’s goals were more about attitude: one R’s goal in her very boring job was “to stay in a good mood, despite being yelled at most of the time” (#14), another’s goal was to be accurate and responsive (#20). Those who spoke of daily goals were really mostly describing a to-do list. Most everyone felt they spoke from experience: they had set goals for themselves and reached them.
|
I’m certain that I can do my work well despite time pressures
Respondents did not give as high responses as in the prior questions. 4 Strongly Agreed, 3 Agreed, 2 Neither Agreed nor Disagreed, and one Disagreed. They did not all feel they do equally well under time pressures. The question worked well: Rs spoke of coping with deadlines, and gave examples of what time pressures they had at their jobs. Some felt a little uneasy about answering without knowing particulars, as there could be unreasonable requests that made it impossible to do a job well if the time was too short. In some cases, Rs felt the job was designed to follow extreme time pressures that did not allow them to do the job well (call center) |
I’m confident that I can do my work well even when I need to juggle work with non-work responsibilities (e.g., in my family or community)
Question worked very well. Rs focused on the idea of not letting outside things interfere with the performance of the job. Those Rs who do not have family near them, tended to answer about work vs. non-work things, such as doing laundry, seeing friends, or taking care of the place where they live.
|
Occupation Interests
I am really interested in my work.
2 Strongly Agreed, 6 Agreed, 1 between Agreed and Neither, 1 Neither, and 1 Disagreed
For those Rs who seem to have found their field of interest, this question had positive responses, despite the fact that they are not able at the time to find a job in that line of work. Others had no investment in their most recent job, so they were not as enthusiastic.
For those who have had multiple jobs, this question made them compare in their minds how interesting they found this job as opposed to other jobs they had before. Asking this question retrospectively elicits their perception now, looking back in perspective. The question may differ from how they would have answered while they were working. #2: “it always depends on what I'm doing. The roofing job I wasn't necessarily interested in it, but the crew that I was working with kind of made up for it, so to speak.”
#22: Answered Agree, but not Strongly Agree, because although at the time he was working he was very interested in the car sales job, he now does not want to do that type of work.
#28: R who answered between Agreed and Neither: “I mean it got a little rough and tedious at times at a time when I just felt like I wanted more responsibility. And you know, I learned a lot about recruiting, and how the industry works, and I found it very interesting. And that’s something I look at wanting to maybe do as a career is be a recruiter, so in that sense, I was interested in the whole staffing concept. It was just the work I was doing. I wanted to feel like I existed more, or had, like I said, more roles”
|
I often get totally absorbed in my job tasks.
Question worked generally well. Probed on meaning of ‘totally absorbed’. A few thought question asked if work consumed their lives (#20)3 Strongly Agreed, 6 Agreed, and 2 Disagreed (call center operator and national park accountant)#2: I guess just your mind is in the job. You have forgotten about other problems because everyone wants to solve problems but when you get to a job you have to forget about all of your other problems and focus on the problems at hand.#4: Enveloped, you know, in that same level of concentration where if you were a horse, you would have blinders on.
#9: Maybe like consumed whereas all you’re doing or thinking about, so that there’s little room for other things.
#14: This has never happened to me at a job, and I just graduated college last year. I assume they’re talking about like when you’re in a class and you find the subject really interesting, and you’re really focused on it, and you’re even thinking about it when you’re not working on it because it’s so fascinating to you, and you just want to know more about it.
#17: I would guess you’re really involved in what you’re doing. You really care about what you’re doing. When you’re at work, everything else outside that’s going on is on the back burner till you leave your job for the day.
#18: I feel like the job I’ve had, the days go by pretty quickly. And even if somebody comes up to me and starts talking to me, I get startled, because I’m too into what I’m doing. #20: I was thinking people that talk about their work outside of work, and think about their work, or I was thinking when you’re at work, you’re all about your job and you can’t focus.
#22: I mean I don’t have a perfect attention span, but I’m pretty focused. I like to do my job right. I like to do it 100-percent.
#25: When I’m doing my job, I am like a chainsaw. You don’t let anything distract you, and you get the job done. You stay focused.
#27: it’s like you really like what you’re doing and you were stuck with it, or not stuck with it but you are stuck on it; whereas, for me, it was just really busy.
#28: Nothing else around you matters. But sometimes, that’s not always a good thing, because when because when you have to multitask you have some…
|
I rarely get bored when I am doing my job.
The phrasing of this item as ‘rarely’ threw off at least some respondents. To disagree that something rarely happens is cognitively difficult, like a double negative.
Choice of response 3 was not for lack of opinion but rather trying to average the boring and the non-boring aspects of the job. 1 Strongly agreed, 4 Agreed, 3 neither agreed or disagreed, 2 disagreed, and 1 strongly disagreed#2: Answered agree even though he was thinking of a job he found very boring. He said he would not have agreed if he’d been thinking about the roofing job he found interesting. For this R, an Agree response meant he often got bored.
#9: R said she had to “switch everything around” because it was phrased as “a negative”.
|
Work Support
The three questions in this series were more closely related than earlier questions, based on how the responses clustered. Six of the respondents answered all three questions the same (all 1s or all 2s).
In the sequence, if not ALL people at work were supportive, people one could learn from or people one could turn to for help, then several Rs did not answer Strongly Agree. They dampened their agreement if there were some people or department who did not meet these characteristics.
People at work are pretty supportive of me
Generally the idea of supportive people at work was well understood. Rs tended to talk about co-workers or direct supervisor caring about how they were doing, whether they were struggling with their job, giving encouragement and positive feedback.
#2: “I think the big thing about being supportive is helping someone when they are in need. We will go back to the roofing job. I will be doing something and someone could see that I was maybe struggling a little bit maybe trying to pull up a piece of plywood and its stuck. I am trying to do it myself because I want to get the job done and people are standing around and eventually somebody looks and notices and says hey you know I'll give him a hand. That is being supportive adding that extra hand, or it goes as far as to say nice job to somebody. I always try to confirm to people that they are doing a good job and then it comes back in my favor 90-percent of the time.”
#25: “You can’t really rely on positive feedback all the time from the employer. So that’s why I said I neither agree, nor disagree, because sometimes you’ll get credit, sometimes you won’t.” (R was not thinking of peers, just of boss)
#28: “They ask if you need anything if you’re real busy, you know, tells you’re doing a good job, like gives some feedback when you need it.”
|
There are people I can learn from at work
Some Rs were thinking about learning about the job or how to do job tasks, while others were thinking in wider terms and saying things like “everyone has something you can learn from”.
Otherwise, the question worked fine, with the caveat expressed above for this whole sequence.
|
There are people I can turn to for help in solving a work problem
Several respondents stressed that some people were not approachable or willing to help, or just would not be able to help, but others were.#2: “Yeah, there is always somebody that knows what they're doing. I always try to figure out the problem myself. If a question comes up or I don’t know to do it, you go and ask someone else. “
#25: “I feel that there are people that you work with that you would be able to go to for advice, or to help resolve an issue, and there are some people who will either escalate the issue, or make it work. So there’s appropriate people that you go to for problems or issues.”
|
Findings from Testing Form B2
Please answer the following questions thinking about your current job. Circle one answer for each question, indicating how strongly you agree or disagree with each statement.
Respondents focused on their current job (or their main current job) without difficulties. They reported being comfortable with the response scale provided, which they have dealt with before. They commented on the fact that job or school applications sometimes include questions in this format.
The jobs they had in mind while answering were:
#5: Artist representation and public relations work
#6: Construction work, building and repairing homes
#7: Financial consultant. “I work with companies, anywhere from small, to mid to large size companies as far as doing their employee benefits, profit sharing plans, things such as that. I’m really just kind of assigned to current accounts, and I maintain relationships, and help build more sales, really, within the companies that I work with”
#10: Makeup artist. “My main gig is seasonal, so it’s sort of October until April. And then I do freelance work in films, and commercial TV, that sort of thing kind of over the summer, and wherever I can squeeze it in.”
#11: Retail sales associate
#12: Manager in a spa
#15: Graphic designer in government agency
#16: Server at a restaurant
#24: Driver and dispatcher at towing company (family business)
|
Questions about Your Current Job
The intent of this set of questions showed to be ambiguous in administration. The questions could be aimed at finding out what respondents feel that staying over a longer time will let them acquire other people’s respect, eventually get to do satisfying work, eventually make enough money…, or eventually work with other people who share their interests and values. Or, they could be about continuing to get all those things (which, it would be implied, they are already getting at this job). R #7 answered thinking of gaining more respect in the future, while R#16 mixed time frames in her responses: “Getting respect, I answered about current. The others, I interpreted as “Will allow me in the future.” I was thinking like if I stay there forever, how would I feel?” The other respondents answered about their present situation at work, not about the future.
Remaining at my current job will allow me to … get respect from other people
Interpretation of ‘getting respect’ ranged from getting promotions, being hired sooner at a new job, “Respect in the sense that I just think of the way in which I am talked to, the way in which I’m interacted with”, “being professional with your co-workers”, prestige for the position in society at large,
Who are ‘other people’? Some respondents interpreted the question as referring to other people at work, while others answered about people in general.
#12: This is the only one that I actually had a pause, and I wasn’t sure if it was other people in my job, other people in the community. I kind of answered it from the perspective of other people in general in the community, my friend circle, my family and things like that
#15: This R was not sure how to answer so he answered 3. He said the question was ‘too vague’ but could not explain why. This R is a project hire. His term of employment will end in December. He knows he won’t be allowed to remain on the job beyond that time, which is bothering him a great deal. Therefore, he answered most of this series as Disagree, because the clause ‘remaining at my current job’ did not make sense in his case.
#24 Answered between agree and neither/nor because it depends on the people, some do and some don’t respect his job.
|
Remaining at my current job will allow me to … do work that I find satisfying
Other than the general note about this series of questions above, this question did not pose any particular problems. It was interpreted across the board as intended.
|
Remaining at my current job will allow me to … earn enough money for the lifestyle I want to have
A few respondents associated ‘lifestyle’ with high lifestyle: nice cars, travel, dining in expensive restaurants. Others more neutrally, as the style of life they want to live. Of interest:
#5: This R answered about the lifestyle, unrelated to the money. She does not make the money but because she is around artists, she lives some of their lifestyle.
#7: This R marked strongly agree when he meant strongly disagree – he thought his mistake was probably due to thinking about the fact that at present he makes more than he needs, but that is not the lifestyle he wants for himself in the long run.
|
Remaining at my current job will allow me to … work with other people who share my interests and values
For one of the respondents (#7) this question was double barreled: he explained that he strongly agrees as far as values are concerned, but that his interests are very different from other people at work.
No other issues identified. |
Job Satisfaction
I feel fairly well satisfied with my present job.
Only one R, #12, answered Strongly Agree. Three disagreed, and the remaining answered in between. There were no interpretation issues. Level of satisfaction had to do with R expectations, whether the job was in the type of work they want to be doing whether they felt they were successful in their jobs, etc.
|
Most days I am enthusiastic about my work.
Several Rs defined this as getting up in the morning and going to work happily, and being happy to be there once they arrive. “It’s that get-out-of-bed-in-the-morning feel.” Those who spoke about lack of enthusiasm, generally focused on the low pay.
No interpretation issues.
|
I find real enjoyment in my work.
Sources of real enjoyment in their work varied substantially across respondents. While some focused on the kind of work they like to do, others focused on work relations, others on the pleasure of knowing you are doing your job well, and even on just on the fact of having a job in this economy.
No interpretation issues. |
Job Persistence intentions
I plan to remain in my current job over the next year
Respondents were able to answer this question without much hesitation. Responses were generally independent of the answer to the subsequent question. That is, even though some Rs think about changing jobs, that did not necessarily go hand in hand with actual plans.
No interpretation issues identified.
|
I don’t usually think about leaving this job
This question appears to have low reliability. As they explained their answer, several respondents expressed conflicting feelings and were unsure of what to answer. It may be worth testing quantitatively for reliability.
No problems identified in interpretation. |
I feel pretty strongly committed to my current job
The notion of commitment to a job varied substantially across respondents. If this is a wider range of interpretations than intended, it may be advisable to make the wording more specific.
#6: “Commitment as showing up every day and doing my job to the best of my ability.”
#10: R interpreted commitment as passion for the kind of work she does.
#11: “Always following through on the work, showing up on time, following through with procedures, putting on a happy face, even when you’re not, trying to get along with all the people that you work with, just doing what you’re supposed to be doing and doing a good job of it”.
#12: “I feel a sense of ownership in the work that I do, and want to honor the relationship I have with my job.”
#15: R answered disagree, because his position is ending.. However, upon probing he said: “I know speaking for myself, like right now, no one really knows that as soon as the project ends, I’m basically making preparations to find something else. But also, at the same time, since I’m so committed to the job, and responsible with everything, and going to each project 110 percent, I don’t think that anyone can assume that I have other goals, or that I plan to leave.”
#16 “Committed meaning you plan on staying. You are committed to working there; committed meaning that’s what you’re already decided to stay.”
#24: “To be at least 85-90% focused on your job” |
Overall respondents’ comments about the questions
Generally respondents said the questions were easy to understand and answer. One R commented: ´I think maybe if there was a sentence that will reiterate it to like kind of stay focused on this job, and don’t bring in past jobs.” It may be harder for those with multiple jobs or short term jobs to keep thinking about the same job across questions. |
Findings from Testing Form C3
Please answer the following questions thinking about your current educational program. Circle one answer for each question, indicating how strongly you agree or disagree with each statement.
Respondents were in a variety of different programs, some pursuing degrees (Associate’s, Bachelor’s, and Master’s) and some in certification programs.
Version 1 Cases:
#1: in a 2-year certification program for a physical therapy assistant
#8: in a community college pursuing an AA – No major yet, but perhaps in communication
#19: pursuing a BA in political science
#23: pursuing a Master’s in counseling psychology
#30: pursuing an AA – no major declared but thinking about psychology
Version 2 Cases:
#3: pursuing an LPN degree
#13: studying to be a paramedic
#21: in an MBA program
#26: CPA preparation certificate program
#29: pursuing a Bachelor’s in Psychology
|
Education Self-Efficacy
In Form C v2, where these questions are presented as the last set, there appeared to be less discrimination by respondents across the first three items on confidence and certainty, which all five answered with the same rating across items (either Agree or Strongly Agree for all 3 items). In Form C v1, there was more variation.
I’m confident that I can earn strong grades in my education program
Version 1 Cases
Agree = 3 / Strongly agree = 2
Respondents had no difficulty answering. There were no issues of interpretation. What they considered strong grades ranged from A’s only to include up to Bs. Respondents explained their confidence either based on their motivation to do well or on their prior experience that they can get strong grades.
#1: R cited strong motivation as reason for her level of confidence – Strong grades = As
#8: Grades depend on one’s effort: “If you want a good grade, then you’ll get the good grade, and you’ll work as hard as it takes to get the good grade.” As and Bs
#19: “Basically, just looking at everything that factors into academic success, so quality of professor, availability of outside help with tutors or anything like that, factors that are more dependent on me, so like the amount of free time that I can commit to doing homework, versus study, things like that. And in my case, we do have tutors available at the school. The professor, I think, is doing a satisfactory job. And I have budgeted enough for myself that I’m able to have free time in which to work on this outside of class.” B or better.
#23: “Well, just based on the grades I’ve received in the past year and, you know, grades I received from my life in high school and undergrad, I feel like, you know, I’m capable of getting strong grades.” A, A-, B+.
#30: “I am confident, because for starters, I’ll be paying for everything, so if I don’t put in the time and effort, I’d be wasting my own time and money. I don’t have many distractions. I have the time to focus on my studies, and I have been focusing on my studies. The study skills class I took helped me refocus and get back into getting a proper schedule and a proper routine to complete the program.” Honor roll, A minimum of 3.0 GPA.
Version 2 Cases
Strongly agree = 2 / Agree = 3
#3: R was thinking of As and Bs. “I’m very confident. I study. I try and study a lot. I do know that there are some hard courses coming up, so I’m going to work my best to always get good grades. There are times when there are some things that you just don’t understand and your grades in that class might not be the best but you try the hardest that you can. That is why I put agree instead of strongly agree.”
#13: A’s. If it’s a B, I’m after school, whatever I can do to pull it up, because the paramedic program, that’s pretty competitive. They usually want you to have A’s. B’s, you usually have to get on a long waiting list to get in.
#21: B average or higher
#26: B’s and up
#29: A’s & B’s
|
I’m certain that I can understand the most difficult material presented in my courses
Respondents sometimes answered the questions thinking not about the most difficult material presented but about the most difficult courses. Others focused on the difficulty of the material, but answered about understanding it after using all available outside resources, including tutors in school or support from the instructors.
Version 1 Cases
Strongly agree = 2 / Agree = 2 / Neither-nor = 1
#1: Said agree (and not strongly) because “Probably my neurology class was a lot about the brain and certain scientific aspects that you are not—I was a biology major in my under grad and so I do understand a lot of it but there is certain things that you just might never get completely.”
#8: R was thinking only of the courses he is taking this semester. If he were taking math as he did last semester, he knows there is a lot he would not understand.
#19: R feels that being in the first days of his first semester back in school after a number of years, he is not yet sure how difficult the material can get. Thus, he answered 3.
#23: “I feel like the material that’s presented in the past year has not been—it’s been challenging, but it hasn’t been, you know, incomprehensible. I guess it’s just more a matter of how much time I allow myself to stay with the material, versus not being able to understand it.” R answered Strongly Agree.
#30: R said agree despite some concerns that in a community college he will not have the support necessary and will have to make it on his own.
Version 2 Cases
Strongly agree = 2 / Agree = 3
#13: “I strongly agree, because there’s just so much technology out here. You have the Internet. You have all type of library accesses. It’s always somebody that you can talk to about something you don’t understand.”
#21: R agreed that would be the case if one devotes enough time and accesses available resources
#26: “Yeah, I said I agree. There are very hard concepts in accounting, and I try my best to understand them. But if I were to reach out to maybe the instructor more in his time off, and try to understand them more thoroughly, then I would have put strongly agree. That’s why I put agree right now, because I believe that I go the mile, but I don’t go the extra inch.”
#29: R answered not about difficult material but about a difficult course because students were expected to argue and debate, which was hard for her to do.
|
I’m confident that I can do an excellent job on my education assignments
‘Education assignments’ uniformly interpreted across the board as in class or homework given by instructors. One respondent included readings after answering probes made him think of it. He said if he had included readings in his answer, he would have softened his confidence because he knows he does not do nearly all the readings assigned (#23, a grad student).
Version 1 Cases
Strongly agree = 2 / Agree = 3
#1: Based answer on strong motivation to do well.
#8: Based answer on his willpower.
#19: R’s confidence is based on comparing school with a full-time job. He feels school is less work, so he should be able to do an excellent job.
#23: R answered Agree: “I didn’t put strongly agree because I do feel like if I give myself adequate time with material, or if I had better time management skills, for example, then I would strongly agree.” R discussed doing an excellent job in terms of getting high grades, rather than on the quality of the work.
#30: R answered Agree: “It’s more about the amount of effort I put into it. I know that if I can minimize my distractions, I can achieve pretty much—I feel like I can achieve whatever I want. ”
Version 2 Cases
Strongly agree = 2 / Agree = 3
#13: R’s confidence is based on prior experience.
|
I’m confident that I could complete my degree despite financial pressures
A few respondents understood the item asked about one’s ability to focus on school work and complete the degree when financial pressures loomed large in one’s mind. Others interpreted as the sheer ability to pay for school (and therefore be able to stay on the program). For some Rs, to express confidence about something that they felt could be out of their control was a difficult notion.
Version 1 Cases
Strongly agree = 3 / Neither-nor = 1 / Disagree = 1
#1: R interpreted item as referring to whether financial pressures could interfere with one’s ability to focus on the program and complete the degree.
#8: R has faced such problems and was able to get loans and proceed with the program.
#19: R answered neither-nor because of concerns about difficulties to get loans.
#23: R has lots of financial support from his family and does not need to worry about financial pressures.
#30: R strongly agreed: “I have support from family and friends that will help me, you know, so money is not really an object this time around. And I do have a little bit of money saved up, so I feel like I’m in great—it’s going to be up to me if I complete school or not.”
Version 2 Cases
Strongly agree = 1 / Agree = 1 / Neither-nor = 1 / Disagree = 4
#3: R agreed. He felt that he has learned the hard way how to get financial assistance. “That way you can earn your degree just the same and still receive more like financial aid and more grant money where you don’t have to take out as many loans. I can find ways around it. There are always scholarships. I do complete different essays and different things like that so whatever it kind of takes where I don’t have to take out as many loans.”
#13: “That’s one of the confusing questions, because you really never know about tomorrow. And then the financial aid is usually pretty shaky. And then I know I don’t have all the funds for it, so I am dependent on the financial aid. So I’m sure that like if there was some type of law passed, or if they cut back on financial aid that would be something that’s like not something that’s in play for now, but you really can’t say if it did ever happen. So that question was kind of like I didn’t know if they was talking about right now. I didn’t know how to answer that one.” R answered neither-nor.
#21: R disagreed. He had recent experience that made him think that way: “Because I just recently failed my first college course period and that was part of the reason why. I’d be in class, sitting there hungry, and not being able to pay attention, so I wasn’t going. And I wasn’t completing assignments. And that was stress, because I wondered how I’m going to get money and all that kind of stuff. So just recently having an experience with that basically influenced the answer that I just gave, because I just went through that.” This R was thinking of inability to focus due to financial pressures.
#26: R disagreed: Yeah, because my financial obligations and pressures are really weighing heavy at the moment. I mean I want to be in the mindset of being confident that I am going to complete it. You know, if something tragic happens, I’m not going to be able to complete it.”
|
I’m confident that I could complete my degree despite having competing demands for my time (e.g., from work or family)
For some Rs this question was closely related to the previous one, because work demands are closely related to availability of funds for school.
Version 1 Cases
Strongly agree = 2 / Agree = 2 / Disagree = 1
#8: R has to work to be able to afford staying in school, which makes her worry about having enough time for school. She does feel the competing demands of work.
#19: R stopped working to focus on school and feels no competing demands. Hence his confidence.
#23: R does not work and does not have family in the area or making demands from afar.
#30: “I agree, but not strongly, because you can’t really expect the unexpected. There’s always something that happens. You know, I help take care for my mother, who’s disabled. There is a lot of outside circumstances that might stop me, slow me down. I can’t predict the future. If I could, I wouldn’t be going to school.”
Version 2 Cases
Strongly agree = 1 / Agree = 2 / Neither-nor = 1 / Disagree = 1
#13: “It’s pretty closely related, because work, that’s the other side of the money. So like if work becomes a little more demanding, like if they wanted me on full-time, I would have to, I mean the schedule’s pretty flexible, but I really can’t tell about tomorrow. So it’s kind of hard to answer questions about tomorrow when you are not really certain what tomorrow’s going to be like.”
#21: R disagreed: “Pretty much it’s from I’m not confident, not from the work, but from the family standpoint, because like I said, I just tend to get bugged about everything every five-minutes. And pretty much all this revolves around, once again, like I said, being able to manage your resources, being able to manage your time, and discipline, and being able to organize yourself, and focusing and concentrating. All of these are intangible qualities and characteristics that the culture of my family is not—I feel in my humble opinion does not understand. You cannot be pursuing any kind of goal with the attitude of I’ll put it off, and I’ll wait till the last minute. That leads to inefficient assignments, and inefficient productivity. So pretty much that’s why I said I’m not confident, because the culture of my family is not conducive to the qualities that it would take to be successful in education.”
#26: R agreed, but in giving an explanation R contradicted himself: “I guess it depends how heavy the demands are from my family or work. Well I’m out of work, so it’s probably from the family. Like if my mother gets sick and really I wouldn’t be able to complete my degree.”
|
Education Interests
I really like my education program on the whole
What respondents understand by ‘education program’ varies. Some talk about their interest in the career they have chosen, the occupation they are training for, or the courses they are currently taking.
Version 1 Cases
Strongly agree = 1 / Agree = 3 / Disagree = 1
#1: Strongly agreed: “It’s what I want to do now. It’s part of my occupation. You have to like it otherwise you shouldn’t do it. I do like it. It’s very interesting. It’s hands on so you can really-really see the meaning of what you are learning, and that is why I enjoy it.”
#8: R focused on the courses he’s taking this semester.
#19: “Yeah, I’m interested in the subject matter. I think that the faculty and the adjunct professors are pretty interesting, and certainly interested in the topic that they’re teaching. And I think that makes all the difference. When you’re actually interested in something that you’re studying, and when you have somebody teaching it to you that’s equally interested.”
#23: “I don’t know, just maybe like frustrations, and methods in terms of expressing criticism about professors, or classes, and just things on those lines. But I guess at Northwestern, we’re on a quarter system, and I’m not a huge fan of that. I think it’s just too much pressure, and you don’t have enough time with the material… But then again, I chose to go to Northwestern, so I knew what I was getting myself into.”
#30: “Well the program, you know, I don’t feel like it’s customized to—you know, I don’t feel like there is really much, you know, they give the classes, and everything is out there, but it’s pretty much you have to pick your own program. …That’s why I would neither agree, nor disagree with it, because it’s pretty much me. I picked it on my own. Counselors weren’t as much help.”
Version 2 Cases
Strongly agree = 1 / Agree = 3 / Between Agree and Neither-nor = 1
#3: “Like for the medical assistant program I just felt like we have to take a pharmacology program. Within that program medical assistants are not allowed to write prescriptions. We are not even allowed to administer medicine. So, I just didn’t feel there was a need for that program. As far as that, that was probably about it. Now some of the courses were a little bit, were a little hard I guess, AP, Anatomy and Physiology, you know to learn all of the terms and memorize them, and you know how to talk in a certain language, the medical language and everything it can be a little frustrating, but I made it through it pretty good.”
#13: “I agree, because you’re just really hands-on, a lot of practicals. It really gives you a good feeling, so that when you do go out into the field, oh yeah, we went over this is class, or we went over this in the study, or I remember talking to my teacher about this. I’ll run into it, and then I already kind of know what those are signs.”
#21: “The fact that I’m learning something I can use, the fact that it’s a convenient class. I only have to go one day a week. I also like the fact that the faculty are people who are working in the field, as compared to my undergrad, some of them weren’t necessarily in the field, per se. I just feel like you could relate to them more.”
#26: “I said agree. Some of the professors are helpful. Some of them are not helpful, though I would like them to be. The books are okay. It’s a typical institution. You know I mean I probably may lean towards the neither agree, nor disagree, because I don’t think everybody learns the same way, you know, off of just reading a book. I’m a visual, hands-on learner. I would probably have to change that to neither agree nor disagree if that is okay. What do I like? I like some of the people that I bump into in class and the networking. I like learning new concepts. I get along with most of my professors. So that’s what I like about the program.”
|
I am enjoying the courses in my field of study
This question was problematic for those who do not have a field of study, still doing general education requirements for college. For those who already answered the prior question thinking of courses, this question seemed repetitive. No interpretation problems detected.
Version 1 Cases
Strongly agree = 1 / Agree = 3 / Neither-nor = 1
#30: R has no major declared but in answering was thinking only about courses he is interested in, and excluding other required ones.
Version 2 Cases
Strongly agree = 2 / Agree = 2 / Neither-nor = 1
#26: This R was impatient to complete his program and start working in the field. “So I’m semi enjoying what I’m learning, and kind of not at the same time, because I want to use it in real life.”
|
I get totally absorbed in what I am studying
There was variation in the interpretation of “Getting absorbed”. For some Rs this referred to the moment when one is in class or doing school assignments and can concentrate to the point of becoming oblivious to everything else, being very focused on the material at hand. For others, it referred to school work taking over all other aspects of life.
Version 1 Cases
Strongly agree = 1 / Agree = 3 / Neither-nor = 1
#1: “Don’t get distracted by other things going on, really be interested in it when you are actually reading it. Not as reading it and forgetting it. It actually sticks in your mind, and you memorize it kind of something.”
#8: “No distractions while working on assignments”
#19: R answered Neither-nor: “As a blanket statement, I don’t think that that works. I think at times, I’m definitely very interested and absorbed in what I’m working on. I think there is boring parts to everything though.”
#23: “I guess I’m just not the type of person to get really enmeshed in his work. It’s just it hasn’t happened for me.” R did not think of absorbed in a specific task, but more generally.
#30: R used ‘absorb’ both in the intended sense and in the sense of taking in the material: “I feel like I’m able to focus for once, so I can get absorbed. You know, I can drown out, I turn my phone off and put it in the other room, and just focus on what I need to do. I feel like I’m absorbing it. I feel like when I’m in class, I’m a sponge, and I just soak up as much information. I’m not trying to like talk to the girl next to me or anything like that. I’m there to learn.”
Version 2 Cases
Strongly agree = 1 / Agree = 2 / Neither-nor = 2
#3: “that I’m 100-percent all about school. Nothing else matters but school but this program and it’s not like that for me.”
#13: “Absorbed, to really take it all the way in. To be absorbed means to soak it all in like a sponge, just take in every element of it and just embody it.”
#21: “Absorbed means like you’re just oblivious to everything else.”
#26: “Like you’re focused, just you and the material.”
|
Education supports
I feel support from important people in my life for completing my education program
Respondents spoke both of emotional support and financial support. No interpretation issues. Some Rs had difficulty answering because they had some important people in their lives who give support and others that don’t.
Version 1 Cases
Strongly agree = 5
#8: R feels his full family rooting for him; he will be the first grad in the family.
#19: “They have been very supportive of all of my choices, but particularly to go back to school. So my parents have definitely gone out of their way to help me with this, and my sisters, as well. I mean everybody just seems to be kind of pulling for it.”
#23: “My parents are supporting me financially. You know, they’re always like encouraging me to do that. Maybe not at first, I don’t know if at first they were excited about me becoming a therapist, for various reasons, but in general, they are very supportive.”
Version 2 Cases
Agree = 2 / Neither-nor = 1 / Disagree = 2
#3: R disagreed. She was thinking about her family who does not give her support. She was also thinking about financial support. She gets help and moral support from her boyfriend but was not thinking of him in answering this item. She said she thought about him in subsequent items.
#13: R marked Agree. “For the most part, I didn’t want to mark disagree or strongly disagree, because there is at least one or two people, so I just kind of focused on the one or two people.”
#21: R disagreed: “Yeah, I don’t feel them to be supportive, because to me, support is working with me and understanding. And when you’re egotistical and egocentric, just calling it what you want, instead of creating a mutual understanding, then there’s no support.”
#26: “Because some people support, and some people don’t. And it’s just I want to go with my gut feeling to finish it and whatnot. So that’s why I put neither agree, or disagree.”
|
My family members support my choice of education program
Some Rs indicated that their family does not know much about their education program. Rs know more clearly if their families support their furthering their education, but when it comes to the specific program, not all families are involved, and that makes it hard to answer the question.
Version 1 Cases
Strongly agree = 3 / Agree = 1 / Neither-nor = 1
#1: “They know my interest in it. My dad and my brother are massage therapists so there is kind of a background there so it seems like everybody supports that.”
#8: R did not answer specifically regarding choice of program. He said: “Well I mean they only know what I tell them. I’m completely 110 percent independent, and whatever I decide to do is what I decide to do. They’re just there to back me up.”
#19: “I didn’t really know what to interpret the question as, whether it was referring to like a major, like a political science, or just in being a program at all, so I interpreted it as the latter. They certainly want me to be back and school, and are happy that I’m there.”
#23: “I think now they do, now that they see sort of that there are a lot of job opportunities. I think one of my parents, in particular, was very focused on… they wanted me to get a PhD or something. I think that was more part of if then the actual counseling. It’s better now.”
#30: R answered neither-nor because some family members did and others did not.
Version 2 Cases
Strongly agree = 1 / Agree = 2 / Neither-nor = 2
#3: R answered neither-nor because her family is not involved.
#13: R marked Agree, even though he explained his mom and girlfriend are very supportive in every way while the rest of the family is not.
|
I have access to a good mentor or advisor who can offer me advice and encouragement
Some Rs wondered if this was asking about a mentor or advisor at school or outside of school. Some did not focus on a single person but rather on multiple people. At least in one occasion, the respondent made a distinction between advice and encouragement.
Version 1 Cases
Strongly agree = 2 / Agree = 1 / Neither-nor = 1 / Disagree = 1
#1: R wondered if at school or outside but decided to answer about the program. R would expect advice from school and encouragement from friends or family.
#8: R answered thinking only of his family. Upon probing, he said: “now that we’re talking about it, I’m thinking I formed an outside relationship with my English professor last semester, and I know that in any type of qualm that I might have, she would be the go-to girl.” For R: mentor is outside of school, advisor at school.
#19: “So within the context of that mentor or advisor being like somebody in a professional capacity at school, no. They’re there, but it’s sort of the same scenario that I painted with the financial aid officer there, understaffed and overworked. I ultimately looked at quality and service efforts. The other way that I could have seen it interpreted is just in general in my life, do I have that. And I would say yes, I do, but not knowing what the question is, I chose a neutral answer.”
#23: “Oh, definitely, although I don’t really have like an academic advisor. …I do have other people that have been supportive. In our first year in the program, we had this – it was just like a part of one of our classes is called our back home supervisor, which is like we’d go there and talk about sort of our experience in the counseling room, versus talking about our clients. And that helped us a lot, and we can kind of talk about almost, but like almost everything in terms of counseling is on the floor to discuss. That was very helpful. And then I also have my own therapist, so I could talk about different things about my program there. So I’ve had access to different mentors, for sure.”
#30: R was thinking only about in-school mentor or advisor.
Version 2 Cases
Agree = 3 / Disagree = 2
#3: R sees a mentor as more involved while an advisor advises about specific things.
#21: R agreed but was not thinking of a single person, rather about his circle of friends.
#26: R was only thinking about in school.
|
Education Persistence Intentions
For this set of three questions, all five v1 respondents gave the same answer for each of the three items (either strongly agree or agree for all 3). In v2, all but one respondent also gave the same answer across the three items. In explaining each response, they referred to very similar notions of determination and motivation.
|
I am totally convinced that I will complete my education program
Respondents did not have problems in answering this question. No interpretation issues were apparent. Most responses focused on student motivation to complete the program.
Version 1 Cases
Strongly agree = 4 / Agree = 1
#8: “Well, because I’m certain that I want to finish school, and so usually, if you want to do something, you do it.”
#19: “I made a realistic timetable of how I’m going to progress through coursework. I started budgeting for this like a year-and-a-half ago, so I have that mostly figured out. You know it’s a lot of the scary variables that you have to plan for. And truly, the only thing that’s left is actually performing well enough in the classes to pass them. So I think as long as you can take those controllable steps and get them under control early enough, it becomes much more realistic proposal.”
#23: “I think so. I mean I’ve done well in my first year. I can’t see myself sort of falling apart in the second year. I think I should be okay in terms of finishing. I don’t think I’ve ever had that happen before, so it would kind of surprise me.”
#30: “I don’t feel like failure’s an option. There is no way that I’m going to go through this a second time and fail.”
Version 2 Cases
Strongly agree = 2 / Agree = 2 / Neither-nor = 3
#3: R did not pick Strongly Agree (chose Agree), because she may decide instead of completing the program to continue studying toward a higher degree (RN instead of LPN). She would then complete the requirements but apply them toward a different program.
#21: Neither-nor because ‘anything could happen’
#26: R is not convinced because of financial worries.
|
I think that earning a degree or certificate in my field of study is a realistic goal for me
Some Rs felt this item overlapped a lot with the previous one. There was consistent interpretation of ‘realistic’ across interviews.
Version 1 Cases
Strongly agree = 4 / Agree = 1
#1: So, to me it's definitely realistic because I know that I'm doing well and there are no other distractions for me.
#8: “a realistic goal is something that’s real, and not just far-fetched. So a realistic goal would be like okay, I can complete a degree in two-and-a-half to three years. That’s realistic. Okay, I would like to complete my degree by the end of this year, not realistic.”
#19: “I thought that it was the same as the question we just went over, just in other words.”
#23: “I mean it’s very similar to the first question. I think it’s very realistic. I mean graduate school is challenging, but I think I can get through it, and be okay.”
#30: “Exact same thing as above. Failure is not an option this time around.”
Version 2 Cases
Strongly agree = 2 / Agree = 3
#3: “Like is it something that you can achieve realistically within a certain amount of years, within a timeframe and everything. I guess like if I was going to be a heart surgeon, the question would be am I going to stick to that or am I going to take so many years, or am I going to drop out and everything. So, is it realistic? Is it something that I think I can really achieve.” |
I am fully committed to completing my education program
No interpretation issues. Respondents interpreted ‘full commitment’ consistently.
Version 1 Cases
Strongly agree = 4 / Agree = 1
#1: “That means not letting distractions like work and family, or whatever get in the way. You know putting all of your strength forward and trying to do your best and not just getting by.”
#8: “Yes, and I am fully committed to it. Like I said, I took off last semester because I didn’t really know what I wanted to do. I didn’t want to be wasting my time, the government’s money on just classes that I didn’t know what I was doing. And plus, I just wanted to make sure that I had financial security, so I looked for a job and I found one. And now that I have that, I’m acclimated there. I feel more confident that I can start school knowing that I don’t have to worry about necessarily where my next paycheck is coming from, how I’m going to be able to survive, and get to school and things like that.”
#19: “For me, it’s realizing what the purpose of being there is in the first place. I went to college right out of high school because it just seemed like the next logical thing to do, because I didn’t know what else to do, and for that reason, I’m absolutely certain that’s why I didn’t finish. I got to a point where I just didn’t really know why I was there.”
#23: “I mean with commitment comes sacrifices. In a sense you might sacrifice certain things, like friendships, or like not having as much free time, not making money for school. So having all those things shows signs of commitment to a program. Yeah, I mean I’ve already gone through a year of this, over a year of those things. So I would think I’m committed.”
Version 2 Cases
Strongly agree = 2 / Agree = 3
#3: “Finish up the program whatever it takes to finish the program.”
#13: “Strongly, because nothing’s going to get in my way for this. When I hear committed I think about married to, like I’m just kind of joined to, or bonded to.”
#21: “Committed means that you’re dedicated. Commitment means that you’re willing to do the necessary actions. You’re willing to take the necessary actions, and do the things that have to be done to achieve what you want.”
#26: “I didn’t put Strongly Agree because if a job came into play, I would be more committed to that than the education.” |
Research Support Services, Inc.
November 7, 2011
Introduction
This report summarizes findings of the cognitive testing conducted by Research Support Services Inc. for the ELS 2002/12 study. The cognitive interviews were conducted in Illinois, in the Chicago SMSA, between September 19 and October 23, 2011. A total of thirty interviews were conducted using three different sets of questions. Form A tested for Loans and Financial Literacy, Form B tested for Grants and Scholarships, and Form C for Financial Burden, Volunteerism, Voting and Social Capital.4
For recruiting participants, an advertisement asking for interested candidates for the interviews was posted on Craigslist on the Internet. Callers were screened using the screener in Appendix 1. The respondents interviewed had the following age distribution: age 24 (7 respondents), age 25 (8 respondents), age 26 (6 respondents), age 27 (2 respondents), and age 28 (7). Fifteen were women and 15 were men. As for race/ethnicity, three respondents self-identified as Asian, seven as African-American, four as Hispanic, 15 as White, and one as Other (‘mixed’). Respondents were screened also for level of education achieved, and having had scholarships, grants, and loans to finance their post-secondary education.
While many of the questions tested are working very well, testing identified important interpretation issues in other items.
The
following table displays respondent characteristics.
R# |
Age |
Gender |
Race-Ethnicity |
Form Tested |
Highest schooling |
Loans |
Grants/ Scholarships |
ASG001 |
25 |
F |
White |
A |
Graduated college |
Yes |
Yes |
ASG002 |
28 |
F |
White |
C |
Graduated college |
No |
Yes |
ASG003 |
28 |
F |
Asian |
C |
Graduated college |
No |
No |
ASG004 |
26 |
F |
Hispanic |
B |
Graduated college |
Yes |
Yes |
ASG005 |
24 |
F |
Mixed |
B |
Some College |
Yes |
No |
ASG006 |
25 |
F |
African-American |
A |
Some College |
Yes |
Yes |
ASG007 |
24 |
M |
White |
A |
HS diploma |
Yes |
Yes |
ASG008 |
26 |
M |
White |
B |
Graduated college |
Yes |
Yes |
ASG009 |
28 |
M |
White |
C |
Graduated |
Yes |
Yes |
ASG010 |
25 |
M |
African-American |
A |
Technical diploma |
Yes |
Yes |
ASG011 |
26 |
M |
African-American |
B |
Some College |
Yes |
Yes |
ASG012 |
27 |
M |
White |
C |
Some College |
Yes |
Yes |
ASG013 |
28 |
F |
White |
C/ B |
Graduated college |
Yes |
Yes |
ASG014 |
26 |
F |
African-American |
A |
Some College |
No |
Yes |
KK001 |
25 |
M |
White |
C |
Graduated college |
No |
Yes |
KK002 |
25 |
F |
White |
C |
HS |
No |
No |
KK003 |
24 |
F |
Asian |
A |
Graduated college |
Yes |
Yes |
KK004 |
25 |
M |
Asian |
C |
Graduated college |
Yes |
Yes |
KK005 |
27 |
M |
African-American |
A |
Some college |
Yes |
Yes |
KK006 |
24 |
F |
White |
A |
Graduated college |
Yes |
Yes |
KK007 |
24 |
F |
White |
B |
Graduated college |
Yes |
Yes |
KK008 |
26 |
M |
African-American |
A |
Some college |
Yes |
Yes |
KK009 |
28 |
M |
White |
C/ B |
Graduated college |
Yes |
Yes |
KK010 |
25 |
F |
White |
B |
Graduated college |
Yes |
Yes |
KK011 |
25 |
F |
White |
A |
Graduated college |
yes |
no |
KK012 |
28 |
M |
White |
C/ B |
Some college |
Yes |
Yes |
KK013 |
24 |
M |
African-American |
B |
Some college |
No |
Yes |
KK014 |
28 |
M |
Hispanic |
B |
Graduated college |
No |
Yes |
KK015 |
26 |
M |
Hispanic |
B |
Graduated college |
Yes |
Yes |
KK016 |
24 |
F |
Hispanic |
B |
Graduated college |
Yes |
Yes |
Findings by Form
Form A – Loans
A-1. Did you take out any loans to help pay for your undergraduate education since high school? (Do not include money you borrowed from family or friends.)
Yes go to A-2
No. go to A-7
We elicited some unexpected patterns of interpretation for ‘undergraduate education’, or did not pay attention to the qualifier about the level of schooling.
|
A-2. What types of education loans did you take out?
(Check all that apply.)
Federal student loans, such as Stafford or Perkins loans. go to A-5
Private loans, such as Sallie Mae, CitiAssist, Chase, or Nellie Mae loans. (Federal loans usually have low interest rates, and are received after completing a Free Application for Federal Student Aid, or FAFSA. Private loans usually require a co:signer and have market interest rates based on credit history.) go to A-3
Responses: About two thirds had federal loans only. The others had both private and federal. KK011 also wrote in: ‘PCN loans” “Credit cards”
Noteworthy Issues: Respondents exhibited confusion or lack of understanding about loans. Not only did some not know which specific loans they had or even what type, but some were confused about the loans vs. grants, and about who gives out a federal loan.
|
A-3. How much in total did you take out in private loans for your undergraduate education since high school?
$ ____________
(If you are unsure, provide your best estimate.)
Noteworthy Issues: There was not one specific problem but a few cases of lack of information or inconsistent amounts.
|
A-4. How much do you still owe on your private loans? $____________If owes >$0, go to A-5; else go to A-6
(If you are unsure, provide your best estimate. If you no longer owe money on your private loans, enter 0.)
Most respondents estimated the amount, not remembering the exact amount.
Noteworthy Issues:
|
A-5. How much do you pay each month for your federal and/or private loans? $________
(If none, enter 0.)
Responses:
Except for one respondent they were able to answer with an amount ranging from zero to $250. The exception was KK006 who did not enter an amount but wrote in a question mark and “Various amounts depending on loans, interest and my available funds.”
Noteworthy Issues: Some Rs were not very clear about details of their loans. In some cases it seemed that parents may have taken care of the loan details. Not all were clear as to how much they have to pay for how long.
|
A-6. While you were still enrolled as an undergraduate, did you receive any of the following information about your student loans?
(Check all that apply.)
Information on options for repaying your loans.
Information about how much your monthly loan payment would be.
Information about what to do if your loan payments became too much to handle.
Information on what might happen if you didn’t make your loan payments or defaulted on the loan.
Other (specify)
Responses:
Noteworthy Issues:
Respondents reported receiving different types of information through different means.
|
A-7. Have you ever learned how to manage money in a class?
(Please indicate Yes or No for each row below.)
Yes No
|___|___| At school, in a subject or course specifically about managing money
|___|___| At school as part of another subject or class
|___|___| In an activity outside school
Responses:
Noteworthy Issues: If ‘school’ in this question is intended to be just high school or just college, it should be specified. Respondents assumed one or the other, but not necessarily both. Also respondents interpreted ‘activity outside school’ in varied ways.
|
A-8. How often do you discuss money (e.g. talk about spending, saving, banking, investment) with these people?
(Please check one box in each row.) |
Never or hardly ever |
Once or twice a month |
Once or twice a week |
Almost every day |
Parents/guardians or other adult relations
Spouse or partner
Friends
Responses:
Noteworthy Issues:
In answering, several Rs were thinking about discussing their ability to do things or not given their cost. This was true, in particular, regarding talking with friends. Several added ‘earning money’ as another topic they discuss. The undefined (present tense) timeframe of the question appears to lend itself for reporting about events that happened even a year ago, and in prior relationships.
|
A-9. If you don’t have enough money to buy something you really want (e.g. an item of clothing, sports equipment), what are you most likely to do?
(Please select one answer below.)
Charge it
Try to borrow money from a family member
Try to borrow money from a friend
Save up to buy it
Not buy it
Responses: All but one replied ‘Save up to buy it’. Only one respondent answered they would not buy it.
Noteworthy Issues: This question and the next one seem to elicit socially acceptable responses rather than be a good reflection of Rs’ attitudes toward saving/borrowing. This was more noticeable in the next question. In most cases, the examples below show that in probing the responses did not validate the answer given.
|
A-10. Which of these statements about saving money best applies to you?
(Please select one answer below.)
I save the same amount of money each week or month
I save some money each week or month, but the amount varies
I save money only when I have some to spare
I save money only when I want to buy something
I do not save any money
I have no money so I do not save
Responses:
Noteworthy Issues: In a couple of cases, Rs answered they save when they want to buy something. Yet when probed about the last time they did so, they could not think of any examples. Rs seemed to be giving a socially acceptable response: what they thought they should be answering.
|
A-11. Do you have any of the following?
(Please check one box in each row.) |
Yes |
No |
I don’t know what it is |
Bank checking
Bank savings (including money market,
certificates of deposit, etc)
Credit card/line of credit
Debit card
Responses:
Noteworthy Issues:
Question worked very well. No interpretation issues identified. The debit card Rs referred to was the type that is both a debit and ATM card.
|
1.2 FORM B – Scholarships and Grants
B-1. Did you ever receive a grant or scholarship to help pay for your undergraduate education?
Yes go to B-2
No administer form C
Responses: Because Rs who received form B had been prescreened for having had grants or scholarships all unanimously answered Yes here. Some respondents were confused as to what is a grant vs. a scholarship as well as to what counts as undergraduate education.
Noteworthy Issues:
|
B-2. While enrolled at the last college you attended, did you ever receive a grant or scholarship to help pay for your undergraduate education?
Yes go to B-3
No go to B-8
Responses:
Noteworthy Issues:
Probing uncovered a certain degree of confusion as to the reference period for this question. Some respondents also felt this question was the same as B-1.
|
B-3. During the last academic year in which you were enrolled in as an undergraduate, did you receive any grants or scholarships?
Yes. go to B-4
No. go to B-8
Responses:
All but two respondents answered Yes. ASG008 continued to answer the rest of the questions without attention to the skip instruction.
Noteworthy Issues:
|
B-4. How much of the tuition and fees for your last year of undergraduate enrollment did those grants or scholarships pay for?
All
More than half, but not all
Less than half
Responses:
Most Rs answered ‘More than half but not all’, with the following exceptions:
Noteworthy Issues:
Not all Rs were clear as to what fees cover.
|
B-5. During the last academic year in which you were enrolled in as an undergraduate, how much would you estimate you received in grants or scholarships?
Responses:
Noteworthy Issues: Overall, Rs were not always sure of the amounts received. However most provided answers on the questionnaire. Upon probing the uncertainties and confusions became evident.
|
B-6. While enrolled at the last school you attended as an undergraduate, did the amount of your grant or scholarship stay the same each year you went to school there, or did it change?
It stayed the same
It changed
I was only there a year
Responses:
Noteworthy Issues:
Assorted problems were identified in the responses. For the respondent who only spent one year at their last undergraduate school, the question cannot be answered.
|
B-7. While enrolled at the last school you attended as an undergraduate, how much did you receive each year in grants or scholarships? If you are unsure, use your best estimate.
$________________
Responses:
Noteworthy Issues:
Some respondents answered this question as though it were asking for an annual average, while others answered as a total across all years.
|
B-8. Did you receive any grants from the federal government, like the Pell Grant, to help pay for your undergraduate education since high school?
Yes
No
Responses:
Noteworthy Issues:
No particular issues detected.
|
1.3 Form C: Financial Burden, Volunteerism, Voting, And Social Capital
C-1. Many young adults experience financial problems. How much stress have you felt in trying to meet your financial obligations during the past year?
1 Not at all stressed
2
3
4 Moderately stressful
5
6
7 Extremely stressful
Responses:
There was a good distribution of responses across the scale. The number in parentheses represents how many Rs endorsed that number on the scale.
1 (3) 2 (0) 3 (1) 4 (2) 5 (1) 6 (1) 7(2)
Noteworthy Issues:
Assorted small issues came up in administration and probing:
|
C-2. How difficult is it for you to pay your bills on time? These bills might include insurance, rent, mortgages, car payments, credit cards, etc.
1 Not at all difficult
2
3
4 Moderately difficult
5
6
7 Extremely difficult
Responses: There was a good distribution of responses across the scale. The number in parentheses represents how many Rs endorsed that number on the scale.
1 (4) 2 (1) 3 (1) 4 (2) 5 (0) 6 (1) 7 (1)
Noteworthy Issues:
No issues of interpretation identified. Rs mentioned types of issues that could create stress such as credit card debt, car payments.
Respondents had suggestions for other examples that might be included, relevant to them.
|
C-3. How much burden do you feel from debt (from credit cards, mortgages, personal loans, etc.)?
1 No burden at all
2
3
4 A moderate burden
5
6
7 Extremely high burden
Responses:
Most respondents either answered 1 (no burden at all) or 5 (more than a moderate burden)There was a strong tendency. Responses were 1 (4), 3 (1), 4 (1), 5(3).
Noteworthy Issues:
No interpretation issues were identified.
|
C-4. In the last 12 months, have [you or your spouse or partner] donated money or goods with a combined value of at least $25 to religious or non-profit organizations, such as schools, hospitals, museums, charities, or the United Way?
Yes go to C-5
No go to C-6
Responses:
Seven respondents answered Yes, and three No.
Noteworthy Issues:
There were some interpretation issues with ‘goods’. Apparently two respondents interpreted ‘goods’ as doing something good.
|
C-5. What is the total value of the money or goods that you or your spouse or partner have donated to these causes in the last 12 months?
$25 - $100
$101- $500
$501 - $1000
More than $1000
Responses:
Noteworthy Issues:
|
C-6. Which of the following types of organizations are/were you involved with in your volunteer or community service work? Were you involved with...
(Please indicate Yes or No for each item below.)
Yes No
|____|____| Youth organizations, such as coaching Little League or helping with the scouts?
|____|____| Service organizations, such as Big Brother/Big Sister or the Red Cross?
|____|____| Political clubs or organizations?
|____|____| Religious or spiritual organizations, including churches, synagogues, and mosques (but not including attending worship services)?
|____|____| Community centers, neighborhood improvement, or civic associations or groups?
|____|____| Volunteering in a hospital, nursing home, or retirement community or in a program making home visits to people in need?
|____|____| Educational organizations, including schools and libraries?
|____|____| A conservation, recycling, or environmental group such as the Sierra Club or the Nature Conservancy?
|____|____| A group providing international aid or promoting world peace?
|____|____| A group that helps people in need of food, shelter, or other basic necessities?
|____|____| Activities related to arts or culture?
|____|____| Any other kind of group or organization?
Responses:
Noteworthy Issues: This question does not provide a time frame. Additionally, the verb tense is ‘are/were’, which suggests to the R that the question is about the present and some vague past period. Rs were inconsistent in their use of time frame; while some were letting the timeframe carry over from the past two questions (‘In the last 12 months’), others were thinking of the present.
There were inconsistencies in including an activity depending on how often it is done.
Some organizations were double reported under different categories:
|
C-7. Were you registered to vote in the November 2, 2010 election?
Yes
No
Responses: Two Rs answered No, all others answered Yes.
Noteworthy Issues:
|
C-8. Which of the following things have you done during the last 12 months? Indicate all that apply.
contributed money to a political party or candidate
contacted a government official regarding political or community issues
run for a public office
run for a non-public office
attended a political rally or march
filed a tax return
The question does not offer a way to answer for someone who did nothing listed. There would be no way to tell if someone skipped the question by mistake.
Responses:
Most Rs answered only ‘Filed a tax return’. The exceptions were:
KK009: Contacted government official; attended rally; filed tax return ASG003: Marked no responses. R was living out of the country until very recently. ASG012: Contacted government official; tax return
Noteworthy Issues:
The concept of a ‘non-public office’ was not clear to Rs. Respondents were asked for examples.
|
C-9. People sometimes belong to different kinds of groups or associations. The next few questions contain different types of groups. For each type of group, please select a response to say whether you have participated in the activities of this group in the past 12 months. (For each item below, please answer with one of the following response options:
A political party, club or association
A trade union or professional association
A church or other religious organization
A sports group, hobby or leisure club
A charitable organization or group
A neighborhood group or association
Another voluntary organization (please specify)
Responses:
Noteworthy Issues: Although the reference period is mentioned, Rs did not always see it. The length or complexity of the item may be the cause. There was perceived overlap with question C6. The concept of ‘trade union’ was not clear to all. The distinction between membership and participation was not always clear to Rs.
|
1 Analysis of Form A interviews was conducted using the interview transcripts that appear in Appendix 2
2 Analysis of Form B interviews was conducted using the interview transcripts that appear in Appendix 3
3Analysis of Form C interviews was conducted using the interview transcripts that appear in Appendix 4.
4 In three instances respondents who had screened in as having grants or scholarships ended up answering No to having grants or scholarships during their last year in school (Q. B3). Because these interviews only took about 10 minutes beyond the consent form, the interviewer administered form C as well. Responses to both forms are included in the discussion of findings.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | sji |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |