2012/17 Beginning Postsecondary Students Longitudinal Study (BPS:12/17) Cognitive Testing – First Round

Attachment III
Eligibility Screening Questions

Intro: Thank you for inquiring about this study. I need to ask you a few questions to determine if you are eligible to participate. The screening is expected to take approximately 5 minutes.

1. What is your full name?

______________________________________________________________

2. What is the best telephone number and email address at which to reach you?

______________________________________________________________

3. How did you hear about this study?

_ Craigslist posting

_ Facebook posting

_ Other online posting

_ Flyer

_ Word-of-mouth

_ Other_____________

4. In which month and year did you first attend any college, university, or trade school after completing your high school requirements?

- Month

- Year

(If July 1, 2010 through June 30, 2012, go to question 5; If prior to July 1, 2010 or after June 30, 2012, Not eligible)

5. What is the name of the first college, university, or trade school you attended after completing your high school requirements, and in what city and state is it located?

______________________________________________________________

6. What degree or certificate were you working on at [FIRST SCHOOL NAME] between July 1, 2010 and June 30, 2012?

_ Associate’s degree (usually a 2-year degree)

_ Bachelor’s degree (usually a 4-year degree)

_ Undergraduate certificate or diploma (usually less than 2 years), including those leading to a license (example: cosmetology)

_ Undergraduate level classes

_ Other__________________ (Not eligible)

7. Prior to attending [FIRST SCHOOL NAME] and after you completed your high school requirements, did you earn a degree or certificate at any college, university or trade school?

• Yes (Not eligible)

• No

8. Are you currently attending any college, university, or trade school?

(Answer "No" if you are on a break between semesters, terms, or quarters, such as on summer break, and not currently attending classes.)

• Yes

• No

If eligible: Thank you for answering our questions. You are eligible to participate in this study.

What days of the week and times of day would you be available to participate in an interview?

___________________________________________________________

Thank you. We will contact you soon to schedule an interview time and answer any questions you may have about the study.

Participation will involve answering questions and reviewing a survey for an upcoming study that will be used to learn about the experiences and choices of individuals who entered college at the same time, whether or not they are still enrolled in college. During the interview, we will ask you to complete a survey and give us your thoughts on the questions you are being asked. In addition, we will ask you questions as you go, in order to measure your understanding and interpretation of the questions in the survey. 

Each full interview is expected to last approximately 60 minutes. The interviews will be recorded so that we can review your ideas and comments. We will prepare a report that does not single out any one person, but will summarize what we heard from the group as a whole. We will keep the recording only until our report is finished. No names will be mentioned in the report.

After your interview has been scheduled, you will receive a follow-up message to confirm the date, time, and location. At that point, you must RSVP to the invitation to guarantee an interview time. Upon completion of the interview you will receive $40 to thank you for your time.

If ineligible: Thank you for inquiring about this study. If you are eligible to participate, we will contact you at a later date.

Attachment IV
Cognitive Interview Protocol

Section 1: Introduction

Welcome participant and thank them for their time.

Introduce self: [Name/RSS employee]

Read or paraphrase: As we mentioned in our earlier conversations, we are working with the National Center for Education Statistics, within the U.S. Department of Education, to develop and refine questions for the Beginning Postsecondary Students Longitudinal Study (BPS). BPS tracks students’ paths through postsecondary education and helps answer questions of policy interest, such as why students leave school, how financial aid influences persistence and completion, and what percentages of students complete various degree programs. Today, I’m hoping to get your honest feedback on the survey questions we are planning to use.

During the interview, I will ask you to read the survey questions and respond to them. There are no right or wrong answers. In this study, I am less interested in your actual answers to the questions and more interested in what you thought about the questions and how you understood them. After you have gone through the survey, I will ask some questions about your answers, such as what you understood the questions to mean and whether the questions reflect your actual circumstances.

In order to concentrate on what you are saying rather than having to take notes, and to make sure that we don’t miss anything that you say, we would like to audio record this interview. Is that okay with you? [IF NOT OKAY WITH RECORDING: Okay, that’s no problem.]

I want you to know that your participation is voluntary and your responses are protected from disclosure by federal statute. Responses that relate to or describe identifiable characteristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose, unless required by law. We want you to know that you can speak freely. Also, you do not have to answer any question you don’t want to and can stop the interview at any time if you wish.

Your participation in this interview is very important because it will help us improve the questionnaire. We’re scheduled to be here for about 60 minutes, and I won’t keep you any longer. As a thank you for your help, we will give you $40 cash after the interview is completed.

Do you have any questions?

Write down questions

Section 2: Informed Consent

Before we start, I’m going to give you a written consent form. This document explains the purpose of the study, who can participate in the interviews, how we will protect your information, and who you can call if you have questions.

Hand the Informed Consent document to the participant. Read the document to participant or let participant read it (as appropriate). If participant chooses to read the document, tell him or her:

In addition to the points I have just mentioned about confidentiality and being observed I would like to draw your attention to the fact that:

• You are 1 of 20 people taking part in the interviews;

• Your participation is voluntary and you may stop the interview at any time

• You can refuse to answer any of the questions asked or take a break at any time

• Your responses in this study may be used only for statistical purposes and will not be disclosed, or used, in identifiable form for any other purpose, except as required by law. (20 U.S.C. § 9573)

When completed:

1. Ask participant if (s)he has any questions.

2. Ask participant if (s)he agrees to participate and agrees to be audio recorded.

3. If participant agrees, obtain the respondent’s signature to proceed – on two (2) copies.

4. Make sure respondent takes one copy with them when the interview is over.

Section 3: Cognitive Interviewing

If consent was given, start recorder

Okay, let’s get started. As you proceed though the interview I would like you to try and remember your thought process for the answer to each question. That is, be able to tell me how you went about answering the question, tell me if something didn’t sound right, was hard to answer, there were words you didn’t understand, etc. If any questions are unclear to you as you go through, please stop and let me know.

Do you have any questions before we begin?

If consent was given, start recorder

If necessary, remind the respondent to “think aloud”.

Ask unscripted probes for questions where the respondent has given some indication of difficulty with the questioning based on answers/behaviors.

Examples of general probes you may use:

• What did you think the question was asking?

• Can you rephrase the question in your own words?

• How did you go about answering?

• What were you thinking of?

• How did you decide on your answer?

• What was the reason for that answer?

• How easy or difficult was it to answer the question?

• PROBE for examples.

• How did you feel about answering the questions?

• Could you explain what you understand by this term?

• Can you tell me more about that?

• You seemed to hesitate/take a moment before giving/entering your response. Can you tell me what you were thinking about?

• What do you mean by…?

• Is there any additional information that we should have provided to help you answer that question?

• Does this term/question sound okay to you, or would you prefer something different?

Section 4: Debriefing Procedures

• Ask for any last observations/comments/concerns.

• Stop the digital audio recorder.

• Thank the participant for coming in and for helping test the interview.

• Give the $40 incentive and escort them from the interview room.