Employment and Earnings Survey 2nd Round of Cognitive Interviews

Attachment VI
Cognitive Interviewing Protocol

Section 1: Introduction

Welcome participant and thank them for their time.

Introduce self: [Name/RTI employee/RTI International is a non-profit research organization.]

Read or paraphrase: As we mentioned in our earlier conversations, we are working with the National Center for Education Statistics, within the U.S. Department of Education, to develop an Employment and Earnings Survey. The items in this survey will be used in an upcoming administration of the National Household Education Survey. Today, I’m hoping to get your honest feedback on the survey questions we are planning to use.

During the interview, I will ask you to read the survey questions and respond to them. There are no right or wrong answers. In this study, I am less interested in your actual answers to the questions and more interested in what you thought about the questions and how you understood them. After you have gone through the survey, I will ask some questions about your answers, such as what you understood the question to mean and whether the questions reflect your actual circumstances.

In order to concentrate on what you are saying rather than having to take notes, and to make sure that we don’t miss anything that you say, we would like to audio record this interview. Is that okay with you? [IF NOT OKAY WITH RECORDING: Okay, that’s no problem.]

One [a few] of my colleagues will be [observing our discussion from behind this mirror/on the telephone, listening to our session]. They want to learn from what you say about your experiences completing this survey. Is that okay with you? [IF NOT, ASK OBSERVERS TO LEAVE]

I want you to know that your participation is voluntary and your responses are protected from disclosure by federal statute. Responses that relate to or describe identifiable characteristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose, unless otherwise compelled by law. We want you to know that you can speak freely. Also, you do not have to answer any question you don’t want to and can stop the interview at any time if you wish.

Your participation in this interview is very important because it will help us improve the questionnaire. We’re scheduled to be here for about 40 minutes, and I won’t keep you any longer. As a thank you for your help, we will give you $40 cash after the interview is completed.

Do you have any questions?

Write down questions

Section 2: Informed Consent

Before we start, I’m going to give you a written consent form. This document explains the purpose of the study, who can participate in the interviews, how we will protect your information, and who you can call if you have questions.

Hand the Informed Consent document to the participant. Read the document to participant or let participant read it (as appropriate). If participant chooses to read the document, tell him or her:

In addition to the points I have just mentioned about confidentiality and being observed I would like to draw your attention to the fact that:

• You are 1 of 15 people taking part in the interviews;

• Your participation is voluntary and you may stop the interview at any time

• You can refuse to answer any of the questions asked or take a break at any time

• Your responses in this study may be used only for statistical purposes and will not be disclosed, or used, in identifiable form for any other purpose, except as required by law. (20 U.S.C. § 9573)

When completed:

1. Ask participant if (s)he has any questions

2. Ask participant if (s)he agrees to participate and agrees to be audio recorded and observed

3. If participant agrees, obtain the respondent’s signature to proceed – on two (2) copies

4. Make sure respondent takes one copy with them when the interview is over

Section 3: Cognitive Interviewing

If consent was given, start recorder

Okay, let’s get started. As you proceed though the interview I would like you to try and remember your thought process for the answer to each question. That is, be able to tell me how you went about answering the question, tell me if something didn’t sound right, was hard to answer, there were words you didn’t understand, etc. If any questions are unclear to you as you go through, please stop and let me know.

Do you have any questions before we begin?

If consent was given, start recorder

If necessary, remind the respondent to “think aloud”.

Ask unscripted probes at questions where the respondent has given some indication of difficulty with the questioning based on answers/behaviors.

Examples of general probes you may use:

• What did you think the question was asking?

• Can you rephrase the question in your own words?

• How did you go about answering?

• What were you thinking of?

• How did you decide on your answer?

• What was the reason for that answer?

• How easy or difficult was it to answer the question?

• PROBE for examples.

• How did you feel about answering the questions?

• Could you explain what you understand by this term?

• Can you tell me more about that?

• You seemed to hesitate/take a moment before giving/entering your response. Can you tell me what you were thinking about?

• What do you mean by…?

• Is there any additional information that we should have provided to help you answer that question?

• Does this term/question sound okay to you, or would you prefer something different?

Section 4: Debriefing

• Stop the digital audio recorder

• Thank the participant for coming in and for helping test the interview

• Give them the $40 cash incentive, and have them complete the receipt

• Escort participant out of the building and sign them out

• Notify the task leader that the interview was completed

• Save digital recording file. Put recording in ESN, ASAP (XXX\PATH\XXX). Test that recording was transferred correctly. Delete the recording from digital recorder

• Let the task leader and project director know if something strange happened during interview

• Enter comments within 48 hours. Be mindful of closing the comment log (Excel spreadsheet) as soon as you are done entering comments so that other observers can also enter their comments. Log is located here: (XXX\PATH\XXX)

• Submit expense report for incentive funds