Attachment H6 - Quality Control (Interviewer Training Manual)

Quality Control

6

6.1 Quality Control Procedures

We will use a variety of methods to ensure that every field interviewer on NFS Field Test is collecting high quality data and following the study protocol. Please view any feedback as a way to develop your skills and improve data quality, which is a goal of all team members. The NFS Field Test will utilize a blend of tried and true methods along with newer technologies to maximize our efforts to maintain cooperation with the NFS Field Test study participants and to provide you with specific feedback on your work.

Quality control (QC) begins with training and continues throughout the course of the project. Collecting and recording data accurately is a vital aspect of your field interviewer role. It is your responsibility to complete all of your work accurately and thoroughly for each respondent. This chapter describes the following quality control procedures:

• Training and certification;

• Observations;

• Global Positioning System (GPS) data review;

• Validation;

• Feedback from Westat home office; and

• Making comments.

6.2 Interviewer Training and Assessment

As part of training, you will be evaluated on important skills, such as interviewing skills, project knowledge, cooperation-gaining skills, administration, computer skills, and your ability to implement each of the NFS Field Test data collection activities. You must successfully complete all three components of the NFS Field Test training. These include the entire pre-classroom home study, including the Learning Management System (LMS) activities, if these have been assigned to you, the in-person classroom training, and the post-classroom home study training.

Your supervisor will monitor the completion of your home study activities. In the classroom, your trainer and co-trainer will evaluate your progress each day and provide feedback where needed. On conclusion of your post-classroom home study training your supervisor will evaluate your readiness to begin data collection and will certify you to begin data collection. Once she certifies you, you will receive your initial case load in the IMS.

Additionally, during the field period, you will meet with your supervisor weekly. This is your opportunity to clarify concepts, discuss strategies for the coming week, and bring up any concerns.

6.3 Interviewing Techniques

There are several steps you can take to ensure you are conducting high quality and accurate interviews. There are three main focus areas to be aware of when interviewing.

1. Read verbatim. Question wording is critical, and it is important for you to read each question text exactly as it is written. Always read the reference periods and categories when they are part of the question. Text in ALL CAPS is for your information only and should not be read out loud to the respondent. There are a few exceptions to this rule that will be highlighted during classroom training.

2. We only ask questions, we never answer them. The respondent is the only one who can answer a question. Never assume or guess what you think might be the best answer.

3. When you need to probe for a complete answer, the best probe is to reread the question or the part of the question the respondent did not hear or understand. Do not reword or simplify the question in your own words. Never lead when you probe. Your respondent must hear all answer categories—do not narrow down to what you assume are the possible answers. If the respondent does not understand a question, reread the question and then respond with “whatever it means to you.” Below are some exhibits with standard Westat probes that can be used in various circumstances. Please review and be familiar with them. We will give you hard copies of these probes at training.

Exhibit 6-1. List of Standard Probes

Exhibit 6-2. List of Clarifications and Comments Standard Probes

Exhibit 6-3. List of Positive Reinforcement Standard Probes

When working on a national study it is essential that all field interviewers follow the same procedures and protocols during the interview so that data can be compared and contrasted across the NFS Field Test sample and with other national and international studies.

Making even the slightest change to the question wording changes the meaning of the question and it may no longer mean what the questionnaire designer intended. A definition of an activity, reference period, or answer category is critical to quality data collection. For example, if you drop the reference period “In the last month” from your question, the respondent may answer incorrectly. Leaving out answer categories may result in those categories being selected less often or it may make it more difficult and confusing for the respondent to answer the question. We have consistently found that interviewers who take these “short cuts” cause confusion for the respondent and add extra time to the overallinterview time.

6.3.1 Other Reminders

Though showing excitement about the study and its objectives it critical to ensuring cooperation, when administering the study procedures you must remain neutral about any question or comment about the NFS Field Test study or any other topic. Your facial expression, your body language, or your voice can affect respondent answers. Do not indicate by your voice or gestures that you are agree or disagree, or are judging the respondent’s behavior.

Your neutrality should extend to conversations outside the interview itself. Do not volunteer anything from your personal life. Respond with neutrality, but remain courteous, towards stories or opinions on social topics. For example, limit your responses and use neutral statements such as “I see” “I can understand how you feel about that” or “That’s why this study is so important, to learn about everyone’s experience.” The respondent will appreciate a true professional in their home.

Support the science. Indicate to respondents that you are interested in getting accurate, useful, and objective information and that you do not have any biases, preference, or theories about the study results. Do not state your opinions.

6.4 Observation

If necessary, during the field period, you may be observed conducting one or more interviews at the discretion of your supervisor or the Westat home office. The purpose of observation is to assess all aspects of your performance in following the NFS Field Test data collection protocol, such as reading questions verbatim, collecting data efficiently in the household’s residence, and training the household in the food entering system. Other skill areas that will be observed are the ability to obtain cooperation from the PR (and household members if necessary), explain the study, answer questions, and maintain rapport.

The observer typically meets with you prior to the appointment. You will introduce the observer as a colleague who is coming along to observe the interview. Keep in mind that observers maintain a low profile so respondents are not greatly distracted by them. Once you have left the residence, the observer will discuss your performance and, if needed, areas for improvement.

6.5 Global Positioning System (GPS) Data Review

A requirement of your job as a field interviewer on NFS Field Test is to be prepared for each case you are assigned and to work efficiently in the field. Your supervisor will help you to organize your work to ensure the use of best practices while working in the field. As part of this responsibility, you must log in to your mFOS application at the beginning of your workday and track your work activities throughout your day in the MyDay portion. This system collects GPS data any time the MyDay application is running.

Westat uses GPS data, along with case information, to perform quality control checks and identify potential areas for gaining efficiency in field work.

6.6 Validation

Validation consists of reviewing the cases you have completed through the CAPI system, reviewing MyDay logs including GPS data, or re-contacting a randomly selected portion of your assigned cases by telephone. On the NFS Field Test, if GPS and other data review show any anomalies, we will validate interviews that have been completed.

6.7 Feedback From Westat Home Office

During the weekly conference, you and your supervisor will also discuss quality control issues. Your supervisor will discuss any problems you may be having completing interviews. In addition, your supervisor will discuss any problems in the cases you completed compared to those completed by others and feedback from review of MyDay data.

Be open and receptive to this feedback to improve the quality of the data that you are collecting and your cooperation rates. As a professional field interviewer, view the feedback as an opportunity to learn about your performance and improve your skills.

6.8 Using “Other (Specify)”

An important part of your job is to record clear, accurate responses during the interview. Some questions offer a response option of OTHER (SPECIFY) if a respondent’s answer doesn’t fit any of the provided response options. When you select a response option that includes the “OTHER (SPECIFY)”, you may see one or more followup questions requesting additional details about the respondent’s answer.

When entering a response, be sure it is complete and that it will be clear even if taken out of context.

6.9 Open Ended Text Questions

A special type of text entry similar to entering OTHER (SPECIFY) text includes open-ended questions that require you to enter text based on what the respondent says. You want to capture the response as close as verbatim to what the respondent says, but you will also need to be sure you get sufficient detail from the respondent to be sure the question has been answered sufficiently.

If you’re not clear exactly what the response means, probe to get a more detailed response. Use probes such as “Could you be more specific” to get addition specific information.

6.10 Respondent Training

You will be responsible for training respondents on how to use the NFS Field Test FoodLogger. To ensure this, make sure you follow all the procedures from your classroom training. Following all the steps as outlined will ensure that all respondents get the identical training.

Attachment H6 – Quality Control (Interviewer Training Manual)	6-0