ATT-K-NSFG-VerificationProcedures_revMay2021

NSFG OMB Attachment K OMB No. 0920-0314

In-person Interview Verification Procedures and Questionnaires

for Screener Survey and Main Survey

A random ten percent of NSFG surveys, both screener and main, completed by each interviewer are selected to be rechecked using a brief interview to verify the completeness and accuracy of the interviewer’s work. An Interview Verification Report will flag these cases and populate the questionnaire responses. This report will update daily in the Integrated Field Management System (IFMS). More details about numbers/percent of cases selected for verification, based on outcomes, are included below in “Flagging rules”.

The verification effort will be supplemented by other standard quality control activities, including data reviews, timing data reviews, and use of RTI’s Field Interviewer Tracker (FIT) to compare GPS coordinates captured on interviewer tablets during screening and interviewing to the locations of sampled addresses.

General Information

• Verifications will be conducted by the Field Supervisor and other project staff as needed. A Survey Specialist will manage the verification process with oversight by the Field Manger and Data Collection Task Leader. Verifications will be conducted promptly after the case has been finalized.

• A maximum of 15 calls will be made at different times of day and days of the weeks within two weeks of the survey being flagged for verification. If verification is not completed after 15 calls, the case will be coded “Unable to Verify.”

• If a case is coded “Unable to Verify”, the verifier will manually flag a different completed case by the same interviewer for verification.

• Verifiers will use a programmed script when calling households (see below). The script will identify the verifier and explain why they are calling.

• The Field Manager, Field Supervisor, and Survey Specialist will review the verification reports on at least a weekly basis.

• The verification instrument will check whether each question was verified as well as whether the survey as a whole was verified.

• All verification data will be stored on the enhanced security (NIST-moderate) project share.

Flagging Rules

The following flagging rules will be used for the screener survey and the main survey:

Screener Survey

• Screener survey verification will include three different situations:

1. AGE INELIGIBLE – NO ELIGIBLE RESPONDENT, SCREENER CONDUCTED. This includes households in which no member is age 15 – 49 as entered in the screener instrument. Ten percent of these cases will be flagged for verification. The RTI NSFG Management team will have the ability to manually flag cases for additional verification.

2. AGE INELIGIBLE – NO ELIGIBLE RESPONDENT, NO SCREENER CONDUCTED. This includes households in which no member is age 15 – 49 where no screener was conducted. One hundred percent of these cases will be flagged for verification.

3. LANGUAGE INELIGIBLE – LANGUAGE BARRIER. Ten percent of language barrier cases will be flagged for verification. The RTI NSFG Management team will have the ability to manually flag cases for additional verification.

Main Survey

• Main survey verification will include two different situations:

1. Each NSFG field interviewer will have her first two completed main surveys verified. The verification report will automatically flag the first two completed main surveys. The RTI NSFG Management team will have the ability to manually flag cases for additional verification.

2. Ten percent of all completed main surveys thereafter will be randomly and automatically flagged and verified throughout the course of production. Flagging occurs daily. The RTI NSFG Management team will have the ability to manually flag cases for additional verification.

Verification Procedure

1. The verifier will complete the following checks before contacting the respondent/informant.

   1. The verifier will review the record of calls for any information that could be useful in contacting the respondent/informant. Useful information could include good times to call (including day of week and time the original survey was conducted), respondent/informant concerns, and so forth.

      1. If the record of calls indicate that the respondent/informant was very resistant to doing the interview, the verifier will discuss the case with the Field Manager to determine whether the case is too resistant to be called for verification. If it is determined that the case is too resistant to be called for verification, the Field Manager or Survey Specialist will code the case “Unable to Verify” and add details as to why. This should be a very rare occurrence. An additional case will be selected to replace the “Unable to Verify” case.

2. The verifier will call the household and ask to speak to the informant or respondent listed. We will speak with the person listed (or one of the people listed) to conduct the verification interview.

3. The verifier will complete the verification questionnaire and record the answers in the programmed verification instrument. Contact attempts will be recorded in the verification instrument’s record of calls.

4. The verifier will compare the answers recorded for each question with the survey response or the expected answer. If there are discrepancies, the verifier will probe the respondent/informant to find out if the situation has changed since the survey in a way that would cause the answer to the question to be different. After probing to resolve any discrepancies, the verifier will document whether the verification response matches the survey response or expected outcome. Cases coded as language barrier will be coded as verified if the informant does not speak English or Spanish well enough to complete the screener verification.

5. The verifier will then assign a verification outcome code to the case based on the definitions given in this document. The verifier will enter the outcome code and include details necessary to explain the outcome code. Details will include the results of any probing that took place to resolve discrepancies. Details will be included for every case.

6. If the verifier is unable to reach the respondent/informant by phone after 15 attempts, the verifier will code the case as “Unable to Verify.” The Survey Specialist will then manually flag a different survey for verification from the same interviewer.

7. Verifiers will be in close communication with the Field Manager and Data Collection Task Leader regarding specific cases, as needed. In addition, the verifier will notify the Field Manager and Data Collection Task Leader by telephone or email immediately if they find a case that has failed verification. The verifier will explain the details of the failure using the documentation they recorded in the verification questionnaire. The Data Collection Task Leader will notify RTI project management immediately regarding the findings. In situations of failed verifications, additional cases will be selected for verification as appropriate.

8. The Survey Specialist will review on an ongoing basis any cases coded “Verified with Discrepancy”. They will investigate case notes and discuss with the verifier. As appropriate, the Survey Specialist will engage the Field Interviewer who originally completed the case. The Survey Specialist will then add comments about the case to the verification questionnaire. Based on this review and investigation, the Field Manager or Data Collection Task Leader may recode the case “Failed” or “Verified” or they may leave the case coded “Verified with Discrepancy.”

Verification of Screener Surveys: questionnaire

Screener verification is completed for a random ten percent of completed screener surveys in which no age eligible respondent was identified in the screener or in which there was a language barrier. All cases determined to be age ineligible without a screener survey being completed are flagged for verification.

Questionnaire:

1. Hello, my name is [NAME] with RTI International. May I please speak with [first name or initials of screener informant]? Our records show that one of our interviewers recently contacted you/your household regarding the National Survey of Family Growth. I'm calling to verify that our interviewer, [IWER NAME], spoke with you or someone at your household on [DATE OF INTERVIEW] for the National Survey of Family Growth. Do you remember [IWER NAME] contacting you?

2. According to our interviewer, your household includes no one between the ages of 15 and 49. Is this correct?

3. On (SCREENER RESULT DATE) how many people usually lived in your household?

4. In your opinion, did the interviewer conduct herself in a professional manner?

5. Are there any additional comments that you want to share about the interview or the interviewer?

Those are all the questions that I have. Thank you very much for your help with this study.

Verification of Main Survey: questionnaire

Main survey verification includes two different situations:

• Each NSFG interviewer will have her first two completed main surveys verified.

• Ten percent of all completed interviews thereafter are randomly and automatically flagged and verified throughout the course of production. Flagging occurs daily.

1. Questionnaire: Hello, my name is [NAME] with RTI International. May I please speak with [first name or initials of respondent]? Our records show that one of our interviewers recently contacted you/your household regarding the National Survey of Family Growth. I’m calling to verify that our interviewer, [IWER NAME], spoke with you or someone at your household on [DATE OF IW] for the National Survey of Family Growth. Do you remember [IWER NAME] contacting you?

2. [IWER NAME] would have asked you about childbearing, contraceptive use, sexual activity, marriage and divorce, and your attitudes about marriage, children, and family life. Do you remember a visit where someone asked you questions about such things?

3. How long did the interview take?

4. Was the interview conducted over the phone, in person, or online?

5. To verify our records, what is your date of birth?

6. For MINOR Rs: Did you sign a form on the tablet agreeing to participate in the survey?

7. Did you receive a monetary token of appreciation for participating in the interview?

For ADULT Rs: Did you sign a receipt on the tablet for the token of appreciation you received?

8. Did you enter some of your responses into the tablet yourself?

9. In your opinion, did the interviewer conduct herself in a professional manner?

10. Are there any additional comments that you want to share about the interview or the interviewer?

These are all the questions I have. Thank you very much for your help with this study.

Verification Codes and Definitions

Due to the shorter nature of the screener survey, the respondent/informant could indicate in the answer to just one question that the household composition is very different than what was collected in the screener survey. The verification outcome code definitions are therefore more stringent for the screener verifications than for the main survey verification.

Screener Survey Verification Code Definitions

FLAGGED	(1)	Flagged for verification.
VERIFIED	(2)	All verification data match expected data.
VERIFIED WITH DISCREPANCY	(3)	One piece of verification data (not including professional behavior if the respondent does not remember speaking to the interviewer) does not match expected data.
FAILS VERIFICATION	(4)	More than one piece of verification data (not including professional behavior if the respondent does not remember speaking to the FI) does not match expected data and the verifier’s overall judgment about the interview indicates a failure.
UNABLE TO VERIFY	(5)	Respondent or informant is contacted but is unwilling to participate in verification exercise OR the verifier is unable to reach the respondent/informant (including reaching the household by telephone, but not reaching the respondent/informant).
LIMIT MET	(6)	Case selected for verification, but verification not attempted due to interviewer’s 10% verification level being met.

Main Survey Verification Code Definitions

FLAGGED	(1)	Flagged for verification.
VERIFIED	(2)	All verification data match OR there is one piece of verification data that does not match.
VERIFIED WITH DISCREPANCY	(3)	Two pieces of verification data do not match.
FAILS VERIFICATION	(4)	More than two pieces of verification data do not match and the verifier's overall judgment about the interview indicates a failure.
UNABLE TO VERIFY	(5)	Respondent or informant is contacted but is unwilling to participate in verification exercise OR the verifier is unable to reach the respondent/informant (including reaching the household by telephone, but not reaching the respondent/informant).
LIMIT MET	(6)	Case selected for verification, but verification not attempted due to interviewer’s 10% verification level being met.

5