Results of Pretest

Attachment 3
Results of HCPS Pretest

Cognitive Testing of the HCPS
Cognitive interviews are one-on-one interviews used to assess the questionnaire in terms of general
understanding, question and response wording, skip logic, and visual aids. The goal is to gain an
understanding of how well the questions work when administered to a sample of the survey’s target
population. The interview follows a cognitive protocol with pre-scripted probes. Cognitive interviewers
also use spontaneous probes to gain a better understanding of questions.
RTI conducted a total of 69 cognitive interviews over three rounds of testing. The cognitive interview
process identified a number of issues that have been resolved in the current version of the questionnaire.
These revisions largely fall into the following categories:
•
•
•
•
•
•
•
•

Errors in skip logic were corrected;
Questions that adolescents were not able to answer were identified, and the questionnaire was
revised to skip over these for adolescents;
Some series of questions that seemed overly long were shortened;
Questions that seemed repetitive were eliminated;
For some terms that patients did not understand; explanations were added;
Show cards were added for questions that needed them;
Lengthy lists of response categories were shortened;
Transition statements were included to smooth transition between modules.

First Round Cognitive Interviews
Language

Respondent Profile

Focus of interviews

English

Interviews were conducted with a total
of 8 participants ranging in age from 29
to 56. Two of the respondents
completed the survey in proxy for their
children.

1

Interviews were conducted using a
paper instrument prior to CAPI
programming. Focus was to determine
issues with comprehension and skip
logic.

Second Round Cognitive Interviews
Language

Respondent Profile

Focus of interviews

English

Interviews were conducted with a total
of 12 English participants ranging in age
from 26 to 75. One of the respondents
completed the survey in proxy for her 1
year old child.

Interviews were conducted by CAPI.
Focus was to determine issues with
comprehension and to test the computer
application to ensure the questions were
routing properly.

Spanish

Interviews were conducted with a total
of 8 Spanish participants ranging in age
from 12 to 71. Two of the respondents
completed the survey in proxy for their
children

Interviews were conducted by CAPI.
Interviewer focused on comprehension
of Spanish translation.

Korean

Interviews were conducted with a total
of 6 Korean participants ranging in age
from 15 to 48. One of the respondents
completed the survey in proxy for her 8
year old child.

Interviews were conducted by CAPI.
Interviewer focused on comprehension
of Korean translation.

Vietnamese Interviews were conducted with a total
of 4 Vietnamese participants ranging in
age from 17 to 71. Two of the
respondents completed the survey in
proxy for their children
Chinese

Interviews were conducted with a total
of 6 Chinese participants ranging in age
from 29 to 67. One of the respondents
completed the survey in proxy for her 8
year old child.

2

Interviews were conducted by CAPI.
Interviewers focused on comprehension
of Vietnamese translation.

Interviews were conducted by CAPI.
Interviewers focused on comprehension
of Chinese translation. Three interviews
each were conducted in Cantonese and
Mandarin.

Third Round Cognitive Interviews
Language

Respondent Profile

English

Interviews were conducted with a total
of 8 English participants ranging in age
from 15 to 73. Two of the respondents
completed the survey in proxy for their
children

Interviews were conducted by CAPI.
Focus was to provide timing estimates
and look for questions that could be cut
back due to the length of the
questionnaires. Average length of
administration time was 7 minutes
longer than proposed.

Spanish

Interviews were conducted with a total
of 5 Spanish participants ranging in age
from 28 to 56. One of the respondents
completed the survey in proxy for her 3
year old child.

Interviews were conducted by CAPI.
Interviewer focused on providing timing
estimates.

Korean

Interviews were conducted with a total
of 3 Korean participants ranging in age
from 18 to 49.

Interviews were conducted by CAPI.
Interviewer focused on providing timing
estimates.

Vietnamese Interviews were conducted with a total
of 5 Vietnamese participants ranging in
age from 15 to 58.

Interviews were conducted by CAPI.
Interviewer focused on providing timing
estimates.

Chinese

Focus of interviews

Interviews were conducted with a total
of 4 Chinese participants ranging in age
from 16 to 59.

3

Interviews were conducted by CAPI.
Interviewer focused on providing timing
estimates.