14. 48-Month Survey Pretest Findings 6/2014 (Appendix G1)

APPENDIX G1:
48 MONTH SURVEY PRETEST MEMO
6/2014

This page left intentionally blank for double-sided copying.

111 East Wacker Drive, Suite 920
Chicago, IL 60601-4303
Telephone (312) 994-1002
Fax (312) 994-1003
www.mathematica-mpr.com

MEMORANDUM

TO:

Cynthia Miller, MDRC

FROM:

Lisbeth Goble, Lauren Maul, and Lisa Schwartz

DATE:

5/30/2014

- 184
SUBJECT:

YouthBuild 48-Month Youth Survey Pretest Findings

A. Introduction

In preparation for the 48-month survey for the YouthBuild Evaluation, Mathematica Policy
Research under subcontract to MDRC, conducted a pretest of three new series of questions for
inclusion in the 48-month survey. The first series aims to capture attributes generally associated
with one’s character. The second two series of questions measure respondents’ activities
surrounding civic engagement, leadership, and their sense of responsibility to others such as their
family or people in their community. The new questions that were the focus of this pretest are
included in Appendix A.
The pretest was conducted in two rounds utilizing a mixed-method approach. All pretest
interviews were conducted in person. The first round included a card-sort activity with debrief
for the personal attributes question and cognitive interviews for the activities questions. For the
second round, youth completed either interviewer-administered or self-administered
questionnaires and then participated in a cognitive interview using a retrospective protocol for
the entire series of questions. The pretest protocols are included in Appendices B and C.
There were four main objectives of the pretest: (1) to assess the consistency and accuracy
with which respondents interpreted abstract concepts such as personal attributes; (2) to identify
questions that seemed redundant with items already included on the survey; (3) to assess the best
placement for the new questions in the overall survey; and (4) to assess the additional burden the
new questions would place on respondents. This memo provides an overview of the pretest
design and presents findings and recommendations based on those findings.
B. Pretest Methodology

Pretests were conducted in-person with a total of 18 former YouthBuild participants from
YouthBuild Philadelphia and Great Falls YouthBuild in Patterson, NJ. Participants were given a
$25 gift card for their participation after completion of the interviews. Trained Mathematica staff
conducted all pretest interviews.
During the first round of interviews, we used a card-sorting activity to test the personal
attributes questions (Q1 in Appendix A). The card-sorting method is useful when trying to
discern categories or flesh out sets of questions that tap into a shared underlying concept or
construct (Righi et al, 2012). The primary goal of the card sort activity was to help determine the
An Affirmative Action/Equal Opportunity Employer

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
2
relative consistency with which people interpret and group the 21 words in this battery when
each item is looked at independently. In addition, we wanted to test respondents’ understanding
and interpretation of the question stem “Please indicate how the people you hang out with would
say that each of the following describes you…”. We used separate cognitive probes to assess
youth’s understanding of the phrase “people you hang out with” and how well those individuals
would be able to judge the respondent’s character. The round 1 protocol (Appendix B) provides
further details of the card sort activity.
In round 1, the activities questions were tested using cognitive interviews. Similar to the 12month pretest, we identified questions or terms that could be ambiguous or unclear. We then
created specific probes for these questions. The cognitive interview protocol is included in the
round 1 protocol (Appendix B).
Round 2 consisted of a mix of interviewer-administered interviews and self-administered
questionnaires followed by a retrospective debriefing. The combination of interviewer- and selfadministered modes allowed us to approximate CATI and web survey conditions, the ultimate
modes in which these surveys will be administered. To assess the additional time need to answer
the new questions, we embedded them into the questionnaire sections with two goals in mind: (1)
including them where we felt they fit best substantively and (2) minimizing the number of
sections we would have to change in order to implement the questions into the 48 month survey.
The latter is important to minimizing reprogramming costs for the 48-month survey. The
personal attributes questions were added to section G, which asks questions about Social and
Emotional Development. The activities questions were added to section H, which asks about
Identity Development.
C. FINDINGS AND RECOMMENDATIONS

Findings and recommendations based on the two rounds of pretests are documented below.
They are organized by each series of questions.
1. Personal attributes battery.
The personal attributes questions originated from a formative evaluation of YouthBuild
conducted by Ferguson and colleagues (Ferguson et al, 1996). At the end of one-on-one
interviews, respondents were asked “If someone asked you to describe yourself, to say who you
are, what would you say?” This was an open ended question and the responses were coded into
the groups that were ultimately used for this pretest. All respondents in that study were

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
3
YouthBuild participants. The battery of questions was adapted as presented in Appendix A.1
Below we present the findings on the question stem and the constructs themselves.
Question stem:
“Please indicate how the people you hang out with would say that each of the
following describes you. Would you say the following are “not at all like you,” “kind
of like you” “like you” or “very much like you?” 2
We asked respondents who they thought of when asked about “people you hang out with”.
We found respondents consider many groups of people, including friends or associates, cousins,
nieces and teachers, among others. To further understand how well these individuals would be
able to judge the respondents’ character, we asked them to rate, on a 1-5 scale [1 meaning not at
all, 5 being very well], how well that person would be able to judge their character. The average
rating was 4.1 with responses ranging between 2 and 5.
In general, respondents struggled with this question. As noted above, there was little if any
consistency across respondents in terms of who they considered “the people [they] hang out
with.” There were also inconsistencies within individual respondents. For example, one
respondent reported that he found himself thinking of different groups of people for different
items and noted that he would answer the question differently depending upon who he had in
mind as he responded. The lack of consistency both within and across respondents raises
concerns about the reliability and validity of the data we would capture from this series of
questions.
Recommendation: Eliminate the inconsistency and more closely approximate the original
question wording by changing the stem to read, “For each of the following, please indicate how
much you think the word describes you. Would you say “not at all like you,” “kind of like you,”
“like you,” or “very much like you.”
Constructs:
We had hoped the card sort activity would confirm that items presented together belonged to
a single generally recognized construct, but ultimately we found that word groupings were varied
1

We were unable to find documentation of the rationale for changing from self-reference – that is, how you
would describe yourself-- to how other people would describe you.
2

This is the version of the question that we were given for testing purposes. For Round 1, we did not include
the full question text in our testing efforts. For Round 2, we revised the question to read, “Would they say the
following…” to ensure subject-pronoun agreement.

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
4
considerably across respondents. Respondents sorted the 21 words or phrases into between four
and seven groups, with an average of five groups. For reference, there were nine groups in the
original question. The limited number of groupings produced by respondents suggests that they
may not understand all of the terms. The second round of pretesting was used to further assess
respondents’ understanding of the groupings of words in their original form.
Findings for each construct are summarized in table 1 below. In general, findings fell into
one of three categories: (1) not all the terms fit together, (2) respondents felt they could be
described by some of the terms in a series, but not all of them or (3) they did not understand all
the words.
Table 1. Personal Attributes Findings
Items

Card sort findings

General findings

a.

Helpful/caring/
loving/I can give advice

•

While the first three were
often grouped together, “I
can give advice” was not

b.

Intelligent/trying to
learn/hard-working

•

Intelligent was in various
groups

•

Being hard-working does not
necessarily mean you’re
intelligent and vice versa.

c.

Nice/friendly/easy to get
along with

•

Generally grouped together

•

Not all words describe
respondents – can be nice,
but not always easy to get
along with

d.

Honest/direct/sincere

•

Sincere was in multiple
groups
One felt that sincere didn’t fit
with direct or honest; another
felt that direct doesn’t fit with
honest or sincere

•

Some confusion over the
meaning of the words
sincere and direct

•

e.

Trying to make something of
myself/determined

•

Often grouped together as
well as with terms in group b

f.

Fun/carefree/easy-going/like
to party

•

Like to party was in various
groups

•

“Like to party” does not fit –
one can be fun and not like
to party

g.

Faith in God

•

Often grouped with “Downto-earth”

•

One respondent felt this was
limiting in terms of religious
diversity
Language needs to be
adapted to be consist with
other categories “Have faith
in God”

•

h.

Down-to-earth

i.

Social dexterity/
I can fit in

•

Many respondents felt that
this didn’t fit in a category or
included it with ‘down-toearth’

•

None of the pretest
respondents knew what
social dexterity meant

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
5
Recommendation: We have strong reservations about including this series of questions in
the 48-month survey in its current form. In addition to the issues raised in the table, this series
has a strong bias towards words or phrases that are common among YouthBuild programs. 3 As a
result, the treatment group is likely to respond differently to these questions than will those who
did not participate in YouthBuild and do not have the same context for the terms, potentially
leading to false impacts. Further this question only highlights positive attributes of the individual
and is not balanced with potentially negative attributes one might self-identify.
Before adding this series to the 48-month survey, we recommend restructuring the questions
based on our findings, including:
• changing the stem of the question
• simplifying the categories into single constructs that would be familiar to all
respondents
• balancing the attributes to reflect both positive and negative attributes
If we make these changes, further pretesting is needed to determine whether the changes
improve the clarity of this series of questions. It will be especially important to test with both
program participants and non-participants. Revising and testing the series have implications for
our timeline and budget. Further, the revised questions may no longer be comparable with those
that were administered as part of Ferguson et al.’s original study. While we recommend retaining
the original self-referent question stem, we note that it is unlikely that all of the individual items
will be retained.
2.

Activities questions

To pretest the ‘activities’ questions, first, we regrouped and reorganized them to be in
substantively similar groups. In general, we didn’t find major issues with these questions, but
have a number of recommendations for streamlining and integrating them into the 48-month
survey. We present question by question suggestions for changes in table 2 and general
recommendations below that. The final version of these questions, including our recommended
ordering of the questions, is included in Appendix D.

3

For example, when conducting the pretests, interviewers noted that several of the words in this series
appeared on bulletin boards at the YouthBuild sites.

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
6
Table 2. ‘Activities’ questions findings.
Question

Implemented Changes
after pretest round 1

Findings

Recommended Changes

2. In the past year how often have you done the following activities:
a.

Helped family
members

•

This question is
redundant with H10g,
“How many hours do
you spend taking care
of a family member?”

•

Suggest dropping this
question.

b.

Helped other
members of the
community

•

This question is
redundant with 3h
below.

•

Suggest dropping 3h.

c.

Discussed current
events and social
issues

•

Many respondents
thought of “current
events” in a narrow
context, such as
things that were going
on in their own lives or
neighborhoods (for
example, a house
burning down) and
“social issues” as
things that were going
on in their own
personal social lives
(for example, social
drama or prom).

•

Suggest expanding the
question to include a
definition or example.

d.

Served in a
leadership role in the
workplace or union

•

No recommendations
based on updated
questions text.
Respondents
understood the revised
version of the question.

•

Dropped “union”,
because respondents
didn't differentiate this
from the workplace

•

Changed response
categories to “yes/no”
because respondents
struggled with the
frequency scale

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
7
Question
e.

Served in a
leadership role in the
neighborhood

Implemented Changes
after pretest round 1
•

Changed response
categories to “yes/no”
because respondents
struggled with the
frequency scale

f.

Volunteered as a
youth mentor or
sports coach

g.

Worked in a
community based
program

h.

Participated in
political rallies,
campaigns or
demonstrations

•

Dropped this because
it is redundant with a
question in existing
question H5b [Since
[REF DATE[ have
you gotten involved in
a national, state or
local effort or
electoral campaign?].

i.

Served in a
leadership role in
your child’s school

•

Changed response
categories to “yes/no”

•

Added skip logic for
those without children

Served in a
leadership role in
church

•

Changed response
categories to “yes/no”

j.

Findings

Recommended Changes

•

Respondents primarily
indicated informal
leadership roles, such
as block captains.

•

We don’t suggest any
changes, but think this
is important to
understand analytically

•

Several respondents
indicated that they
had school leadership
roles (such as
secretary/ recorder for
YB student
government), which is
not captured in the
current series.

•

Suggest adding school
leadership

•

No recommendations,
understood as is.

•

Suggest dropping this
question.

•

This question should
be moved to Section A,
which asks other
questions about
respondents’ children.

•

Suggest adding ‘or
another religious or
spiritual group”

•

•

This question is
redundant. If the
question is intended
to measure “work” we
should capture it in
the employment
section; if we want
volunteering captured,
that is captured
elsewhere in section
H.

Respondents
indicated that ‘church’
is narrow when it
comes to religion.

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
8
Question
k.

Ran for or was
appointed to public
office

Implemented Changes
after pretest round 1
•

Findings

Recommended Changes

Dropped because it
was not relevant to
respondents in the
timeframe in which
this question will be
asked (48 months
after they started the
program) and they
did not take it
seriously,

3. In the past year, have you done any of the following activities:
a.

Advocated for my
child at her/his school

•

•

Added skip logic for
those without children

•

In general,
respondents did not
understand the
meaning of the word
“advocated.”

•

Suggest rewording this
question.

•

This question should
be moved to Section A,
which asks other
questions about
respondents’ children.

•

This question should
be moved to Section A,
which asks other
questions about
respondents’ children.

b.

Volunteered at my
child’s school

c.

Given money to other
family members who
needed it

•

No recommendations,
understood as is.

d.

Taken steps to
improve my family’s
housing situation

•

No recommendations,
understood as is.

e.

Influenced my friends
to turn their lives
around for the better

•

No recommendations,
understood as is.

f.

Cleared most of my
debts

g.

Paid off parking
tickets so I could get

•

•

Added skip logic for
those without children

We added an “N/A”
response option
because many
respondents
indicated that they did
not have debt.

We added an “N/A”
response option

•

One respondent did
not understand the
meaning of “debt.”

•

Suggest adding a
definition or example
language

•

Respondents reported
that the question
made them feel
uncomfortable

•

Inclusion of an N/A
option should alleviate
any discomfort with
this question.

•

Suggest moving this
question to Section I
(Health and Wellbeing), which asks
questions about risky
behaviors .

•

Inclusion of an N/A
option should alleviate

•

Respondents reported
that the question

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
9
Question

Implemented Changes
after pretest round 1

my driver's license

Findings

because many
respondents
indicated that they did
not have parking
tickets.
•

Recommended Changes

made them feel
uncomfortable,

any discomfort with
this question.
•

Suggest moving this
question to Section I
(Health and Wellbeing).

•

Suggest dropping this
question.

Dropped “…so I could
get my driver’s
license” because
respondents were
confused by the
conditional structure
of the question.
•

h.

Assisted a neighbor or
community member
who needed help

i.

Attended community
meetings to help
improve the
conditions within my
community

•

No recommendations,
understood as is.

j.

Served as a positive
role model for the
younger kids in my
neighborhood

•

No recommendations,
understood as is.

Redundant with 2b
above.

General Recommendation:
In addition to the question specific suggestions, we recommend the following updates to this
series of questions.
• Reference period: There are several issues associated with identifying the
appropriate reference period for these questions. Questions about helping others may
happen frequently but have fairly low salience. Under these circumstances, a shorter
reference period – for example, the past month – may yield more accurate
information. In contrast, serving in leadership roles are likely to be low
frequency/high salience events. To avoid missing important leadership experiences, it
may be important to allow for a longer reference period such as in the past year. Best
practices in survey methods suggest that limiting the number of reference periods in a
survey and particularly within a section of a survey facilitates more accurate recall.
When this is not possible, as may be the case with these questions, sequencing of
items is important so that respondents move from one reference period to another in
logical way. We recommend changing the sequence to allow for ease of recall and
limiting the number of reference periods in this section of the survey.

MEMO TO: Cynthia Miller, MDRC
FROM:
Lisbeth Goble, Lauren Maul, and Lisa Schwartz
DATE:
5/30/2014
PAGE:
10
• Response options: Initially these questions used a response scale that included vague
quantifiers like “Always/Often/Sometimes/Never.” For most of the activities, this
scale did not make sense and was confusing to respondents. One example where this
confusion was particularly pronounced was when we asked about the frequency with
which youth serve in certain leadership roles. For most questions, we suggest
changing to a more discrete scale: “Not at all/A few times a month/A few times a
week/Every day”. There are several questions, noted in table 2, where we suggest
changing to a “yes or no” structure.
• Substantive reorganization: We suggest grouping questions together that are
substantively related, for example, grouping all the leadership questions together. In
addition, we suggest moving a few questions to sections other than Section H. These
are noted in the table.
While these questions did not pose major issues during pretesting, we still have some
reservations about including them for logistical reason. If we implement the changes we
recommend, we will be changing more than the two sections we thought would be impacted.
This will require more programmer and instrument testing time than we anticipated.
3. Timing and Respondent Burden

We administered sections G and H in their entirety in order to compare the timing of those
sections to their timing on the 12 month survey. On the 12 month survey, we found that section
G and H took just over 7 minutes when administered by phone. On the pretest, we found that the
interviewer-administered survey took about 13 minutes, suggesting that the inclusion of these
questions will add about six minutes to the survey administration time.
The 12 month survey was, on average, over 45 minutes in length (5 minutes longer than we
anticipated), which is less than ideal. The inclusion of these new questions will increase the
chances that we will experience break-offs and non-response. Further, these questions are
included in sections that respondents found particularly challenging during the 12 month pretest.
We have reservations about adding more to these already challenging sections.

cc:

APPENDIX A
48-MONTH PRETEST QUESTIONS

06796.P23

MATHEMATICA POLICY RESEARCH

1. Please indicate how the people you hang out with would say that each of the following
describes you. Would you say the following are “not at all like you,” “kind of like you”
“like you” or “very much like you?”
Not at all
like me
a.
b.
c.
d.
e.
f.
g.
h.
i.

Kind of
like me

Like
Me

Very much
like me

Helpful/caring/loving/I can
give advice
Intelligent/trying to learn/hardworking
Nice/friendly/easy to get
along with
Honest/direct/sincere
Trying to make something of
myself/determined
Fun/carefree/easy-going/like
to party
Faith in God
Down-to-earth
Social dexterity/I can fit in

2. In the past year how often have you done the following activities: (always, often,
sometimes, never)
Not at all
like me
a.
b.
c.
d.
e.
f.
g.
h.
i.
j.
k.

Kind of
like me

Helped family members
Helped other members of the
community
Discussed current events and
social issues
Served in a leadership role in
the workplace or union
Served in a leadership role in
the neighborhood
Volunteered as a youth
mentor or sports coach
Worked in a community
based program
Participated in political rallies,
campaigns or demonstrations
Served in a leadership role in
your child’s school
Served in a leadership role in
church
Ran for or was appointed to
public office

2

Like
Me

Very much
like me

06796.P23

MATHEMATICA POLICY RESEARCH

3. In the past year, have you done any of the following activities:
Not at all
like me
a.
b.
c.
d.
e.

f.
g.
h.

i.

j.

Kind of
like me

Advocated for my child at
her/his school
Volunteered at my child’s
school
Given money to other family
members who needed it
Taken steps to improve my
family’s housing situation
Influenced my friends to turn
their lives around for the
better
Cleared most of my debts
Paid off parking tickets so I
could get my driver's license
Assisted a neighbor or
community member who
needed help.
Attended community
meetings to help improve the
conditions within my
community
Served as a positive role
model for the younger kids in
my neighborhood

3

Like
Me

Very much
like me

APPENDIX B
YOUTHBUILD 48-MONTH FOLLOW UP SURVEY:
PRETEST QUESTIONS

Mathematica Reference No: 06796

YouthBuild 48-Month Follow Up Survey:
Pretest Questions
Round 1: Pretest Protocol
April 2014

TABLE OF CONTENTS

MATHEMATICA POLICY RESEARCH

CONTENTS
Section

Page

IN-PERSON INTERVIEW SCRIPT .................................................................................................. 1
PART I: CARD SORT ACTIVITY ..................................................................................................... 2
PART II: OBSERVATIONAL DEBRIEF ........................................................................................... 3

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

INTERVIEWER:

MATHEMATICA POLICY RESEARCH

READ CONSENT SCRIPT AT NORMAL INTERVIEWING PACE,
THEN START FIRST ACTIVITY.

In Person Interview Script:
Hi, my name is [name] and this is my coworker [name] from Mathematica Policy Research, a social
research company based out of Princeton New Jersey. Thanks so much for being willing to help us out
today.
As we go through the activities, you may notice that at some points I’m reading word for word from
my notes. We just want to make sure that we are giving each person the same information. There will be
plenty of time for us to talk through any questions you might have after we finish the activity.
As someone may have explained to you, we are currently testing a series of questions that will
eventually be part of a larger survey and we want your help to figure out how to make the questions and
answer categories clear and easy to understand. To do this we are going to ask you to do two different
types of activities. You will be doing the first activity with me and and then you will do a second activity
with [insert name of partner]. All together the activities will take about 40 minutes of your time.
Please keep in mind that there are no right or wrong answers to any of the activities or questions we
ask you. We really just want to find out as much as possible about what you are thinking, so don’t hesitate
to speak up whenever something is unclear, hard to answer, or doesn’t seem to apply you.
I will be recording the interview so that I can go back and listen to it later when I’m summarizing the
results. While we are doing our activity, my partner will be observing us and timing our activities and
taking notes, so that I can just focus on talking with you.
Everything that you tell us will be kept private. You don’t have to answer any questions that make
you feel uncomfortable.
Do you have any questions before we start?
Let’s begin.

Please hit the record button on your digital voice recorder

Prepared by Mathematica Policy Research

1

4/18/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

MATHEMATICA POLICY RESEARCH

PART I – CARD SORT ACTIVITY
A. Materials:

•
•
•
•
•

Set of index cards with the words and phrases on them (each set should have 23 cards)
Round 1: Pretest Protocol
Paper clips (at least 10)
Pen or pencils
Tape recorder (and batteries)

While the lead interviewer introduces the activity, the partner should start filling this out:
Card Sort Results
PRETEST RESPONDENT NUMBER (R1 – R9):
LEAD INTERVIEWER:
SECONDARY INTERVIEWER:
CARD SORT START TIME:

CARD SORT END TIME:

DEBRIEF START TIME:

DEBRIEF END TIME:

GENERAL NOTES ON THE CARD SORT TASK:
Word and Phrases

Specific Notes

Helpful
Down-to-earth
Trying to learn
Direct
Loving
Faith in God
Honest
Fun
Easy to get along with
Social dexterity
Caring
Nice
Easygoing
Trying to make something of myself
Like to party
Hard-working
Intelligent
Friendly
Determined
I can fit in.
Carefree
I can give advice.
Sincere

Prepared by Mathematica Policy Research

2

4/18/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

MATHEMATICA POLICY RESEARCH

Step 1: Give respondent instructions:
Here’s how it works. In front of you is a stack of cards. Those cards represent words that
might be used to describe you. I’d like you take the cards and try to sort them into groups that
make sense to you.
If you don’t understand the meaning of the word or phrase on the card, you can put them in
a “Don’t know” pile. If you think something doesn’t belong or “fit” with any other words, you
can make an “other” pile. At the end of this activity, I will ask you some questions about why
you grouped certain cards together and what some of the words and phrases mean to you.

Step 2: Card sort debrief with respondent
While the respondent is sorting their cards, you or your partner should start filling out the
Card Sort Results table on the next page making notes about the following things:
•

Respondent ID. Enter an ID between R1 and R9, where the first pretest respondent in
round one is R1 and the last pretest respondent in round one is R9.
• Lead/secondary interviewer names. Write down the name of the person leading the
card sort activity and the name of the person observing the card sort activity and taking
notes.
• Start and stop times for the card sort. How long does it take the respondent to
complete the card sort task? Extremely long or short card sort times can be a sign that the
respondent had difficulty understanding the task, the meaning of words on the individual
cards, or was not taking the task seriously.
• Observation notes. As the respondent is conducting the sorting activity, please watch to
see if there are non-verbal clues that any words or phrases are confusing. Make notes in
the “General Notes” box in the Card Sort Results table.
Once the respondent has finished sorting the cards into piles you are going to want to ask
them about the groups of cards they made. Some questions to ask respondents are listed below:
•

I see that you have [X] groups, can you please tell me a little bit about why you grouped
each of these together?

•

If you had to name the group, what would you call it?

•

Can you tell me any other words you would include in that group (words that aren’t listed
on the cards).

•

Can you please tell me a little bit about the cards you didn’t put into groups?

Step 3: Document the number of groups, the words in each group
In order to analyze patterns across respondents, we will want to rubber band the
respondent’s cards into the groups they formed and then put those groups of cards into an
envelope and write the respondent ID on the envelope (R1 – R9).

3

06/04/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

MATHEMATICA POLICY RESEARCH

We will later data enter these so that we can analyze the card sort data across respondents to
identify patterns and underlying constructs.
Step 4: Follow-up on the stem to this question
Interviewer Note: The original question stem to this question was “Please indicate how the
people you hand out with would say that each of the following describes you. Would you say the
following are “not at all lot you”, “kind of like you”, “like you”, “very much like you”.
I just have a few more questions for you.
1. If I said the phrase “people you hang out with”, who would that include?
2. [Interviewer - For each person they ‘hang out with] – On a scale from 1 to 5 with 1 being not at
all and 5 being really well, how well do you think that person could answer questions about your
character?

4

06/04/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

MATHEMATICA POLICY RESEARCH

PART II – OBSERVATIONAL DEBRIEF
Materials:

•

Round 1: Pretest Protocol

•

Pen or pencil

•

Tape recorder (batteries)

Step 1: Explain the purpose of the activity
Next, I want to ask you a series of questions that will be part of a larger survey later.
Step 2: Start filling out the observational debrief notes table
Observational Debrief Notes
PRETEST RESPONDENT NUMBER (R1 – R9):
LEAD INTERVIEWER:
SECONDARY INTERVIEWER:
INTERVIEW START TIME:

INTERVIEW END TIME:

DEBRIEF START TIME:

DEBRIEF END TIME:

GENERAL NOTES ON THE OBSERVATIONAL DEBRIEF TASK:

Step 3: Administer the survey questions and verbal probes (see next page)

5

06/04/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

MATHEMATICA POLICY RESEARCH

Before we start, can you tell me if you have any chidren? There are a few questions that are specific
to people with children, but I will skip those if they don’t apply to you
INTERVIEWER NOTE: Note if they have a child. Certain questions will contain the check for if
child=yes
These first questions are about activities you may have done in the past 12 months.
PT1.

In the past year, how often have you done the following activities?
Would you say always, often, sometimes, or never?
Always

Often

Sometimes

Never

a. Helped family members

1

2

3

4

b. Helped other members of the
community

1

2

3

4

INTERVIEWER PROBES:
Can you tell me what types of helping activities you were thinking of when
you answered this question?
[For each, if not ‘never’] How many times did you help?

PT2.

In the past year, how often have you done the following activities?
Would you say always, often, sometimes, or never?
Always

Often

Sometimes

Never

a. Discussed current events and social
issues

1

2

3

4

b. Worked in a community-based
program

1

2

3

4

c. Participated in political rallies,
campaigns, or demonstrations

1

2

3

4

d. Ran for or was appointed to public
office

1

2

3

4

INTERVIEWER PROBES:
What does the phrase ‘current events or social issues’ mean to you? Can you
give me an example?
What does the phrase ‘community-based program’ mean to you? Can you
give me an example?
[IF PT2c<4] Can you describe the political rallies, campaigns or
demonstrations you participated in (or if ‘never’ what those are)?
What does it mean to run for or be appointed to public office?
[For each, if not ‘never’] How many times did you help?

6

06/04/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

PT3.

MATHEMATICA POLICY RESEARCH

In the past year, how often have you done the following activities?
Would you say always, often, sometimes, or never?
Always

Often

Sometimes

Never

a. Served in a leadership role in the
workplace or union

1

2

3

4

b. Served in a leadership role in the
neighborhood

1

2

3

4

c. Volunteered as a youth mentor or
sports coach

1

2

3

4

d. [If child = yes] Served in a leadership
role in your child’s school

1

2

3

4

e. Served in a leadership role in church

1

2

3

4

INTERVIEWER PROBES:
What does the phrase ‘leadership role’ mean to you? Can you give me an
example?
What does the phrase ‘workplace or union’ mean to you? Can you give me
an example of each?
What did you think of when I asked about your neighborhood? What did you
include?
[If have extra time ask for those activities not ‘never’] How many times
did you do these things?

7

06/04/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

PT4.

MATHEMATICA POLICY RESEARCH

In the past year, have you done any of the following?
YES

NO

a. [If child = yes] Advocated for your child at his/her school

1

0

b. [If child = yes] Volunteered at your child’s school

1

0

c. Given money to other family members who needed it

1

0

d. Taken steps to improve your family’s housing situation

1

0

e. Assisted a neighbor or community member who needed help

1

0

1

0

f.

Attended community meetings to help improve the conditions
within your community

INTERVIEWER PROBES:

What does the phrase ‘advocated’ mean to you? Can you give me an
example?
[If child= yes] Can you describe how you might volunteer at your
child’s school?
Which ‘family members’ did you include when I asked you if you had
given other family members money?
What does the phrase ‘improve your family’s housing situation’ mean
to you? Can you give me an example?
Can you describe how you assisted a neighbor or community member?
Who did you include when thinking about your ‘neighbors or
community members’?
What does the phrase ‘improve the conditions within your community’
mean to you? Can you give me an example?

PT5.

In the past year, have you done any of the following?
YES

NO

a. Cleared most of your debts

1

0

b. Paid off parking tickets so you could get your driver’s license

1

0

INTERVIEWER PROBES:
What does the phrase ‘cleared most of your debts’ mean to you? Can you
give me an example?
Do you have a license?
Do you have parking tickets?

8

06/04/14

YOUTHBUILD 48-MONTH FOLLOW UP SURVEY: PRETEST QUESTIONS

PT6.

MATHEMATICA POLICY RESEARCH

In the past year, have you done any of the following?
YES

NO

a. Influenced your friends to turn their lives around for the better

1

0

b. Served as a positive role model for the younger kids in your
neighborhood

1

0

INTERVIEWER PROBES:
What does the phrase ‘turn your life around for the better’ mean to you? Can
you give me an example?
Can you describe how you might influence your friends?
What does the phrase ‘positive role model’ mean to you? Can you give me
an example?

FINAL INTERVIEWER PROBES:
-

Were there any questions in this interview that you did not know how to answer? Tell me more about
that.
Were there any parts of the interview that you found confusing? Tell me more about that.

-

Were there any questions in this interview that made you feel uncomfortable? Which questions?
This was the last question I had. Did you have any other comments about the questionnaire?
Thank you for helping us test the questionnaire.

Persons are not required to respond to this collection of information unless this survey displays a currently valid OMB control
number (OMB xxxx-xxxx, expires xx/xx/xxxx). Responding to this questionnaire, which seeks to help the U.S. Department of
Labor understand how YouthBuild programs are serving disadvantaged youth, is voluntary. Public reporting burden for this
collection of information is estimated to average 30 minutes per response, including the time for reviewing instructions,
searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of
information. Send comments regarding this burden estimate to Eileen Pederson, U.S. Department of Labor, Employment and
Training Administration, Office of Policy Development and Evaluation, Room N-5641, 200 Constitution Avenue, NW,
Washington, DC 20210.

9

06/04/14

APPENDIX C
YOUTHBUILD RESPONDENT DEBRIEFING PROTOCOL: ROUND 2

YouthBuild Respondent Debriefing Protocol: Round 2
Materials:
•
•

Debriefing protocols
Questionnaires
o 5 for SAQ
o 5 to provide in-person interview respondents with a blank hard copy during
debriefing
o One for interviewer

•

Digital recorders

•

Pens and pencils for respondents

•

Gift cards for respondents

Administer Survey:
Thank you for your time and willingness to help us out.
First, let me introduce myself and tell you a little bit about what we are going to be doing here
today.
•

My name is [name] and I’m from Mathematica Policy Research, Inc., a research
company in Princeton New Jersey.

•

You may notice that at some points I’m reading word for word from my notes.
We just want to make sure that we are giving each person the same
information. There will be plenty of time for us to talk through any questions
you might have after we finish the activity.

•

Today, we are asking for your help to test a few sections of a questionnaire that
will eventually be given to a larger group of youth. Your input is very valuable,
as it will help us identify questions that are not clear and address any concerns
that you or other youth might have about the survey. Your feedback will be
incorporated into the final version of the questionnaire. Ultimately, your input
will help ensure that we collect the highest quality data.

•

[For in-person interviews]: First, I am going to ask you several survey questions,
and then I would like to get your feedback about those questions. This should
take about 30 minutes. Do you have any questions before you begin?
I would like to record our discussion so that I can go back and listen to it later if I
need to. Is that ok?
1

•

[For PAPI]: Please complete these questions, and then I would like to get your
feedback on them. This should take about 30 minutes. Do you have any
questions before you begin?

•

INTEVIEWER NOTES
o Note the start and stop times on your questionnaire.
o For PAPI: Please note if respondent appears to be having any noticeable
difficulties while completing the survey. When interview has ended or
respondent has completed the survey, enter stop time.

2

YouthBuild FOLLOWUP Phone Pretest Timing Sheet
APPOINTMENT DATE:
TIME:
INTERVIEW START TIME:
INTERVIEW END TIME:
INTERVIEW MODE:

__________SAQ

__________ “Phone”

Comments
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
_____________________________________________________________________________

3

Debriefing Protocol:
Thank you for answering these questions. As I said before, I want to ask you some questions
to get your feedback about the survey.
[For in-person respondents]: Here is a copy of the questions so that you can refer to any
specific questions I ask you about]. If you have any other comments as we go through,
please feel free to share.
[FOR PAPI respondents]: I would like to record our discussion so that I can go back and
listen to it later if I need to. Is that ok?
INTERVIEWER: For each page/section you can start with the following stem. “On [page],
[section], were there any questions that you found confusing or did not understand?”
•

Were there any terms or words that you did not understand?

•

Were there any questions you found especially difficult to answer?

INTERVIEWER: Go though section by section to ask the specific probes below.
PAGE

1

SECTION

QUESTIONS

G

How you feel
about
yourself/people
in your life

NOTES

How others
describe you

Specific probes: Please look at Question G2.
•
Potential
Areas of
Confusion
for Page 1,
Section G

In part [list grouped item, starting with a],
o Is there one word or phrase you can think of that would
describe all of the words in this group?
o Were there any words in this group that you did not know the
meaning of?
o Do these words fit together or do some of the words not
belong
o [Ifanswer>1]: You mentioned that these words describe you.Do
all of these words describe you, or only certain ones?
4

a. Helpful/caring/loving/I can give advice ..................

b. Intelligent/trying to learn/hard-working ...................

c. Nice/friendly/easy to get along with .......................

d. Honest/direct/sincere .............................................

e. Trying to make something of myself/ determined ..

f. Fun/carefree/easy-going/like to party ....................

g. Have faith in God ...................................................

h. Down-to-earth ........................................................

i. Social dexterity/I can fit in ......................................

5

On pages 2:
Were there any questions that you found confusing or did not understand?
Were there any terms or words that you did not understand?
Were there any questions you found especially difficult to answer?

2

G

People in
general

On pages 3:
Were there any questions that you found confusing or did not understand?
Were there any terms or words that you did not understand?
Were there any questions you found especially difficult to answer?

3

H

How you make
decisions and
solve problems

3

H

Volunteer or
community
service work

6

On pages 4:
Were there any questions that you found confusing or did not understand?
Were there any terms or words that you did not understand?
Were there any questions you found especially difficult to answer?

4

Volunteer or
community
service work

H

Specific Probes: Please look at H5/H6.
Potential
Areas of
Confusion
for Page 4,
Section H,
Volunteer
or
Community
Service
Work

•

Are there any terms or words that you do not understand?

•

Are there any questions you found especially difficult to answer?

•

[INTERVIEWER: If they say yes to ‘neighborhood’ and any other leadership
role clarify] Can you tell me more about that role? Is it different than the
other leadership roles you mentioned?

•

Have you served in any other leadership roles not listed here?

7

On pages 5:
Were there any questions that you found confusing or did not understand?
Were there any terms or words that you did not understand?
Were there any questions you found especially difficult to answer?

5

H

Help family or
community

8

Specific Probes: Please look at H7.
• [INTERVIEWER: Look at the response options to try to tease out any
reducancies, specifically how they answered H7b and d] Can you give
me some examples of what you were thinking about when you
answered these questions? Were you thinking about the same type of
help or was it different?

• [INTERVIEWER: if answered “a few times a month,” “a few times a
week”, or “every day” to item c]: Can you give me some examples of
what you were thinking about when you answered this question?
Potential
Areas of
Confusion
for Page 5,
Section H,
Help family
or
community

•

Please look at c: What does the phrase ‘community-based program’
mean to you? Can you give me an example?

•

[INTERVIEWER : if answered “a few times a month,” “ a few times a
week”, or “every day” to item b]: Can you give me some examples of
what you thinking about.

•

Please look at i: Can you tell me what the phrase “current events”
means to you? Can you tell me what the phrase “social issues” means
to you?

9

Specific Probes [if respondent has child]: Please look at H8.

Potential
Areas of
Confusion
for Page 5,
Section H,
Children

•

What does the phrase ‘advocated’ mean to you? Can you give me an
example?

•

[INTERVIEWER: If answered greater than “not at all”]: Can you give
me an example of the ways you advocated for your child?

On pages 6:
Were there any questions that you found confusing or did not understand?
Were there any terms or words that you did not understand?
Were there any questions you found especially difficult to answer?

6

H

Things you do
in a typical day

General questions:
•

Were there any parts of the interview that you found confusing or repetitive?
Tell me more about that.

•

Were there any questions in this interview that made you feel uncomfortable?
Which questions?

•

Do you have any other comments about these questions that you would like to
share?

WRAP-UP (5 minutes)
We are coming to the end of our discussion. Thank you for sharing your experiences and
opinions. Is there anything else that you would like to add about anything we discussed today?
Thanks again for your participation. [Give gift card].
10

OMB No.: 1205-0503
Expiration Date: 12/31/2015

YOUTHBUILD
48-Month Pretest
April 30, 2014

Persons are not required to respond to this collection of information unless this survey displays a currently valid OMB control number (OMB 12050503, expires 12/31/2015). Responding to this questionnaire, which seeks to help the U.S. Department of Labor understand how YouthBuild
programs are serving disadvantaged youth, is voluntary. Public reporting burden for this collection of information is estimated to average XX
minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed,
and completing and reviewing the collection of information. Send comments regarding this burden estimate to Eileen Pederson, U.S. Department of
Labor, Employment and Training Administration, Office of Policy Development and Evaluation, Room N-5641, 200 Constitution Avenue, NW,
Washington, DC 20210.

G. Social and Emotional Development

G1.

These first questions ask about how you feel about yourself and the people in your life. Below is a list
of statements dealing with your general feelings about yourself. Please indicate if you strongly agree,
agree, disagree, or strongly disagree.
SELECT ONE RESPONSE PER ROW

STRONGLY
AGREE
a. I feel that I’m a person of worth, at least on an
equal plane with others ......................................

1

b. I feel that I have a number of good qualities ......

1

c. All in all, I am inclined to feel that I am a
failure .................................................................

1

d. I am able to do things as well as most other
people ................................................................

1

e. I feel I do not have much to be proud of ............

1

f. I take a positive attitude towards myself ............

1

g. On the whole, I am satisfied with myself ...........

1

h. I wish I could have more respect for myself ......

1

i. I certainly feel useless at times ..........................

1

j. At times I think I am no good at all ....................

1

G2.

□
□

AGREE
2
2

□

2

□
□
□
□
□
□
□

2
2
2
2
2
2
2

□
□
□
□
□
□
□
□
□
□

DISAGREE
3
3

3

3
3
3
3
3
3
3

STRONGLY
DISAGREE

□
□

4
4

□

4

□
□
□
□
□
□
□

4
4
4
4
4
4
4

□
□
□
□
□
□
□
□
□
□

Now we want to know how other people see you. Please indicate how the people you hang out with
would say that each of the following describes you.
Would they say the following are “not at all like you,” “kind of like you,” “like you,” or “very much like
you?”
SELECT ONE RESPONSE PER ROW

NOT AT
ALL LIKE
YOU
a. Helpful/caring/loving/I can give advice ...................

1

b. Intelligent/trying to learn/hard-working ....................

1

c. Nice/friendly/easy to get along with ........................

1

d. Honest/direct/sincere ..............................................

1

e. Trying to make something of myself/ determined ...

1

f. Fun/carefree/easy-going/like to party .....................

1

g. Have faith in God ....................................................

1

h. Down-to-earth .........................................................

1

i. Social dexterity/I can fit in .......................................

Prepared by Mathematica Policy Research

1

1

□
□
□
□
□
□
□
□
□

KIND OF
LIKE YOU
2

2

2

2

2

2

2

2

2

□
□
□
□
□
□
□
□
□

LIKE YOU
3

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□
□

VERY
MUCH LIKE
YOU
4

4

4

4

4

4

4

4

4

□
□
□
□
□
□
□
□
□
4/30/14

G3.

The next question is about people in general.
Generally speaking, would you say most people can be trusted or that you can’t be too careful in life?
SELECT ONE ONLY
1
2

G4.

□
□

Most people can be trusted
Can’t be too careful

Do you agree or disagree with the following statements about your current relationships with family
and friends?
SELECT ONE RESPONSE PER ROW

STRONGLY
AGREE
a. There are people I know who will help me if I
really need it ......................................................

1

b. There is no one I feel comfortable talking about
problems with .....................................................

1

c. I am with a group of people who think the
same way I do about things ...............................

1

d. If something went wrong, no one would help
me ......................................................................

1

e. I have a trustworthy person to turn to if I have
problems ............................................................

1

f. There is no one who likes to do the things I do .

Prepared by Mathematica Policy Research

1

2

□
□
□
□
□
□

AGREE
2

2

2

2

2

2

□
□
□
□
□
□

DISAGREE
3

3

3

3

3

3

□
□
□
□
□
□

STRONGLY
DISAGREE
4

4

4

4

4

4

□
□
□
□
□
□

4/30/14

H. Identity Development

H1.

The next questions are about how you make decisions and solve problems.
Please read the following statements and rate how strongly you agree or disagree with each one.
SELECT ONE RESPONSE PER ROW

STRONGLY
AGREE
a. Difficult problems make you very upset .............

1

b. When making decisions, you usually go with
your “gut feeling” without thinking too much
about the consequences of each alternative .....

1

c. When you have a problem to solve, one of the
first things you do is get as many facts about
the problem as possible .....................................

1

d. When you are attempting to find a solution to a
problem, you usually try to think of as many
different ways to approach the problem as
possible ..............................................................

1

e. When making decisions, you generally use a
systematic method for judging and comparing
alternatives.........................................................

1

f. After carrying out a solution to a problem, you
usually try to analyze what went right and what
went wrong.........................................................

1

H2.

□
□
□
□
□
□

AGREE
2

2

2

2

2

2

□
□
□
□
□
□

DISAGREE
3

3

3

3

3

3

□
□
□
□
□
□

STRONGLY
DISAGREE
4

4

4

4

4

4

□
□
□
□
□
□

This next question is about what you do when you need to make a decision or solve a problem.
Would you rather get $80 tomorrow, or get $100 three months from now?
SELECT ONE ONLY
1
2

H3.

□
□

$80 tomorrow
$100 three months from now

The next questions ask you some questions about volunteer or community service work.
In the past year, have you ever performed volunteer or community service work, not including court
ordered service?
1
0

H3a.

□
□

Yes
No

GO TO H4

In the past year, about how often have you volunteered? You can answer with the number of hours
per week, month, or year.
|

|

|

| NUMBER OF HOURS

SELECT ONE ONLY
1
2
3

□
□
□

Per Week
Per Month
Per Year

Prepared by Mathematica Policy Research

3

4/30/14

H3b.

In the past year, have you:
SELECT ONE RESPONSE PER ROW

YES
a. Cleared most of your debts?.............................................................

1

b. Paid off parking tickets? ....................................................................

1

H4.

DOES NOT
APPLY TO
ME

NO

□

0

□

0

□

n

□

n

□
□

Are you registered to vote in the U.S.?
1
0

H5.

□
□

Yes
No

In the past year, have you:
SELECT ONE RESPONSE
PER ROW

YES
SKIP H5a IF YOU ANSWERED “NO” TO H4
a. Voted in one or more elections? ........................................................................
b. Taken on a leadership role by serving on a community council or board? .......
c.

H6.

Gotten involved in a national, state, or local political effort or electoral
campaign? .........................................................................................................

1

1

1

□
□
□

NO
0

0

0

□
□
□

These next questions ask about other leadership roles you may have. In the past year, have you:
SELECT ONE RESPONSE
PER ROW

YES
a. Served in a leadership role in the workplace? ...................................................

1

b. Served in a leadership role in the neighborhood? .............................................

1

c.

1

Served in a leadership role in church? ..............................................................

Prepared by Mathematica Policy Research

4

□
□
□

NO
0

0

0

□
□
□

4/30/14

H7.

These next questions ask about things you might do to help your family or community.
In the past month how often have you done the following activities? Would you say, not at all, a few
times a months, a few times a week, or every day?
SELECT ONE RESPONSE PER ROW

NOT AT
ALL
a. Helped family members .......................................................

1

b. Helped other members of the community............................

1

c. Worked in a community based program ..............................

1

d. Attended community meetings to help improve the
conditions within my community ..........................................

1

e. Assisted a neighbor or community member who needed
help. .....................................................................................

1

f. Served as a positive role model for the younger kids in my
neighborhood .......................................................................

1

g. Volunteered as a youth mentor or sports coach ..................

1

h. Influenced my friends to turn their lives around for the
better ....................................................................................

1

i. Discussed current events and social issues ........................

1

H8.

A FEW
TIMES A
MONTH

□
□
□

2
2
2

□

2

□

2

□
□

2
2

□
□

2
2

A FEW
TIMES A
WEEK

□
□
□

3
3
3

□

3

□

3

□
□

3
3

□
□

3
3

EVERY
DAY

□
□
□

4
4
4

□

4

□

4

□
□

4
4

□
□

4
4

□
□
□
□
□
□
□
□
□

In the past month how often have you done the following activities to help your child? Would you
say, not at all, a few times a months, a few times a week, or every day?
IF YOU DON’T HAVE A CHILD, GO TO QUESTION H9 ON THE NEXT PAGE
SELECT ONE RESPONSE PER ROW

A FEW
TIMES A
MONTH

NOT AT
ALL
a. Advocated for my child at her/his school ...
b. Volunteered at my child’s school ...............
c. Served in a leadership role in my child’s
school.........................................................

Prepared by Mathematica Policy Research

1

1

1

□

2

□

2

□

2

5

□
□
□

A FEW
TIMES A
WEEK
3

3

3

□
□
□

EVERY
DAY
4

4

4

□
□
□

I DON’T
HAVE A
CHILD
5

5

5

□
□
□

4/30/14

H9.

The next questions are about things you do in a typical day. First, how many hours do you usually
sleep?
SELECT ONE ONLY
1
2
3
4
5

H10.

□
□
□
□
□

Less than three hours
3 to 5 hours
6 to 8 hours
9 to 11 hours
More than 11 hours

Thinking about a typical day, how many hours do you usually spend:

SELECT ONE RESPONSE PER ROW

NONE OR
ALMOST
NONE
a. Watching TV or listening to music? ...........
b. Hanging out? .............................................
c. Playing video or computer games or using
the internet? ...............................................
d. Reading or studying? .................................
e. In school or training? .................................
f. Taking care of a child? ...............................
g. Taking care of a family member, not
including a child? .......................................

H11.

1

1

1

1

1

1

1

□

ONE
2

□

2

□

2

□

2

□

2

□

2

□

2

□
□
□
□
□
□
□

TWO OR
THREE
3

3

3

3

3

3

3

FOUR OR
FIVE

□
□
□
□
□
□
□

4

4

4

4

4

4

4

□
□
□
□
□
□
□

SIX OR
MORE
5

5

5

5

5

5

5

□
□
□
□
□
□
□

Taking everything all together, how would you say things are these days – would you say that you are
very happy, pretty happy, or not too happy?
SELECT ONE ONLY
1
2
3

□
□
□

Very happy
Pretty happy
Not too happy

Thank you for taking the time to complete this survey.

Prepared by Mathematica Policy Research

6

4/30/14

APPENDIX D
YOUTHBUILD 48-MONTH FOLLOW UP SURVEY:
WEB VERSION

OMB No.: 1205-0503
Expiration Date: 12/31/2015

YouthBuild
48-Month Follow Up Survey
Web version

June 2014

Persons are not required to respond to this collection of information unless this survey displays a currently valid OMB control
number (OMB 1205-0503, expires 06/30/2018). Responding to this questionnaire, which seeks to help the U.S. Department of Labor
understand how YouthBuild programs are serving disadvantaged youth, is voluntary. Public reporting burden for this collection of
information is estimated to average 35 minutes per response, including the time for reviewing instructions, searching existing data
sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments
regarding this burden estimate to Eileen Pederson, U.S. Department of Labor, Employment and Training Administration, Office of
Policy Development and Evaluation, Room N-5641, 200 Constitution Avenue, NW, Washington, DC 20210.

Prepared by Mathematica Policy Research

Preloads

Values Definitions / Format

Source

[YB PROGRAM]

Text

Grantee

REF DATE

MM/DD/YYYY

Last interview date or RA date

RAOutcome

1 = Treatment
2 = Control

RA Outcome from SMS

12M_comp

1 = Completed 12M survey
0= Did not complete 12M survey

1 IF 12M Final Status is 010, 020, 030,
019, 029, 039; ELSE 0

PL_dropout

1 = Ask B1
0 = Do not ask B1

1 IF W1_b1 = D, R, M OR IF 12M_comp =
0;
0 IF W1_b1 = 1 OR 0 (Data collected)
ELSE, 1.

PL_diploma

0 = Data not collected, ask B3
1 = High school diploma
2 = GED
3 = Certificate of completion
4 = None of these reported, ask B3

0 IF W1_b3 = D, R, M OR IF 12M_comp =
0;
1 IF W1_b3 = 1 (Reported high school
diploma at 12M)
2 IF W1_b3 = 2 (Reported GED at 12M)
3 IF W1_b3 = 3 (Reported Certificate of
completion at 12M)
4 IF W1_b3 = 4 (Reported none of these at
12M)
ELSE, 0.

PL_dipdate

1 = Reported education, but no date
0 = No education date provided

1 IF (W1_b4_month OR W1_b4_year = M,
D, R);
ELSE, 0

PL_YBstatus

0 = Did not complete 12M OR did not
report YB services OR did not report
YB status
1 = Reported currently enrolled in YB
at 12M
2 = Reported graduating YB at 12M
3 = Reported stopped going to YB at
12

1 IF W1_d3 = 1 (currently enrolled at 12M);
2 IF W1_d3 = 2 (graduated);
3 IF W1_d3 = 3 (stopped going);
ELSE , 0

PL_job

1 = Had current job at 12M survey
0 = No reported job or no current job at
12M survey

1 IF W1_e1 = 1 (currently working) and
(W1_e5 or W1_e5b) have job name
ELSE, 0

[12M JOB NAME]

Text

IF PLjob = 1, fill from W1_e5 or W1_e5b

FACEBOOK

0 = We have FB account
1 = We do not have FB account in db

0 IF Facebook account exists and we have
URL;
ELSE 1.

MYSPACE

0 = We have MS account
1 = We do not have MS account in db

0 IF MySpace account exists AND we
have URL;
ELSE 1.

TWITTER

0 = We have Twitter account
1 = We do not have Twitter account in
db

0 IF Twitter account exists AND we have
URL;ELSE 1.

Prepared by Mathematica Policy Research

CONTENTS
Section

Page

A.

HOUSEHOLD INFORMATION ...................................................................................2

H.

IDENTITY DEVELOPMENT........................................................................................8

I.

HEALTH AND WELL-BEING ....................................................................................12

Prepared by Mathematica Policy Research

1

A. Household Information
ALL
A1.

The first questions are about your housing situation.
Where do you live now? If you stay in more than one place, where do you stay most often?
Select one only
 Your parent’s home .............................................................................................. 1
 Another person’s home ........................................................................................ 2
 Your own place whether you rent or own ............................................................. 3
 A group home or halfway house ........................................................................... 4
 Dormitory (such as college or Job Corps housing) ............................................. 10
 A long-term homeless shelter ............................................................................... 5

GO TO A3

 An emergency housing shelter including a domestic violence shelter ................. 6

GO TO A3

 An incarceration facility......................................................................................... 7
 Homeless and living on the street ........................................................................ 8

GO TO A3

 Some other arrangement...................................................................................... 9
Specify

(STRING 200)

NO RESPONSE ................................................................................................... M
SPECIFY TEXT: Please specify where you live now:
IF A1 NE 5, 6, 8 (NOT CURRENTLY LIVING IN SHELTER OR HOMELESS)
A2.

Have you been homeless and living on the street or in a shelter at any time since [REF
DATE]?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

ALL
A3.

What is your marital status?
Select one only
 Married .................................................................................................................. 1
 Divorced................................................................................................................ 2
 Separated ............................................................................................................. 3
 Widowed ............................................................................................................... 4
 Never married ....................................................................................................... 5
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

2

PROGRAMMER SKIP BOX A3.1
IF A1 = 1 – 6, 9, 10, M (LIVING IN HOUSE, OTHER, OR EMPTY)
CONTINUE;
ELSE, SKIP TO A6.

A1 = 1 – 6, 9, 10, M
Does your spouse currently live IF A3 = 1;
ELSE Do you have a partner who currently lives
A4.

[Does your spouse currently live / Do you have a partner who currently lives] with you?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

A1 = 1, 2, 3, 9, M
A5.

Including yourself, how many people currently live with you? Include everyone who usually
lives there, even if they are away from home right now.

PEOPLE CURRENTLY LIVING WITH YOU
(RANGE 0-99)
NO RESPONSE ................................................................................................... M
HARD CHECK: IF A5 = 0; Your response to this question should be at least one (1). Please update
your answer below.
SOFT CHECK: IF A4 = 1 AND A5 < 2; In the previous question you mentioned you live with your
spouse or partner. Your answer to this question should include yourself and your spouse or
partner. Please indicate below how many people currently live with you.
To continue to the next question without changing your response, click the continue button.

ALL
A6a.

Do you have any children under 18 years of age? Please include your own or adopted
children, foster, or stepchildren. Please do not include a current pregnancy.
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

3

GO TO SECTION B
GO TO SECTION B

A6a = 1
A7.

How many children do you have? Please do not include a current pregnancy.
CHILDREN
(RANGE 00-99)
NO RESPONSE ................................................................................................... M

SOFT CHECK: IF A7 =15 – 99; You said that you have [FILL A7] children. Please check that this is
correct and either keep your answer or change your answer below.
To keep your answer without making changes, click the continue button.
SOFT CHECK: IF A7 = 0; You mentioned in a previous question that you have children. You just
mentioned that you have 0 children. Click here to go back and change your answer about having
children. You may also change your answer below.
To keep your answer without making changes, click the continue button.
PROGRAMMER SKIP BOX A7.1
IF A7 = 0 OR M (NO CHILDREN OR MISSING), SKIP TO SECTION B;
IF A7 = 1 (ONE CHILD), CONTINUE TO A8a;
IF A7 GT 1 (MULTIPLE CHILDREN), SKIP TO A8b.

A7 = 1
A8a.

Does this child live with you?
 Yes ........................................................................................................................ 1

GO TO A9

 No ......................................................................................................................... 0

GO TO A9

NO RESPONSE ................................................................................................... M

GO TO A9

SOFT CHECK: IF A8a = M; Your response to this question is important. Please provide a response
and continue.
To continue to the next question without providing a response, click the continue button.

Prepared by Mathematica Policy Research

4

A7 GT 1
A8b.

How many of your children live with you?
CHILDREN LIVING WITH YOU
(RANGE 0-99)
NO RESPONSE ................................................................................................... M

SOFT CHECK: IF A8b = M; Your response to this question is important. Please provide a response
and continue.
To continue to the next question without providing a response, click the continue button.
SOFT CHECK: IF A8b GT A7; You mentioned in a previous question that you have [FILL A7]
children. The number of children living with you have should not be greater than the number of
children you have. Click here to go back and change your answer about the number of children
you have. You may also change your answer below.
To continue to the next question without making changes, click the continue button.

A7 GT 0 (HAS CHILDREN)
A9.

IF A7 = 1:

Was this child born after [REF DATE]?

IF A7 GT 1: Were any of your children born after [REF DATE]?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

PROGRAMMER SKIP BOX A9.1
IF A7 = 1 AND A8a = 0 (ONLY CHILD IS NOT LIVING IN HOME),
CONTINUE TO A10a;
IF A7 = 1 AND A8a = 1 (ONLY CHILD IS LIVING AT HOME), GO TO A11;
IF A7 = 1 AND A8a = M, GO TO A11;
IF A7 GT 1 AND (A7 – A8b GT 0) (NOT ALL CHILDREN LIVING IN
HOME), CONTINUE TO A10a;
IF A7 GT1 AND (A7 – A8b LE 0) (ALL CHILDREN LIVING IN HOME), GO
TO A11;
IF A7 = GT 1 AND A8b = M, GO TO A11.

Prepared by Mathematica Policy Research

5

IF (A7 = 1 AND A8a = 0) OR (A7 GT 1 AND (A7 – A8b GT 0))
child, does IF A7 = 1 AND A8a = 0;
children, do IF A7 GT 1 AND (A7 – A8b GT 1);
child, does IF A7 GT1 AND (A7 – A8b = 1)
A10a. Did you spend time with your [child / children] who [does / do] not live with you in the past
week?
 Yes ........................................................................................................................ 1

GO TO A10

 No ......................................................................................................................... 0

GO TO A11

NO RESPONSE ................................................................................................... M

GO TO A11

IF A10a = 1
IF A7 = 1 AND A8a = 0, FILL: child, does
IF A7 GT 1 AND (A7 – A8B GT 1), FILL: children, do
IF A7 GT1 AND (A7 – A8B = 1), FILL; child, does
A10.

Thinking just about the [child / children] who [does / do] not live with you, please indicate
which of the following activities you have done in the past week.
Select all that apply
 Read with your [child/children].............................................................................. 1
 Played with your [child/children], not including sports .......................................... 2
 Did arts and crafts with your [child/children] ......................................................... 3
 Played sports with your [child/children] ................................................................ 4
 Talked with or listened to your [child/children]...................................................... 5
 Attended your [child/children]’s events ................................................................. 6
 Helped your [child/children] with homework ......................................................... 7
 Picked up or dropped your [child/children] off ...................................................... 8
 Done some other activities with your [child/children]............................................ 9
NO RESPONSE .......................................................................................................... M

Prepared by Mathematica Policy Research

6

IF A7 GT 0 (HAS CHILDREN)
IF A7 GT 1 AND (A7 – A8B GT 1), FILL: [Now thinking of all your children]
IF A7 = 1, FILL: child
IF A7 GT 1, FILL: children
A11.

[Now, thinking of all your children] In the past month how often have you done the
following activities to help your [child/children]? Would you say, not at all, a few times a
months, a few times a week, or every day?
PROGRAMMER: CODE ONE PER ROW
Select one per row
NOT AT
ALL

A FEW
TIMES A
MONTH

A FEW
TIMES A
WEEK

EVERY
DAY

a. Talked to someone about my child’s needs at
her/his school.......................................................

1



2



3



4



b. Volunteered at my child’s school .........................

1



2



3



4



c.

1



2



3



4



Served in a leadership role in my child’s school..

IF A7 GT 0
A12.

Do you have any legal agreements or child support orders that require you to provide
financial support for a child?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

A11 = 1
A13.

Did you make this payment last month?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

7

GO TO SECTION B
GO TO SECTION B

H. Identity Development
ALL
H1.

The next questions are about how you make decisions and solve problems.
Please read the following statements and rate how strongly you agree or disagree with
each one.
PROGRAMMER: CODE ONE PER ROW
Select one per row
STRONGLY
AGREE

AGREE

DISAGREE

STRONGLY
DISAGREE

a. Difficult problems make you very upset ...........

1



2



3



4



b. When making decisions, you usually go with
your “gut feeling” without thinking too much
about the consequences of each alternative ...

1



2



3



4



When you have a problem to solve, one of
the first things you do is get as many facts
about the problem as possible.........................

1



2



3



4



d. When you are attempting to find a solution to
a problem, you usually try to think of as many
different ways to approach the problem as
possible ............................................................

1



2



3



4



e. When making decisions, you generally use a
systematic method for judging and comparing
alternatives ......................................................

1



2



3



4



1



2



3



4



c.

f.

After carrying out a solution to a problem, you
usually try to analyze what went right and
what went wrong ..............................................

ALL
H2.

This next question is about what you do when you need to make a decision or solve a
problem.
Would you rather get $80 tomorrow, or get $100 three months from now?
Select one only
 $80 tomorrow ........................................................................................................ 1
 $100 three months from now ................................................................................ 2
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

8

ALL
H3.

The next questions ask you some questions about volunteer or community service work.
Since [REF DATE], have you ever performed volunteer or community service work, not
including court ordered service?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

IF H3 = 1
H3a.

Since [REF DATE], about how often have you volunteered? You can answer with the
number of hours per week, month, or year.
PROGRAMMER:

INSERT DROPDOWN

PROGRAMMER:

RANGE IS 00 – 150

Number of hours Per Week, Month, or Year
Please Select▼
PROGRAMMER: USE THESE UNIT OPTIONS
Per Week .............................................................................................................. 1
Per Month ............................................................................................................. 2
Per Year................................................................................................................ 3

AMOUNT NO RESPONSE................................................................................... M
UNIT NO RESPONSE .......................................................................................... M

ALL
H4.

Are you registered to vote in the U.S.?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

9

ALL
DISPLAY OPTION a ONLY IF H4 NE 0
H5.

Since [REF DATE] have you:
PROGRAMMER: CODE ONE PER ROW
Select one per row
YES

NO

a. Voted in one or more elections? .....................................................................

1



0



b. Taken on a leadership role by serving on a community council or board? ....

1



0



1



0



c.

Gotten involved in a national, state, or local political effort or electoral
campaign? ......................................................................................................

ALL
H6.

These next questions ask about other leadership roles you may have. In the past month,
have you:
PROGRAMMER: CODE ONE PER ROW
Select one per row
YES

NO

a. Served in a leadership role in the workplace?................................................

1



0



b. Served in a leadership role in the neighborhood? ..........................................

1



0



Served in a leadership role in church or another religious or spiritual
group? .............................................................................................................

1



0



d. Served in a leadership role in school? ...........................................................

1



0



c.

ALL
H7.

These next questions ask about things you might do in your community.
In the past month, how often have you done the following activities? Would you say, not
at all, a few times a months, a few times a week, or every day?
PROGRAMMER: CODE ONE PER ROW
Select one per row
A FEW A FEW
NOT AT TIMES A TIMES A EVERY
ALL
MONTH WEEK
DAY

a. Helped members of the community ......................................

1



2



3



4



b. Attended community meetings to help improve the conditions
within my community ............................................................

1



2



3



4



Served as a positive role model for the younger kids in my
neighborhood ........................................................................

1



2



3



4



d. Volunteered as a youth mentor or sports coach...................

1



2



3



4



e. Influenced my friends to turn their lives around for the better

1



2



3



4



1



2



3



4



c.

f.

Discussed current events (events in the news, such as an
election) and social issues (problems that affect society, such
as health care) ......................................................................

Prepared by Mathematica Policy Research

10

ALL
H8.

The next questions are about things you do in a typical day. First, how many hours do you
usually sleep?
Select one only
 Less than three hours ..................................................................................... 1
 3 to 5 hours ..................................................................................................... 2
 6 to 8 hours ..................................................................................................... 3
 9 to 11 hours ................................................................................................... 4
 More than 11 hours ........................................................................................ 5
 NO RESPONSE ............................................................................................. M

ALL
H9.

Thinking about a typical day, how many hours do you usually spend:
PROGRAMMER: CODE ONE PER ROW
Select one per row
NONE OR
ALMOST
NONE

ONE

TWO OR FOUR SIX OR
THREE OR FIVE MORE

a. Watching TV or listening to music .......................

1



2



3



4



5



b. Hanging out .........................................................

1



2



3



4



5



Playing video or computer games or using the
internet .................................................................

1



2



3



4



5



d. Reading or studying .............................................

1



2



3



4



5



e. In school or training .............................................

1



2



3



4



5



f.

Taking care of a child ..........................................

1



2



3



4



5



g. Taking care of a family member, not including a
child .....................................................................

1



2



3



4



5



c.

ALL
H10.

Taking everything all together, how would you say things are these days – would you say
that you are very happy, pretty happy or not too happy?
Select one only
 Very happy............................................................................................................ 1
 Pretty happy.......................................................................................................... 2
 Not too happy ....................................................................................................... 3
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

11

I. Health and Well-Being
ALL
I1.

The next questions are about your general health.
In general, would you say your health is:
Select one only
 Excellent ............................................................................................................... 1
 Very good ............................................................................................................. 2
 Good ..................................................................................................................... 3
 Fair ........................................................................................................................ 4
 Poor ...................................................................................................................... 5
NO RESPONSE ................................................................................................... M

ALL
or women’s health IF GENDER = FEMALE
I2.

Since [REF DATE], have you had…
PROGRAMMER: CODE ONE PER ROW
Select one per row
YES

NO

a. An annual checkup including general health [or women’s health] .................

1



0



b. A dental exam .................................................................................................

1



0



c.

1



0



A flu shot .........................................................................................................

ALL
I3.

What age do you think you will live to?
YEARS OLD
(RANGE 0-999)
NO RESPONSE ................................................................................................... M

SOFT CHECK: IF I3 GT 100 OR LT 20; You said you think you will live to [FILL I3] years old. Please
check that this is correct and either keep your answer or change your answer below.
To continue to the next question without providing a response, click the continue button.

Prepared by Mathematica Policy Research

12

ALL
I4.

Over the last 2 weeks, how often have you been bothered by any of the following
problems?
PROGRAMMER: CODE ONE PER ROW
Select one per row
MORE
THAN NEARLY
NOT AT SEVERAL HALF THE EVERY
ALL
DAYS
DAYS
DAY

a. Little interest or pleasure in doing things ..............................

1



2



3



4



b. Feeling down, depressed, or hopeless .................................

1



2



3



4



c.

Trouble falling or staying asleep, or sleeping too much .......

1



2



3



4



d. Feeling tired or having little energy.......................................

1



2



3



4



e. Poor appetite or overeating ..................................................

1



2



3



4



Feeling bad about yourself – or that you are a failure and
have let yourself or your family down ...................................

1



2



3



4



g. Trouble concentrating on things, such as reading the
newspaper or watching television.........................................

1



2



3



4



h. Moving or speaking so slowly that other people could have
noticed. Or the opposite – being so fidgety or restless that
you have been moving around a lot more than usual ..........

1



2



3



4



1



2



3



4



f.

i.

Thoughts that you would be better off dead, or of hurting
yourself in some way ............................................................

ANY I4a – i = 2 – 4 (ANYTHING CHECKED AT I4)
I5.

How difficult have these problems made it for you to do your work, take care of things at
home, or get along with other people?
Select one only
 Not difficult at all ................................................................................................... 1
 Somewhat difficult ................................................................................................ 2
 Very difficult .......................................................................................................... 3
 Extremely difficult ................................................................................................. 4
NO RESPONSE ................................................................................................... M

ALL
I6.

The next few questions are about gang activity in your neighborhood. Remember, all of
your responses will be kept private to the fullest extent of the law.
Have you ever belonged to a gang?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

Prepared by Mathematica Policy Research

13

ALL
FILL IF I6 = 0, M: even if you aren’t in a gang
I7.

In the past 12 months, have you been involved in gang fights [even if you aren’t in a
gang]?
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
NO RESPONSE ................................................................................................... M

ALL
I8.

The next few questions are about drug and alcohol use. Remember, all of your responses
will be kept private to the fullest extent of the law.
In a typical week, how many times do you have five or more drinks in a row? (A “drink” is
a glass of wine, a bottle of beer, a shot glass of liquor, or a mixed drink.)
Select one only
 Never .................................................................................................................... 1
 Less than once ..................................................................................................... 2
 Once ..................................................................................................................... 3
 Twice .................................................................................................................... 4
 Three to five times ................................................................................................ 5
 Every day or almost every day ............................................................................. 6
NO RESPONSE ................................................................................................... M

ALL
I9.

Since [REF DATE], have you used or tried any of the following drugs?
PROGRAMMER: CODE ONE PER ROW
Select one per row
YES

NO

a. Marijuana ........................................................................................................

1



0



b. Any kind of cocaine – including powder, freebase, or crack cocaine .............

1



0



c.

Inhalants, such as glue or solvents ................................................................

1



0



d. Pills without a doctor’s prescription ................................................................

1



0



e. Any other type of illegal drug, such as methamphetamine, LSD, PCP,
ecstasy, mushrooms, speed, ice, or heroin ....................................................

1



0



PROGRAMMER SKIP BOX I9.1
IF I8 = 2 – 6 OR ANY I9a-e = 1, CONTINUE.
ELSE SKIP TO SECTION J.

Prepared by Mathematica Policy Research

14

IF I8 = 2 – 6 OR ANY I9a – e = 1
drinking IF I8 = 2 – 6 AND ALL I9a-e = 0, M;
drug use IF I8 = 1, M AND ANY I9a-e = 1;
drinking or drug use IF I8 = 2 – 5 AND ANY I9a-e = 1
I10.

Since [REF DATE], how often has your [drinking / drug use / drinking or drug use]
interfered with your responsibilities either at work or school, or something else?
Select one only
 Never .................................................................................................................... 1
 1 time .................................................................................................................... 2
 More than 1 time ................................................................................................... 3
NO RESPONSE ................................................................................................... M

IF I8 = 2 – 6 OR ANY I9a – e = 1
drinking alcohol IF I8 = 2 – 6 AND ALL I9a-e = 0, M;
doing drugs IF I8 = 1, M AND ANY I9a-e = 1;
drinking alcohol or doing drugs IF I8 = 2 – 5 AND ANY I9a-e = 1
I11.

During the past 30 days, how many times did you drive a car or other vehicle when you
had been [drinking alcohol / doing drugs / drinking alcohol or doing drugs]?
Select one only
 0 times .................................................................................................................. 1
 1 time .................................................................................................................... 2
 2 to 3 times ........................................................................................................... 3
 4 to 5 times ........................................................................................................... 4
 6 or more times ..................................................................................................... 5
NO RESPONSE ................................................................................................... M

ALL
I12.

In the past month, have you cleared most of your debts (money you owe such as loans?)
Select one only
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
 Does not apply to me............................................................................................ 2
NO RESPONSE .......................................................................................................... M

Prepared by Mathematica Policy Research

15

ALL
I12.

In the past month, have you paid off parking tickets?
Select one only
 Yes ........................................................................................................................ 1
 No ......................................................................................................................... 0
 Does not apply to me............................................................................................ 2
NO RESPONSE .......................................................................................................... M

Prepared by Mathematica Policy Research

16