May 28, 2019
NOTE TO THE REVIEWER OF:
|
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research” |
FROM: |
Erica Yu Office of Survey Methods Research |
SUBJECT: |
Submission of Materials for “Reducing Respondent Burden” |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The total estimated respondent burden for this study is 253 hours.
If there are any questions regarding this project, please contact Erica Yu at 202-691-7924.
Introduction
In the current climate of declining response rates, a common mandate to survey programs is to reduce respondent burden. Typically, the practical effects of such directives are to reduce the objective measures of burden, such as the length of the survey or the number of questions in the survey. These measurements of objective burden are easy to assess, reliable, and can be administered across all federal data collections. However, such an approach to respondent burden places respondent burden and data quality as rivalling ends in a trade-off equation, whereby reductions in burden must come at the cost of reductions in data quantity and quality.
Research has proposed that subjective perceptions of burden, such as feelings of effort, sensitivity, and lack of interest in the survey, may also play a role in respondent ratings of burden and associated survey outcomes (Yan, Fricker, & Tsai, 2014). Unlike objective factors, subjective factors are grounded in cognition, which need not be tied to objective factors. Yu, Fricker, & Kopp (2015) demonstrated this dissociation in a study where participants who completed the same survey reported feeling higher or lower levels of burden depending on subjective factors (perceived length manipulated by instructions). Objective burden measurements do not capture the effort, tedium, sensitivity and other subjective factors that affect the respondent experience. This gap in coverage of the respondent experience suggests that survey programs might successfully reduce objective burden but fail to achieve any meaningful reduction in overall respondent burden.
One common approach to reducing objective burden is to reduce the number of items asked in a survey. Such an approach to the reduction in the length of a questionnaire—where the reduction of the number of questions asked becomes the goal, apart from any parallel reduction in the amount or type of data to be collected—can be in direct conflict with subjective burden. For example, instead of separately asking respondents to report expenses for shirts and then for pants (“multiple detail questions”), now just one question will be used to ask respondents to report the combined total expense on shirts and pants (“single global question”). This solution may reduce objective burden by cutting the number of questions asked but it has not changed the burden of the underlying response process—the respondent must still recall and report all of their expenses for both shirts and pants.
Some may argue that an alternative approach to measuring objective burden—time spent, whereby more time spent equates to higher levels of burden—would be sensitive to these false changes and more accurately represent burden. In the same scenario, if the time spent answering the single survey question is less than the time spent answering the two survey questions, then burden has been reduced. However, using time spent as a measure of burden ignores the dynamic nature of survey response: respondents can respond to a difficult survey item by changing their response strategy, such as from optimal responding to satisficing, thus spending less time answering a survey question while still feeling burden.
This study aims to evaluate the hypothesis that some attempts to reduce objective burden can have no impact on – or can even increase – subjective respondent burden. The findings from this study will inform whether survey methodologists and designers should prioritize and formally measure subjective burden when evaluating the burden imposed by a survey on respondents.
Methodology
To test the impact on respondent burden of reducing the number of questions asked, we designed a mixed between- and within-subjects design experiment in which groups of randomly-assigned participants respond to pairs of surveys and rate the burden they experienced. Each participant will experience a pair of surveys: one survey will use multiple detail questions to collect information, whereas the other survey in the pair will use single global questions.
Each participant is asked to compare the two surveys. Although no respondents in real life would see two surveys, this approach addresses two methodological concerns. First, that burden ratings for short methodological research surveys are typically at the extreme low end of the rating scale, and so questionnaire design modification effects may not be large enough to evoke detectable differences between groups; Yu, Fricker, & Kopp (2015) found average burden ratings of 1.68 on an ordinal scale ranging from 1 to 5. Second, that with non-probability samples of this size it is possible that two samples recruited separately will differ on characteristics that may be relevant to the research question (e.g., attitudes towards surveys).
To explore the concept of whether participants dynamically change their response strategy (e.g., guessing rather than enumerating) as a reaction to the anticipated burden of the question style, we also collect participant’s contemporaneous reports of response strategy. Given that this intervention may affect the overall survey experience, we ask these contemporaneous probes for only half the sample.
The ideal comparison would be of two surveys collecting the same information thus isolating the manipulation of question style; however, it is necessary to change the content of the surveys within the pair to ensure that the participant must initiate the full response process anew for each question, which is critical to understanding the experience of burden. To account for this difference, survey content will be counterbalanced between groups. The full versions of the surveys are included in Appendix 1.
The order of the surveys will also be counterbalanced. Given that each participant will see two surveys, randomizing the presentation order of the surveys is important to ensure that ratings of a survey are not affected by the experience of taking the other survey.
Given the counterbalancing for order and content, there are eight survey protocols:
Survey 1: Single global questions Survey 2: Multiple detail questions |
With response strategy probes |
Content A - B |
Content B - A |
||
Without response strategy probes |
Content A - B |
|
Content B - A |
||
Survey 1: Multiple detail questions Survey 2: Single global questions |
With response strategy probes |
Content A - B |
Content B - A |
||
Without response strategy probes |
Content A - B |
|
Content B - A |
For each survey in the pair, the instrument will collect the responses entered by the participant as well as the objective measures of time spent on the questions (the number of questions is already known). After completing the target survey, participants will rate the overall burden and the hypothesized components of subjective burden. Then, participants will repeat that protocol for the paired survey. After completing both surveys, participants will be asked for feedback about the differences in burden between the two surveys, their attitudes towards surveys in general (Stocke, 2014; modified for US), and demographics; full protocols are included in Appendix 2.
We expect that modifications may be made during the course of the study based on initial piloting feedback. Although the goals of the testing, and overall design, will remain the same, findings from pilot results may be used to improve the instructions and question wordings to achieve the intended differences in perceptions of burden across survey components.
Participants
Participants will be recruited as a convenience sample from Amazon Mechanical Turk of adults (18 years and older) residing in the U.S.; this study is focused on internal validity rather than representativeness of any population. This research design requires a sample of 960 participants in order to sufficiently explore the range of variables of interest. These participants will be randomly assigned to the 8 groups described (120 participants per group). A prior study that successfully found main effects and was based on measuring respondent burden used a sample with 80 participants per group. The sample size has been increased to account for the possibility that order effects are significant and so the data cannot be collapsed across sub-groups and that the effect size being explored in the present study is smaller.
An additional 75 participants will be recruited for initial pilot tests, also on Amazon Mechanical Turk. These pilot participants will be asked to complete components of the surveys and confirm that questions and instructions are worded clearly and experimental manipulations result in sufficiently different perceptions of the survey experience.
Burden Hours
We anticipate that the task will take no longer than 15 minutes, totaling 240 burden hours across 960 participants. Piloting of sections of the task will involve 75 test participants completing surveys no longer than 10 minutes, and totaling 12.5 burden hours. In total, the study will require 1,035 participants and 253 burden hours. The surveys will be administered completely online at the time and location of the participant’s choosing.
Payment to Participants
We will recruit 1,035 participants from the Amazon Mechanical Turk database (75 participants for pilot tests and 960 participants for the main survey). For the shorter pilot tests, participants will be paid $1.50 for participating in the study ($112.50 total); for the longer main data collection, the participants will be paid $3.00 ($2,880.00 total). These are typical rates offered on the Amazon Mechanical Turk platform for similar tasks. A total of $2,292.50 will be paid directly to Amazon Mechanical Turk for participant fees.
Data Confidentiality
Recruiting of participants will be handled by Amazon Mechanical Turk. Once participants are recruited into the study, they will be given a link to the survey, which is hosted by Qualtrics. The data collected as part of this study will be stored on Qualtrics servers. Using the language shown below, participants in the main data collection will be informed of the voluntary nature of the study and they will not be given a pledge of confidentiality.
This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141 (Expiration Date: March 31, 2021). Without this currently-approved number, we could not conduct this survey. We estimate that it will take on average 15 minutes to complete this survey. Your participation is voluntary, and you have the right to stop at any time. This survey is being administered by Qualtrics and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.
Participants in the pilot tests, which are shorter, will be shown a different time estimate:
We estimate that it will take on average 10 minutes to complete this survey.
Attachments
Appendix 1: Introduction, burden comparison, attitude and demographics questions
Appendix 2: Survey versions
Appendix 1: Introduction, burden comparison, attitude and demographics questions
Welcome! Thanks for your interest in this study.
This study is part of our research into how to improve surveys. Please do your best to respond to the questions accurately. We are not collecting any personally identifiable information in the survey questions.
The study should take about 15 minutes. Please only start the study when you will be able to complete the whole study without interruption.
Please do not use your browser's back button.
This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141 (Expiration Date: March 31, 2021). Without this currently-approved number, we could not conduct this survey. We estimate that it will take on average 15 minutes to complete this survey. Your participation is voluntary, and you have the right to stop at any time. This survey is being administered by Qualtrics and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.
---page break---
On the following pages, you’ll be asked to do the following:
Survey 1
You’ll be asked a short set of questions. Then, you’ll be asked about your experience answering that survey.
Survey 2
You’ll be asked a different short set of questions. Then, you’ll be asked about your experience answering that survey and also how the two surveys compare to each other.
Questions about yourself
You’ll be asked about your attitudes towards surveys, your experience with surveys, and your demographics.
To get started, please click “continue”.
---page break---
[Insert Survey 1, Ratings of Survey 1, Survey 2, Ratings of Survey 2]
---page break---
Now please compare Survey 1 and Survey 2.
Overall, which survey did you prefer?
Survey 1
Survey 2
No preference
What differences, if any, did you notice between the surveys?
[open text entry, large box]
Which survey was more burdensome?
Survey 1
Survey 2
They were the same
On which survey was it more difficult to understand what the questions were asking?
Survey 1
Survey 2
They were the same
On which survey was it more difficult to come up with your answers?
Survey 1
Survey 2
They were the same
On which survey did you put in more mental effort for answering the questions?
Survey 1
Survey 2
They were the same
On which survey did the questions require more effort to provide accurate responses?
Survey 1
Survey 2
They were the same
[If not in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the questions that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Now please think about surveys in general, including surveys outside of mTurk, such as polls, federal surveys, academic surveys, and market research.
For each of the following statements, please rate the extent to which you agree or disagree.
[Grid with 7-point scale where 1 = Strongly Disagree and 7 = Strongly Agree]
Surveys are important for science, politics and the economy
Public opinion surveys make society more democratic
Participation in surveys is a matter of self-interest
Everybody should take participating in surveys for granted
Surveys make life more varied and are interesting in themselves
In surveys you are at last able to articulate your own opinion
In most cases survey results are correct
Most surveys are carried out in a reliable and responsible way
Ordinary, representative people are interviewed in surveys
Survey participants are likely to give their true opinion
Survey participants try hard to respond accurately
Surveys only keep you from doing more important things
Topics are addressed in surveys that don’t concern anybody else
Far too many surveys are carried out in the US
It is exhausting to answer so many questions in surveys
Interviewers are unpleasant and obtrusive people
---page break---
And now just a few final questions about yourself.
In what year were you born?
[open text entry, small box; numeric]
What is your sex?
Male
Female
Are you Hispanic, Latino, or Spanish?
Yes
No
What is your race? (Please select one or more)
White
Black or African American
American Indian or Alaska native
Asian
Native Hawaiian or Pacific Islander
What is the highest degree you have received?
No schooling completed
Elementary school diploma
High school diploma or the equivalent (GED)
Associate degree
Bachelor’s degree
Master’s degree
Professional or doctorate degree
Approximately how many HITs have you completed on mTurk?
0-10
11-50
51-100
101-999
1000 or more
In the last 12 months, how often did you complete survey HITs? Please do not include classification, transcription, or other non-survey HITs.
Never
Seldom
Sometimes
Often
---page break---
Thank you for your participation! Here is your completion code. Please paste this code into the HIT to verify your participation.
[random numeric code]
If you have any other feedback about the study, please leave it here. Otherwise, leave this blank.
[open text entry, large box]
Appendix 2: Survey Versions
Version A
The following questions are about your household’s expenses. Please think of the entire household and do your best to give an accurate response.
Since the first of March, how much has your household spent on
Transportation services such as taxis, local mass transit, and long-distance buses and trains?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be for one month
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, how much has your household spent on
Men’s shirts, pants and shorts, and coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 1 as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
---page break---
That was the end of Survey 1.
Please click “continue” to move on to Survey 2.
---page break---
The
following questions are about your household’s expenses. Please
think of the entire household and do your best to give an accurate
response.
Since the first of March, has your household had any expense for
Women’s shirts, pants and shorts, and coats and jackets?
Yes
No
---page break---
(If yes, then show follow-up questions)
How much did your household spend on
Women’s shirts, pants and shorts, and coats and jackets
during the month of:
Women’s shirts?
[open text-entry; small box; numeric]
Women’s pants and shorts?
[open text-entry; small box; numeric]
Women’s coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, has your household had any expenses for
Pets, pet services, and pet products including pet food?
Yes
No
---page break---
(If yes, then show follow-up questions)
How much did your household spend on
Pets, pet services, and pet products including pet food
during the month of:
Pets?
[open text-entry; small box; numeric]
Pet services?
[open text-entry; small box; numeric]
Pet products including pet food?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 2 only as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
Version B
The following questions are about your household’s expenses. Please think of the entire household and do your best to give an accurate response.
Since the first of March, how much has your household spent on
Women’s shirts, pants and shorts, and coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, how much has your household spent on
Pets, pet services, and pet products including pet food?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 1 as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
---page break---
That was the end of Survey 1.
Please click “continue” to move on to Survey 2.
---page break---
The following questions are about your household’s expenses. Please think of the entire household and do your best to give an accurate response.
Since the first of March, has your household had any expenses for
Transportation services such as taxis, local mass transit, or long-distance buses or trains?
Yes
No
---page break---
(If yes, then show follow-up questions)
Since the first of March, how much has your household spent on
Taxis?
[open text-entry; small box; numeric]
Local mass transit?
[open text-entry; small box; numeric]
Long-distance buses and trains?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, has your household had any expenses for
Men’s shirts, pants and shorts, or coats and jackets?
Yes
No
---page break---
(If yes, then show follow-up questions)
Since the first of March, how much has your household spent on
Men’s shirts?
[open text-entry; small box; numeric]
Men’s pants and shorts?
[open text-entry; small box; numeric]
Men’s coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 2 only as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
Version C
The following questions are about your household’s expenses. Please think of the entire household and do your best to give an accurate response.
Since the first of March, has your household had any expenses for
Transportation services such as taxis, local mass transit, or long-distance buses or trains?
Yes
No
---page break---
(If yes, then show follow-up questions)
Since the first of March, how much has your household spent on
Taxis?
[open text-entry; small box; numeric]
Local mass transit?
[open text-entry; small box; numeric]
Long-distance buses and trains?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, has your household had any expenses for
Men’s shirts, pants and shorts, or coats and jackets?
Yes
No
---page break---
(If yes, then show follow-up questions)
Since the first of March, how much has your household spent on
Men’s shirts?
[open text-entry; small box; numeric]
Men’s pants and shorts?
[open text-entry; small box; numeric]
Men’s coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 1 as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
---page break---
That was the end of Survey 1.
Please click “continue” to move on to Survey 2.
---page break---
The following questions are about your household’s expenses. Please think of the entire household and do your best to give an accurate response.
Since the first of March, how much has your household spent on
Women’s shirts, pants and shorts, and coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, how much has your household spent on
Pets, pet services, and pet products including pet food?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 2 only as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
Version D
The
following questions are about your household’s expenses. Please
think of the entire household and do your best to give an accurate
response.
Since the first of March, has your household had any expense for
Women’s shirts, pants and shorts, and coats and jackets?
Yes
No
---page break---
(If yes, then show follow-up questions)
How much did your household spend on
Women’s shirts, pants and shorts, and coats and jackets
during the month of:
Women’s shirts?
[open text-entry; small box; numeric]
Women’s pants and shorts?
[open text-entry; small box; numeric]
Women’s coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, has your household had any expenses for
Pets, pet services, and pet products including pet food?
Yes
No
---page break---
(If yes, then show follow-up questions)
How much did your household spend on
Pets, pet services, and pet products including pet food
during the month of:
Pets?
[open text-entry; small box; numeric]
Pet services?
[open text-entry; small box; numeric]
Pet products including pet food?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report expenses in each subcategory of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single month and then guessed what the total for all
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 1 as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
---page break---
That was the end of Survey 1.
Please click “continue” to move on to Survey 2.
---page break---
The following questions are about your household’s expenses. Please think of the entire household and do your best to give an accurate response.
Since the first of March, how much has your household spent on
Transportation services such as taxis, local mass transit, and long-distance buses and trains?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Since the first of March, how much has your household spent on
Men’s shirts, pants and shorts, and coats and jackets?
[open text-entry; small box; numeric]
---page break---
[If in condition to see response strategy probes]
Which of the following response strategies best matches how you answered the previous question that asked you to report total expenses across a group of items?
I tried to remember each expense I had and then summed those together
I tried to remember my expenses for a single item and then guessed what the total for all items would be
I made a guess without trying to remember any specific expenses
Other (please specify)
---page break---
Thanks for answering those questions. Now please think about your experience responding to Survey 2 only as you answer the following questions.
How burdensome was participating in this survey?
Extremely burdensome
Very burdensome
Somewhat burdensome
Slightly burdensome
Not at all burdensome
---page break---
How easy or difficult did you feel it was to understand what the questions were asking?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How easy or difficult did you feel it was to come up with your answers?
Very easy
Somewhat easy
Neither easy nor difficult
Somewhat difficult
Very difficult
How much mental effort did you put into answering the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
How much effort would it take to provide accurate responses to the questions?
Very high effort
Somewhat high effort
Neither high nor low effort
Somewhat low effort
Very low effort
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |