Note to Reviewer - CE Recall Study

December 17, 2015

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
FROM:	Brandon Kopp Research Psychologist Office of Survey Methods Research
SUBJECT:	Submission of Materials for the Global Questions Testing for the CE Recall Interview

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden hours for this study are 156.

If there are any questions regarding this project, please contact Brandon Kopp at

202-691-7514.

1. Introduction and Purpose

The Consumer Expenditure Survey (CE) is currently going through a redesign process. A stated goal of the redesign project is to reduce measurement error. An implied goal is that changes to the CE should reduce, or at least not increase, respondent burden. These goals are not always compatible with one another. One may wish to reduce burden by asking fewer questions that each cover a wider range of topic areas (referred to below as global questions), but this may make the respondent’s task of answering a question more difficult, leading to greater measurement error. This, counter-intuitively, may lead to an increase in overall burden if the respondent perceives the additional memory search and answer formulation requirements of the question as more burdensome.

BLS has conducted several studies to investigate the combination of expenditure questions. In these studies, participants were asked about their expenditures for several high-level expenditure categories (e.g., furniture) rather than a series of questions about the items that make up the category (e.g., sofas, living room chairs, living room tables, etc.) BLS determined the measurement error properties of their answers using cognitive interviewing techniques designed to elicit respondents’ comprehension of the questions and response accuracy.

For the categories initially tested, the studies have shown that the challenge is determining the proper level of aggregation for a given category or item that allows for a decrease in burden while minimizing any increase in measurement error. The current study will extend previous work to all categories covered by the CE Recall interview. If the testing shows that the implementation of global questions is successful (i.e., the number of questions are reduced and the interview time is shortened while measurement error is held constant or reduced), then more intensive, future testing would be considered in which measurement error is more thoroughly assessed through financial records collection or some other means.

2. Research Design

The purpose of this study is to develop and test “global” questions – expenditure questions to cover 38 item categories -- for the Recall interview. The objective is to be able to administer fewer questions than the current CE Quarterly Interview Survey (CEQ), while still collecting the required expenditure information as specified by recently revised Consumer Price Index (CPI) data requirements. The challenge will be to develop questions at a level of aggregation that balances burden and response accuracy by identifying wording that concisely defines the expenditure question in a standard way across a variety of survey respondents.

The study design is iterative. We will conduct testing in rounds of 10 participants (note that the last round will have 5 instead of 10 participants). In total, we will schedule 85 participants for interviews.

The expenditure questions that will be initially tested are listed in a table format in Appendix A. Because the overall goal of the study is to develop a final set of questions – by revising the level of aggregation, the wording, and the ordering – the specific questions covered with the test participants will evolve as problems surface and we identify improvements to test in subsequent testing rounds. For each item category currently proposed to be collected within the redesigned Recall interview, Appendix A shows the initial wording for testing. The initial wording of the questions was developed through a process of reviewing the current CEQ questions, the specific items to be captured within each item category, and findings from previous studies, then applying methodologist judgment about wording.

We will use cognitive interviewing techniques to gain an in-depth understanding of a respondent’s thought processes and reactions to the expenditure questions. The cognitive interviews for this study will be in-person. The following table shows the planned schedule and locations for the testing rounds. Each round requires steps to recruit and schedule participants, conduct the cognitive interviews, prepare data for analysis, analyze results, and submit an interim memo of results and recommendations (for changing the questions for the next round) to BLS.

Testing Round	Anticipated Date
Round 1 (Rockville)	January 26 – February 20, 2015
Round 2 (Baltimore)	February 23 – March 20, 2015
Round 3 (Winchester)	March 23 – April 10, 2015
Round 4 (Buffalo)	April 13 – May 1, 2015
Round 5 (Rockville)	May 4 – May 22, 2015
Round 6 (Baltimore)	May 25 – June 12, 2014
Round 7 (Winchester)	June 15 – July 3, 2015
Round 8 (Buffalo)	July 6 – July 24, 2015
Round 9 (Rockville) (n = 5)	July 27 – August 14, 2015

Westat staff who are experienced in cognitive testing methods will conduct a one-hour interview with each participant. The testing protocol is included in Appendix B. For each testing round, within each session, a participant will (1) report which expenditure questions they had expenditures for, and if applicable, the cost and month of the expenditure(s); (2) answer cognitive probes to elicit their understanding of the questions; and (3) complete an activity to judge which items they think do and do not belong within each expenditure question. An example of an item list for one expenditure question is provided in Appendix C. (A similar listing will be used for each question; these are not included for reasons of space.)

Given the time constraint of one interview hour per participant, we have developed a plan to cover all expenditure questions within the interview rounds. We have formed 8 “clusters” – groups of expenditure questions that will be tested together. These clusters are:

1. Apparel and personal items

2. Repairs, replacements, upkeep of home; home operations

3. Household linens, furnishings, items and supplies, and appliances

4. Trips and transportation

5. Recreation

6. Children and infants

7. Electronics/technology

8. Event-related

We anticipate testing the questions in each cluster in the testing rounds as shown in the following table. These cluster assignments are provided as a column in Appendix A to indicate planned question coverage per testing round.

Testing Round	Clusters
Round 1 (Rockville)	1, 3, 4, 7 (in-depth)
Round 2 (Baltimore)	2, 4, 5, 6, 8 (in-depth)
Round 3 (Winchester)	1, 3, 4, 7 (changes from Round 1)
Round 4 (Buffalo)	2, 4, 5, 6, 8 (changes from Round 2)
Round 5 (Rockville)	1 – 8 (changes from Rounds 3 and 4)
Round 6 (Baltimore)	1 – 8 (changes from Round 5)
Round 7 (Winchester)	1 – 8 (refinements)
Round 8 (Buffalo)	1 – 8 (refinements)
Round 9 (Rockville) (n = 5)	1 - 8 (final refinements)

As we progress through the testing rounds and revise the questions, we will move from spending time on in-depth probing about a subset of expenditure questions to covering more of the total number of expenditure questions.

3. Participants

Participants will be recruited by the following methods. Each method will provide a Westat 800 study phone number and an email address.

1. [For testing in Rockville] Contact potential participants from within Westat’s research volunteer database. We will email a short description about the study and invite interested persons to reply to the email or call our study 800 phone voicemail number (to leave a message with their name and phone number). The short study description is provided in Appendix D.

2. [For testing at any location] Place Craigslist ads with general text about a range of types of expenditures, or text targeting a specific desired mix of expenditures and/or demographic characteristics. An example of the ad

text is provided in Appendix E.

3. [For testing in Rockville] Post flyers in locations that have bulletin boards and announcements of opportunities and events, such as community centers, libraries, and stores. An example flyer is provided in Appendix F.

4. [For testing at any location] Place an ad in a local newspaper, using either general text about a range of types of expenditures, or text targeting a specific desired mix of expenditures and/or demographic characteristics. An example of this type of ad is also provided in Appendix E.

5. [For testing primarily at non-Rockville locations, but also if needed for Rockville] To supplement these methods and improve the ability to target eligible participants with a good mix of types of expenditures to report about, we will place pay-per click Facebook and Google Display Network ads to target only people within each the designated geographic regions with a desired mix of demographic characteristics. Examples of Facebook and Google ads are provided in Appendix E.

A Westat recruiter will monitor the emails and voicemail messages as they come in. She will call the individuals to complete the screening questions by phone or provide the link to the screener survey for those who prefer online screening. Appendix G provides the screening questions and examples of the ads that we anticipate placing. Note that although we will collect several forms of PII (name, phone number, and either email or physical address), the online screening survey will be hosted on a secure Westat server. The PII will be collected so that Westat can contact and schedule participants whose screener responses indicate they are eligible to participate.

From the database of screener responses, Westat will aim to select participants with varying levels of education, income, and gender, as well as varied household compositions. History of recent purchases made within the expenditure categories designated for each round of testing will be considered in selecting and scheduling participants at each of the locations.

4. Burden Hours

The total number of participants will be 85. The plan is to conduct each session within one hour.

Our expectation is that over the course of the testing rounds, we will need to recruit and screen more individuals than we will actually end up using, since many individuals will not have expenditures in categories of interest to us. We estimate that approximately five times the total number of interviews, or approximately 425 people, will need to be screened to find individuals who have an expenditure profile that will be relevant for the study. Based on an estimate of 10 minutes per person to complete the survey screener or complete the telephone screener (administered by a recruiter), we estimate the burden hours for recruitment to be 71 hours. The total burden for recruiting and conducting is estimated to be 156 hours.

5. Payment to Respondents

A $40 incentive will be provided to all participants participating in this study.

6. Data Confidentiality

Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national Consumer Expenditure Interview Survey. Participants will be given a consent form to read and sign (see Appendix H). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

Appendix A: Questions to be Tested

Item Categories and Initial Wording for Testing

This is a list of the main ways the categories are asked about. These letters are used in the cells of the table starting on the next page; the text for each letter A, B, C, D, E, F, or G leads into the question wording.

A (purchased)

Since the first of [the reference month], have you or any members of your household purchased …

B (purchased or rented)

Since the first of [the reference month], have you or any members of your household purchased or rented … [for your household or for someone outside of your household]

C (purchased, for own household or someone outside of household)

Since the first of [the reference month], have you or any members of your household purchased, either for members of your household or for someone outside your household, ...

D (had expenses for)

Since the first of [the reference month], have you or any members of your household had expenses for…

E (had expenses, either for own household or someone outside of household)

Since the first of [the reference month], have you or any members of your household had expenses, either for you or any members of your household or for someone outside your household, for… (note this is very long and awkward but corresponds to the wording in the CEQ that introduces a list of items:

Since the first of the reference month, have you or any members of your household had expenses for any of the following, either for you or any members of your household or for someone outside your household?

F (Trips -- spend anything)

How much did you or any members of your household spend for … other than what the package deal covered? 

[for all costs except for transportation]: What costs, including taxes, won't be reimbursed?

G (To be determined/Does not fit well into previous wording categories)

The following table shows, in Item Category order:

• Item Category Number

• Cluster Number

• Item Category Name

• Question Wording for testing

Cat #	Cluster #	Item Category	Question Wording (after intro text)
1 1, 2, 3	1	Apparel	C …Any suits for men, women, or children?
			C …Any sportcoats or tailored jackets for men, women, or children?
			C …Any uniforms or costumes for men, women, and children?
			C …Any type of dresses, for women or girls?
6	1	Jewelry and watches	C …Any jewelry, watches or parts for watches?
11	6	Child care	D …Any child care services inside or outside the home?
14	7	Internet services	D …Internet services away from home, such as internet cafes, and WiFi access fees in hotels, airports, and planes?
15	7	Information and information processing other than internet services	B … GPS devices, aside from those on cell phones or built into cars?
			B …Laptops and desktop computers, and computer equipment that you use with the computer?
			D …Any expenses for installations, repairs, or service contracts for a personal-use computer?
			B …Any portable memory storage for computers and other electronic devices?
			B …Digital book readers or tablets?
			B …Cell or landline phones and accessories? Do NOT include phone service fees.
			B … Office devices for non-business use, such as calculators, fax machines, and paper shredders?
37	6	School meals	D …any meals at school for preschool through high school age children?
38	8	Catered affairs	E …Any expenses for catered events, including rentals and purchases?
39	4	Food on out-of-town trips	F …meals, snacks, or NON-alcoholic beverages at restaurants, bars, or fast food places not counting what the package deal covered?
40	4	Food prepared by CU on out-of-town trips	F …food or NON-alcoholic beverages at grocery or convenience stores on this trip?
43	4	Beer, wine and other alcoholic beverages away from home	F… Did you purchase any alcoholic beverages on this trip?
48	8	Funeral expenses	E …Any expenses related to a death or maintenance of cemetery plots or vaults?
49	1	Other apparel services	E …Any expenses for rental, storage, or repair of clothing, watches, and jewelry?
51	6 4	Miscellaneous personal goods	C…Any other infant equipment? C …Any luggage, briefcases, or gym or tote bags?
54	4	Lodging away from home	F …Lodging, not counting what the package deal covered? Do not include expenses previously reported for vacation clubs.
62	2	Floor coverings	D…Flooring installation, repair, or replacement including wood, vinyl, tile, and wall-to-wall carpeting?
63	3	Household textiles	C …Any bathroom linens, shower curtains, and bath mats?
			C …Any bed linens, blankets, and pillows, for both adults and children?
			C …Any kitchen, dining room, or any other room's linens and fabric materials?
			C …Any window coverings, including curtains, drapes, blinds, and shades?
			C …Any slipcovers, decorative pillows or cushions?
			C …Any coverings for floors and stairs that are not permanent?
65	3	Unpriced maintenance and repair services	D… service contracts, maintenance, or repairs for other household appliances, such as washer, refrigerator, or range/oven?
			D…repairing, refinishing or reupholstering furniture, including the cost for fabric?
			D… service contracts, maintenance, or repairs for garbage disposal, range hood, or built-in dishwasher?
66	3	Furniture	C …Any living, family, or recreation room furniture?
			C … Any office furniture for home use, including filing and storage equipment?
			C …Any patio, porch, or outdoor furniture and covers?
			C …Any Infant furniture that is not transportable?
			C … Any dining room or kitchen furniture?
			C … Any other furniture not already mentioned, for any other areas in the house?
			C … Any bedroom furniture, including mattress and springs?
			G…Since the first of the reference month, have you or any members of your household rented or leased any furniture?
67	3	Appliances	B …Any larger appliances for the kitchen or laundry room, excluding dishwashers?
			B … Any small electric kitchen appliances?
			B …Any dishwasher that is portable, not built-in?
			B …Any window or wall unit air conditioners?
			B …1. Any air purifiers, water filters, carbon monoxide detectors, or smoke alarms? B …2. Any burglar alarm equipment, paid for separately from regular security alarm service fees?
			B … Any electric floor and carpet cleaning equipment?
			B …Any portable cooling or heating equipment?
68	3	Other household equipment	C …Any indoor lamps, ceiling or wall lighting, or ceiling fans?
			C …Any other household decorative items such as clocks, art, mirrors, and seasonal decorations?
			C …any items used to serve, consume, and store food and drink?
			C … Any nonelectric cookware?
			C … Any potted plants, fresh cut flowers, or flowers that can be worn or carried?
69	3	Tools, hardware, outdoor equipment and supplies	B …Any yard equipment and tools that are motorized, electric, or manual?
			B … Any items used for hanging or storage in the closet?
			A …Any termite or pest control treatment? Do NOT include professional service fees
71	2	Household operations	C … Any housekeeping, lawn, or tree care services? Do not include services covered by management or maintenance fees.
			D …for service contracts, maintenance, or repairs for any lawn or garden equipment?
			D …Any termite or pest control professional service fees?
81	7	Video and audio	B … Television sets, all types including those installed in vehicles? Include charges for installation or set-up.
			B…DVD and Blu-Ray players, DVRs, TV converter boxes, or video cameras?
			B … Stereos, radios, speakers, and sound components, including those installed in vehicles? Include charges for installation or set-up.
			B …Any small personal music players?
			B… Other sound or video equipment, including accessories?
			B …Any satellite dishes, receivers, or accessories, paid for separately from regular satellite service fees? Include charges for installation or set-up.
			D …Any expenses for service contracts, maintenance, or repairs for any audio or video electronic equipment or devices?
			Since the first of [the reference month], have you or any members of your household rented any video hardware?
			B…Any video game systems or accessories? Do NOT include actual video games.
82	5	Pets, pet products and services	C …Purchase of pets and their supplies, medical care, and other pet services?
83	5	Sporting goods	B …1. Any sports and recreation goods or equipment? B …2. Any ping pong, pool tables, or other similar recreation room items?
			B … Any health and exercise equipment?
			B …Any bicycles, including parts and equipment, both what you wear and use?
			B …Any equipment used for camping, hunting, and fishing? Include guns.
			B...Winter sports equipment, including both what you wear and use?
			B...Water sports equipment, including both what you wear and use?
			B …Any other sports or recreation equipment, including repairs?
			B …Any playground equipment, sand boxes, or inflatable pools?
84	5	Boats, RV's, and other recreational vehicles	F …Docking and landing fees for boats and planes?
			B…Any outboard motors?
			F …How much did you or any members of your household spend for any rented vehicles for recreational use, other than what the package deal covered?
86	5	Photography	E…Any professional photography services or products?
			E…Any photographic equipment, supplies, or processing?
87	5	Other recreational goods	B …Any musical instruments, including repairs?
			B...Any stamp or coin collecting?
			A…Any souvenirs or memorabilia?
			B…Any toys, games, and arts and crafts kits? Do not include electronic games.
88	5	Sewing	B …Any sewing machines, or sewing, knitting, or quilting materials and items?
89	5	Recreation services	F…Did you or any member of your household pay any fees to play sports, exercise, or rent any sports equipment not counting what the package deal covered?
			F…Entertainment or admissions
			E …1. Any membership at golf courses, country clubs, health and fitness centers, or pools? E …2. Any membership in any other social, civic, fraternal, or service clubs, such as Boy Scouts, Kiwanis, AARP, fraternities, and honor societies?
			D…Any recreational or instructional classes or lessons?
			D …Any fees for participating in sports such as golf, bowling, biking, hockey, football, or swimming?
			D …Any in-person or online dating services, such as matchmaking, eHarmony, Match.com, and speed dating?
			D…Instructional day camps or overnight camps?
			D… Vacation clubs or timeshares dues?
			D …Any shopping club memberships, including warehouse clubs like Sam's Club and discount memberships like Amazon Prime?
90	5	Movie, theater, sporting event, and other admissions	F… Did you or any members of your household spend money for: 1. Tour fees, admission fees, or tickets? 2. Other entertainment? ...not counting what the package deal covered? How much did you or any members of your household spend? / What costs for entertainment and admissions won't be reimbursed?
95	4	Leased and rented vehicles	G…How much did you or any members of your household spend on rental vehicles for getting around, not including fuel, other than what the package deal covered? Do not include any rental costs already collected.
101, 102	4	Motor fuel; Motor vehicle parts and equipment	G…While on the trip did you or any members of your household stop to buy any gasoline, diesel fuel, or any other fuels?
105	4	Motor vehicle fees	G…While on the trip did you or any members of your household have any parking fees? Do not include parking fees you were reimbursed for.
			G…While on the trip, did you or any members of your household have any tolls not already reported? Do not include tolls you were reimbursed for.
			G…Local tolls or electronic toll passes?
			G…Automobile service clubs such as AAA or services such as OnStar or LoJack?
107	4	Airline fares	F…How much did you or any members of your household spend on airfare, including baggage and change fees, other than what the package deal covered?
108	4	Other public transportation	F…How much did you or any members of your household spend for local transportation, such as shuttles, taxis, buses, subways or trains, and ferries? Do not include costs covered by your package deal.
			F…How much did you or any members of your household spend on bus or train fare to reach your destination, other than what the package deal covered?
			F…How much did you or any members of your household spend on cruise ship tickets other than what the package deal covers? Do not include incidental charges.
			G… How much did you or any members of your household spend on mass transportation services such as a bus, subway, mini-bus, or train? Include all commuter and carpool services. Do not include expenses covered by employer-provided transit subsidies.

Appendix B: Cognitive Testing Protocol

1. Introduction

Thank you for taking the time to help us out today. My name is …... [and this is my colleague ( ) who will be taking notes for us today. We also have ( ) from the Bureau of Labor Statistics who is observing the session.]

2. Purpose and Consent Procedure

First, I’ll give you a little background about what we’ll be doing. Westat is working on this project for the Bureau of Labor Statistics (BLS).

I will be asking you a series of questions from the Consumer Expenditure Survey, an ongoing survey that BLS conducts. The survey provides information about the buying habits of American consumers. This information is used for a lot of things, but its primary purpose is to help calculate inflation.

I’ll be asking you to answer the questions and share your reactions to them. The purpose of this session is to help us find out more about how people respond to these questions. Basically, we’re trying to find out what you think each question is asking about, and how hard it is to answer. We are not evaluating you, we are looking to improve the questions, so there are no right or wrong answers. All the information you give us will be kept completely confidential, and will be used to improve the survey questions.

Please read this consent form, and I’ll answer any questions you may have. [ANSWER QUESTIONS, IF ANY.] Please sign the form.

[START RECORDER AND GET ORAL PERMISSION TO RECORD; WHICH RECRUITER EXPLAINED AT TIME OF SCHEDULING CALL.] With your permission, I would like to record the interview so that I have an accurate account of what you say in case I miss anything while taking notes. Do you give your permission to participate in this interview and to be recorded?

Do you have any [other] questions before we begin?

[INTERVIEWER: GO THROUGH THE FOLLOWING SECTIONS 3 – 6 BASED ON THE CLUSTERS OF EXPENDITURE QUESTIONS DESIGNATED FOR TESTING IN THE PARTICULAR TESTING ROUND]

3. Administration of Expenditure Questions and Follow-ups (to assess reporting accuracy)

• [ASK EVERY expenditure question designated for that testing round to determine applicable expenditures]:

Since the first of [the reference month 3 months prior], have you or any members of your household purchased ________?

• [FOR ANY “YES ANSWER]; ASK:

• How much did you [and your household] spend on ________ in [the month 3 months prior]?

• How much did you [and your household] spend on ________ in [the month 2 months prior]?

• How much did you [and your household] spend on ________ in [the month 1 month prior]?

4.  Administration of Follow-up Questions for Applicable Expenditure Questions (to assess reporting accuracy and estimation accuracy)

• [For any expenditure questions participant did have EXPENDITURE(S) for], ASK:

• What specific item(s) did you [or others in your household] purchase?

• What was the cost of each of those items?

• In what month did you [or others in your household] purchase [that/those] item(s)?

• Was there anything you thought of, but decided not to report for [expenditure question]? What made you decide not to?

5. Recall Activity (to assess comprehension)

• [For any expenditure questions the participant did not report expenditure(s) for], ask:

• Have you or anyone in your household ever bought any [expenditure question wording]?

• [For any expenditure questions participant did have EXPENDITURE(S) for], ASK:

• Other than what you already mentioned that you bought since [the first of the reference month 3 months prior], what are some other examples of items you or anyone in your household has ever bought that belong with [expenditure question wording]?

• [For aLL expenditure questions], ASK:

• Can you name for me 5 items you think of when you hear [expenditure question wording]?

• [If participant names fewer than 5 items], ask: What else?

• [If participant still stops short of 5 items and cannot name more, move on to the next question.]

6. Inclusion/Exclusion Judgment Activity (to assess comprehension) and Recheck of Expenditures (to recheck and assess reporting accuracy and estimation accuracy)

• [For all expenditure questions], ask:

• Please review this list and use an X to mark the column for items that you think belong for [wording of expenditure question], which do not belong, and which you are not sure of.

• [For all expenditure questions], ask:

• Did working with these items [refer to list] remind you of any other items you’ve purchased since [the first of the reference month 3 months prior] that you [didn’t previously mention when you gave costs / think of]?

[For any expenditure question participant says “Yes”], ask:

• What items did you or others in your household purchase?

• What was the cost of that item?

• In what month did you or others in your household purchase that item?

• Can you say why you didn’t…

  • [If answered “No” for expenditure questions] …think of this sooner?

OR

• [If answered “Yes” for expenditure questions] …include this when you answered about the other items you or others purchased for [expenditure question wording]?

7. Closing

We appreciate your help today. Thank you for your time. [PROVIDE THE $40 INCENTIVE.]

Appendix C: Item List Example

(“Lures” are marked with shading; lists will be shown with no shading)

Small electric kitchen appliances
Please place an "x" in the column to indicate which items you think belong and do not belong in this category.
	Belongs	Does not belong	Not sure
Bread maker
Burger cooker
Cake rack
Electric corn popper
Donut maker
Ice bucket
Wall oven
Toaster oven
Mini food processor
Freezer
Electric can opener
Electric egg poacher
Spatula
Hand-held mixer
Electric frying pan
Electric juicer
Gas stove
Electric skillet
Chopping board
Measuring spoons
Deep fryer
Rolling pin
Whisk
Electric fondue set
Meat thermometer
Crockpot
Dish dryer rack
Coffee grinder
Kitchen scissors
Salad spinner

Appendix D: Example of Short Study Description

(For email to members of Westat’s Volunteer Database)

The Bureau of Labor Statistics is exploring ways to better measure how Americans spend their money. Participants in this study will work with some questions from the Consumer Expenditure Survey, an ongoing survey that provides information about the buying habits of American consumers.

To determine if you are eligible to participate in the study, we will ask you to answer a few background questions about yourself. You can do this either on the phone or by completing a short online survey. This step will take about 10 minutes.

If you are eligible for the study, we will ask you to participate in a 60-minute interview. During the interview, we will go through questions from the survey, get your reactions to them, and ask you about various types of purchases. Your input will help to identify ways to improve the survey questions.

If you are interested in participating, please either:

1. Send an email to [email protected]; with “Consumer Expenditure Survey” in the subject line, with your name, telephone number, and the best time to reach you,

or

2. Call the voicemail line at 1-800-XXX-XXXX, mention “Consumer Expenditure Survey” and leave your name, telephone number, and the best time to reach you.

Appendix E: Example Text for Ads

Craigslist and Newspaper Ad Example

Survey Research- Receive $40

Date: [FILL]

Westat, a social science research company, needs adults age 18 and over to participate in an interview about household spending. The interview will take place in [Name of Venue]. The interview will last about 60 minutes, and you will receive $40. If you are interested, please either:

1. Send an email to [email protected] with “Consumer Expenditure Survey” in the subject line, and your name, telephone number, and the best time to reach you, or

2. Call the voicemail line at 1-800-XXX-XXXX, mention “Consumer Expenditure Survey,” and leave your name, telephone number, and the best time to reach you.

If we determine that you are eligible for the study, we will call you to schedule an interview during [dates at that venue]. If you have questions about the study, please call 1-800-XXX-XXXX.

WESTAT
EOE

Facebook Ad Examples

Note that ads will vary in:

• Location mentioned – these use Rockville for illustrative purposes

• Specific wording – these use general text which will be revised to target demographic subgroups and/or expenditure categories

Google Ad Examples

Note that ads will vary in:

• Location mentioned – these use Rockville for illustrative purposes

• Specific wording – these use general text which will be revised to target demographic subgroups and/or expenditure categories

Appendix F: Flyer Example

interested in earning $40?

PARTICIPANTS NEEDED FOR IN-PERSON INTERVIEW

Westat, a research organization located in Rockville, Maryland, is seeking adults to take part in a 60-minute in-person interview to review survey questions for a large national survey. Eligible participants must be age 18 or over [and] [other conditions specific to expenditure categories or demographic characteristics may be targeted/adjusted before each testing round].

If you meet these conditions, either:

1. Send an email to [email protected]; with “Consumer Expenditure Survey” in the subject line, with your name, telephone number, and the best time to reach you,

or

2. Call the voicemail line at 1-800-XXX-XXXX, mention “Consumer Expenditure Survey” and leave your name, telephone number, and the best time to reach you.

WESTAT

EOE

Appendix G: Screening Questions

Screener

(in format for telephone screening; web survey screener will have same items)

Thank you for your interest in our study. We need people to participate in an interview about things they have spent money on over the past 3 months. We are going to ask you some questions about yourself since we need to make sure that we include people with varying backgrounds. This will only take about 10 minutes.

1. What is your age?

|___|___|

AGE

IF AGE 17 OR YOUNGER…………………………...(GO TO END STATEMENT ABOUT INELIGIBLITY DUE TO AGE)

AGES 18+ (GO TO 2)

NO NEED TO ASK IF OBVIOUS:

2. Are you male or female?

Male………………………………………………………1

Female…………………………………………………….2

3. Are you of Hispanic, Latino, or Spanish origin?

YES………………………………………………………1

NO………………………………………………………2

4. What is your race?  Please select one or more. 

White……………………………………………… 1

Black or African American…………………………….. 2

Asian…………………………………………………. 3

Native Hawaiian or other Pacific Islander… ……………... 4

American Indian or Alaska Native ……………………….. 5

4. What is the highest level of education you have completed?

1st - 8th grade, some high school, but not a diploma……….1

12 grade/high school diploma or GED………..…………… 2

Some college, Associate’s degree ………….………….……3

Bachelor’s degree …………………………………….….….4

Master’s degree or higher…………………………………...5

4. What is your marital status?

Single …………………………………….……….1

Married …………………………………….…….2

Divorced ……………………………………..…...3

Separated ………………………………………….4

Widowed…………………………………….…….5

4. Do you live alone?

YES…………………………………1 (GO TO Question 9)

NO…………………………………………………………2

4. In your household…

a.…How many other adults are there, besides you?

_______ (number)

b. …How many infants (2 years old or under) are there?

_______ (number)

c. …How many other children (2 to 18 years old) are there?

_______ (number)

4. In the last 3 months, have you [or anyone in your household] purchased...

[List of expenditure categories with “common wording” and lures goes here, with Yes/No/Not sure answer choices for each]

4. What is your total household income – would you say above $50,000 or below $50,000?

Below $50,000…………………………………….……….1

Above $50,000 …………………………………..….…….2

Rather not say…………………………………….……….3

4. What is your name? ___________________________

4. What is your e-mail address?

___________________________

[IF NO EMAIL ADDRESS, COLLECT REGULAR ADRESS SO THAT WE CAN SEND INSTRUCTIONS ON TESTING LOCATION]

12 a. What is your address? ____________________________

____________________________

4. What is your phone number?

___________________________

[IF PHONE SCREENING] – For those being screened over the phone, this may happen immediately. For those who use the web form, recruiter will call those determined to be eligible and attempt to schedule an interview.]

We will be conducting interviews in [TESTING LOCATION, NAME AND ADDRESS OF FACILITY] during the days of [FROM WHEN TO WHEN]. We plan to audio record the interviews; is it okay with you to be audio-recorded? Would you be available to meet at that location for a one-hour time slot on one of those days?

YES………………………………………………………1

NO………………………………………………………2 [Thank you for your interest. [IF EARLY ROUND] We will be conducting interviews at that location again within the next few months. Would you be available to meet at that location for a one-hour time slot in the future?

YES………………………………………………………1

NO………………………………………………………2

Where did you see our ad for volunteers?

_____________________________________________________________

______________________________________________________________

______________________________________________________________

Appendix H: Consent Form

CONSENT FORM

Westat is conducting research on behalf of the Bureau of Labor Statistics (BLS). This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

Westat, the BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio taped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 60 minutes to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires 2-28-2015.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: 02-28-2015

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

28