CEX Card Sort
Final OMB PackageCard Sort 1-10-12.doc
Cognitive and Psychological Research
CEX Card Sort
OMB: 1220-0141
January 10, 2012
NOTE TO THE REVIEWER OF: |
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
|
FROM: |
Jennifer Edgar Research Psychologist Office of Survey Methods Research
|
SUBJECT: |
Submission of Materials for the Consumer Expenditure Survey Card Sort Task |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The estimated respondent burden for the card sort is 43 hours and 20
minutes.
If there are any questions regarding this project, please contact Jennifer Edgar at
202-691-7528.
Introduction and Purpose
The Consumer Expenditure Quarterly Interview Survey (CEQ) provides data on the buying habits of American consumers, including both expenditures and income for the consumer unit (i.e., families and single consumers). CEQ asks respondents questions about a wide range of expenditures, from monthly utility bills to irregular entertainment expenses. The U.S. Census Bureau administers the survey for the Bureau of Labor Statistics.
The CEQ is both a high profile and important survey because it is the only Federal survey to provide information on the complete range of consumers' expenditures and incomes, as well as the characteristics of those consumers. The data from the CEQ are put to a variety of uses, from shaping the economic policy of the nation to businesses evaluations of potential markets, to academics studying the spending habits of Americans. A number of the critical economic indicators use the data from the CEQ, for example, the Consumer Price Index.
The current expenditures in the CEQ are grouped into more than 20 categories (e.g., household appliances, clothing) which were defined more than 30 years ago as part of the initial survey development. New expenditure items (e.g., downloaded videos, iPods) have been put into the existing categories, resulting in groupings which may not be as meaningful or adequately reflect the way respondents think about these expenditures.
The Consumer Expenditure Survey is currently undergoing a redesign that aims to reduce measurement error, improve the efficiency of data collection methods, and reduce respondent burden. Part of this multi-front, broad effort is to verify the way respondents think about and group their expenditures. The proposed card sort task is designed to allow respondents to chose their own categories for expenditures and discuss the reasons for their choices. This research will provide a measure of the degree to which the current expenditure groupings reflect the way respondents think about their expenditures, and provide input into the redesign process.
The study objectives are to:
Identify meaningful groups of expenditure items;
Identify the minimum number of groups meaningful to participants; and
Identify the vocabulary participants use to describe expenditure groupings.
This work will be conducted by Westat, Inc. under a task order contract to BLS. Westat will conduct all of the recruiting, venue selection, card sorting sessions, preparation of the SynCaps cards, and analysis.
2. Overview of Research Design
To examine the way respondents conceptualize the expenditures in the CEQ, a card sorting task will be conducted. The card sort will allow respondents to create and name their own expenditure categories, thus giving us insight into the way respondents think about these expenditures. Respondents will be asked to think aloud as they sort the cards, and they will be debriefed after they have completely sorted their cards. We will analyze the cards using the SynCcaps software to determine which expenditures tend to be sorted into the same categories. The analysis will yield diagrams that display the categories that the participants created. We will record the labels that the respondents gave these categories and the vocabulary that they used to describe the categories.
3. Recruitment
Participants will be recruited from the Westat database of study subjects and from announcements placed in the target venues. Potential volunteers are asked to call into a toll-free number and leave their contact information. They are also given an email address in case they prefer that mode of contact. A Westat recruiter will call the volunteers who have expressed interest in participating and administer a very short screener. Participants with the most desirable mix of demographics and socio-economic characteristics will be scheduled for a session.
Participants will come to one of six testing sites. The target testing sites are:
Rockville, MD,
Baltimore, MD,
Washington, DC,
Buffalo, NY,
Rural North Carolina, and
Rural Maryland or Rural Virginia
These locations were chosen for their ability to provide a geographic spread and for operational convenience. The sites in rural North Carolina that Westat has used previously are in Pitt and Greene Counties and we expect to place these sessions in either one of those counties. Rural Maryland could be the Eastern Shore of Maryland or Western Maryland. Rural Virginia would be the area around Brunswick, VA or even Harpers Ferry, WV. The exact location will depend on the availability of appropriate venues and the success of the recruiting effort.
3. The Card Sorting Task
We will use SynCaps to create the 101 cards that the participants will sort. Each card will contain the name of an expenditure collected by the CEQ (Appendix A) as well as a unique bar code to identify it. Two sample cards from the list of expenditures are pictured below.
Each participant, regardless of location, will be seated at a large table in the testing room. The moderator, also in the testing room, will explain the research procedure to the participant and ask the participant to sign the BLS’s consent form. Informed consent covers both participation and any recording (audio or video). The moderator will give the participant a set of pre-shuffled cards to sort. The card sort protocol can be found in Appendix B.
Participants will be asked to sort cards into groups. There are no criteria on the number of groups that respondents create or on the number of cards in each group, but participants will be instructed to create as few groups as possible. Participants will also be invited to create sub-groups within each main grouping. Each sub-group must have at least two cards in order to qualify as a sub-group. Participants will be given blank “group label” cards and blank “sub-group label” cards. They will then be asked to label any groups and sub-groups using whatever labels they wish to come up with by writing those labels on the cards and placing these cards on top of each pile of sorted cards.
The moderator will then debrief the participant to gain greater insight into the participant’s rationale for creating the particular groups. This debriefing will give the interviewer the chance to inquire about anything unusual or puzzling about the participant’s card groups or subgroups.
4. Data Analysis
With SynCaps, data entry is straightforward. A research assistant will type in the name that a participant gave to a category or subcategory. Then the research assistant will scan the bar codes on the cards in that category or subcategory.
After all the cards from all the participants are scanned, SynCaps software will analyze the sorts and provide the following output:
A dendrogram which graphically displays the consensus groups and the degree to which the cards in each group tended to be sorted together.
A pair map that displays the extent that each card was sorted into the same group as each other card.
An item by group matrix.
The lengths of the branches in the dendrogram suggest the strength of the associations among the various cards. SynCaps can also employ color to indicate the optimum groupings, based upon the desired strength of the associations among the cards within each grouping.
We will conduct these analyses using the entire dataset. Then we will repeat the analyses, for each of the separate audience groups (suburban, urban and rural settings; high and low education; male and female) and will review at the dendrograms of these separate audience groups to see if any clear differences appear between those in different groups.
The interviewer will take notes during each session and if necessary, will review the video recordings of the sessions to study the participants’ statements during the think-aloud sessions and during the post-task debriefings. We will also note the names that the participants gave to the categories.
The session notes and recordings will
be treated as qualitative data and analyzed using standard
qualitative methods. The qualitative analyst will look for the
vocabulary that respondents use to talk about each category of
expenditures. We will probe about any discrepancies between the
vocabulary used by the survey and the vocabulary used by the
participants and report these as part of the qualitative findings.
The qualitative analysis will examine the following types of findings:
Comments that show overlapping groups and the reasons participants think the expenditures can be in more than one group;
Cards that participants have difficulty classifying;
Alternative names of groupings that were considered and the reasons for considering them; and
All grouping alternatives suggested across participants, showing overlaps and variations.
Suggestions for reducing the current number of groups based on the strength of the associations in the dendograms.
5. Participants
The card sort will be conducted with 40 participants in different geographic regions with a reasonable distribution across demographics (age, gender, race, ethnicity, education), and urbanicity.
Respondents will be offered a $40 incentive.
To ensure some regional distribution and a reasonable distribution across urbanicty, card sorts will be conducted in six locations:
-
Location
Type
N
Rockville, MD
Suburban
6
North Carolina
Rural
7
Buffalo
Suburban
7
Baltimore
Urban
6
Wash, DC
Urban
7
Rural MD or Rural VA
Rural
7
Total
40
The recruitment advertisement is found in Appendix C and the recruitment screener is in Appendix D.
6. Burden Hours
Our goal is to obtain no more than 40 participants across all of the venues. We anticipate that each session will last no longer than one hour, for 40 burden hours for the sorting task and 3 hour and 20 minutes for the screening. The screener consists of 6 questions. A rule of thumb for telephone surveys is that 4 questions usually take 1 minute to deliver. To recruit 40 participants across 6 venues, we expect to screen about 100 individuals. To allow for 6 questions and some other conversation, we have assumed 2 minutes per screener, which gives 3 hours and 20 minutes for the entire screening activity. This gives total burden hours of 43 hours and 20 minutes for both the card sort and the screening of participants.
7. Data Confidentiality
No pledge of confidentiality will be given. Participants will be informed of the voluntary nature of the study. Participants will also be informed that the study will be used to improve the design of a national consumer expenditure interview survey. Information related to this study will not be released to the public in any way that would allow identification of individuals.
An informed consent authored by Westat’s IRB will be delivered verbally to all participants.
Appendix A: List of Expenditures
watches & Jewelry
books and supplies for school
college and technical school tuition and fees
elementary and high school tuition and fees
childcare, daycare and nursery school
postage and package delivery services
mobile and landline telephone services
computer hardware and software
internet service to your home or at an internet café
internet entertainment fees (such as online games)
telephones
calculators
cigarettes and tobacco products
hair products
toiletries all combined (include dental, shaving, deodorant, sunscreen, sanitary foot products)
electric personal care products
cosmetics, perfume, bath products and nail polish
haircuts
legal services
funeral expenses
laundry and dry cleaning services
clothing alterations and repair including shoe repair
watch and jewelry repair
banking services
tax preparation and accounting services
stationery, giftwrap and office supplies
luggage
equipment for infants (such as bouncy chairs, high chairs, play mats, and baby bottles)
rental of primary residence
lodging while at school
rental and homeowners insurance
home heating oil
propane, kerosene, and firewood
natural gas
electric utility
water utility
garbage and trash collection
floor coverings including carpets and hardwood floor
window coverings
linens for bedroom, bathroom, kitchen and dining room
indoor furniture including bedroom, living room, kitchen, dining room and infant furniture
outdoor furniture
refrigerators and freezers
washers and dryers
kitchen appliance (range, cooktops, microwave ovens)
lamps and lighting fixtures
household decorative items including clocks, artwork and knick knacks
indoor plants and fresh-cut flowers
dishes and flatware
nonelectric cookware and tableware
paint, wallpaper tools and supplies
power tools
non-powered hand tools
lawn and garden and outdoor equipment and grills
household cleaning and other domestic products
gardening and lawn care services
moving and storage expenses
appliance repair
reupholstery of furniture
indoor home maintenance and repair services
medical, dental, and eye care services
nursing homes and adult day services
home care services for the elderly or disabled
Medicare and health insurance
prescription and nonprescription drugs
medical equipment and supplies
televisions
cable and satellite television and radio service
audio and video media (including DVDs and downloaded files)
pet food, supplies and accessories
veterinarian services
motorized sports vehicles as well as non-motorized vehicles such as bicycles
sports equipment
hunting, fishing, and camping equipment
cameras and camcorders
photographer's services
toys, games, hobbies and playground equipment
videogame hardware, software and accessories
sewing machines, fabric and supplies
music instruments and accessories
club membership for participant sports and group exercises
fees for lessons & instructions
admission to movies, theaters, and concerts and sporting events
newspapers, magazines and books
purchase of new and used vehicles
rental of cars and trucks
motor fuel such as gasoline
tires for motor vehicles
parts and equipment for motor vehicles
motor vehicle maintenance and repair
motor vehicle insurance
motor vehicle fees such as state and local registration and license
parking fees and tolls
automotive service clubs
global positioning subscription services such as OnStar
airline fares
bus and train fares
ship fares
taxi fares
car and van pools
fares for mass transit within a city
Appendix B: Card Sort Protocol
Introduction
[Before the sort begins, make sure that cards are shuffled, and that there are enough group cards and subgroup cards available].
Thank you for agreeing to help us out today.
My name is ______ [introduce others observing, taking notes, etc.] I work at Westat, which is a social science research company located in Rockville MD. We are conducting this study for the Bureau of Labor Statistics.
Today, we are looking at the categories used by the Bureau of Labor Statistics, or BLS, to classify products and services that people pay for. We’re going to ask you to sort a set of cards, each card lists a product or service the BLS is interested in. Be assured that this is not a test of you or your skills in any way and there are no right or wrong answers here.
We will look at how you sort cards and get your feedback on the process. This study will help the BLS improve their surveys.
Informed Consent
There are some points I need to cover about your participation.
Purpose and Task: We are inviting you to participate in a research activity. The research involves sorting cards of expenditure items into categories. The information from this study will help BLS design more effective questionnaires. There are no right or wrong sorts – we are interested in how you think the cards should best be sorted.
Sponsorship: Westat is conducting this project on behalf of, and funded by, the Bureau of Labor Statistics (BLS). The OMB control number for this study is 1220-0141 and expires on February 29, 2012.
Voluntary Nature: Your participation in this research project is voluntary, and you have the right to stop at any time or to refuse to answer any question. The session will take approximately 60 minutes.
Risks: We would like to record the interview. Sometimes it is helpful to review a portion of a recording as we make recommendations for improving the survey. If the recording is reviewed later, it will only be by a few Westat staff and possibly the client, BLS.
You will never be identified by name. The things you say may be put in a written summary of this discussion, but there will be no way to identify who said what, and your name will not be used anywhere.
The only cost to you is the time and effort to answer questions. You may skip any question that you do not want to answer, both in the questionnaire and in the discussion afterwards. The recordings will be destroyed within 6 weeks of the end of the study.
Benefits: There are no direct benefits to you for participating in this study. However, you will be helping with an important research project.
Questions: If you have questions about the project, you may call the Task Manager, Pat Dean Brick, at 800-937-8281, Ext. 4196. For questions about your rights and welfare as human subjects in this study, you may call the Institutional Review Board at Westat at 301-610-8828.
Incentive: You will be paid $40 cash for completing the session.
Instructions
Here’s how the process will work. In front of you is a stack of cards. These cards represent categories of products and services that people pay for. Your task is to sort the cards into groups that make sense to you. What we’re interested in is seeing how you group these cards and why they belong in the same group. Here are some specific instructions:
Put all the cards into groups that seem to make the most sense to you.
If a word or term written on the card is unclear, please let us know as you work.
Once your groups are established, use these [group cards] to give each group a name that makes sense to you.
You are allowed to make sub-groups within a bigger group if you feel that’s appropriate. You can have as many sub-groups as you’d like but each sub-group must contain at least two cards. After creating a sub-group, use these [sub-group] cards to give the sub-group a name that makes sense to you.
Groups can include as many or as few cards as you wish.
Please create as few groups as possible.
If you can’t decide which of two groups you want to put a card into, please tell us and then put the card into whichever group offers the better fit, however slight.
Throughout this exercise please share with us your thought process and logic for the decisions you make. Along the way, I may also ask you some additional questions. Meanwhile, you can feel free to ask questions during the exercise if you feel the need. I can’t guarantee that I can answer them immediately, but I’ll do my best to answer them when you’re finished.
Do you have any questions before we begin?
Card Sort Activity
Participants will now sort the cards.
Participants may be prompted to express what they are thinking if they look confused or frustrated or pause the sort for a length of time. However, it is important to only have participants do what comes naturally to them, as forcing them to talk more than is natural for them would hinder their sorting ability.
Debriefing
How comfortable did you feel with your ability to sort cards overall?
Could you very briefly describe your general approach to sorting your cards?
Which cards were easiest to group? Why?
Which cards were hardest to work with? Why?
Which was the most challenging group for you? Why?
Did you find yourself changing your general strategy at any point? Why?
How easy or difficult was it to come up with the labels for your groups?
Why was it [easy or difficult]?
Thank you all very much for your time.
Appendix C: Advertisements for Participants
Survey Research- Receive $40
Reply to: (see
message body)
Date: 2012-1-XX
Westat, a social science research company, needs
individuals to participate in an interview about household
expenditures. Adults, ages 18 and over, men and women, are encouraged
to participate. We encourage as diverse a group of individual as
possible to participate. The interview will take place in [Name of
Venue]. The interview will last about 60 minutes. If you are
interested, send an email to [email protected] or call
1-800-XXX-XXXX and include your name, telephone number, email
address and the best time to reach you. This is a Federally-sponsored
study
WESTAT
EOE
Appendix D: Screener
NAME______________________________
PHONE_____________________________
RESPONDENT Number ____________________
Thank you for your interest in our study. We need people with diverse backgrounds to participate in a card sorting task. We are going to ask you some questions about yourself so we can make sure that people with varying backgrounds are represented in the task. This will only take a few minutes.
1. What is your age?
|___|___|
AGE
IF AGE 17 OR YOUNGER…………………………...(END STATEMENT)
AGES 18+ (GO TO 2)
NO NEED TO ASK IF OBVIOUS:
2. Are you male or female?
Male………………………………………………………1
Female…………………………………………………….2
3. Are you of Hispanic, Latino or Spanish origin?
YES………………………………………………………1
NO……………………………………………… ………2
What is your race? Please select one or more.
White…………………………………………………….. 1
Black or African American……………………………. ….2
Asian…………………………………………………. 3
Native Hawaiian or other Pacific Islander… ……………... 4
American Indian or Alaska Native ……………………….. 5
What is the highest level of education you have completed?
8th grade, some high school, but not a diploma……………….1
12 grade/high school diploma or GED…………………………2
Some college, Associate’s degree, Bachelor’s degree or higher..3
Where did you see our ad for volunteers?
_____________________________________________________________
______________________________________________________________
______________________________________________________________
IF ELIGIBLE: It appears you are eligible for our study. We’d like to schedule you at XXXX. Let me read you the days and times I have available, and you can tell me what would be best for you. This will take about 1 hour of your time and we will pay you $40 cash. May I please have your full name and address? (We need your address so that we can send you directions to venue. It will include a map that shows you exactly where the room is).
Name:
Address:
E-mail address:
I will send the directions out to you shortly. If you have to cancel your interview, please call back so that we can schedule someone in your place, OK? You can reach me at: [Recruiter’s Phone Number].
If you have any friends, relatives or acquaintances who would be interested in participating, please give them our contact information: [email protected] or call 1-800-XXX_XXXX.
IF NOT ELIGIBLE: Thank you so much for answering our questions. We will get back to you once we figure out how many people we will actually be needing for this study. Please provide your contact information so we can get back to you.
Name:
Address:
E-mail address:
Could we also add your name to our respondent database so we can call you in the future for other studies?
Yes_________
No_________
If you have any friends, relatives or acquaintances who would be interested in participating, please give them our contact information: [email protected] or call 1-800-XXX-XXXX.
IF NOT AGE 18+: It appears that you are probably not eligible for our study. Thank you very much for your interest and willingness to help out.
| File Type | application/msword |
| File Title | October 14, 2003 |
| Author | cory |
| Last Modified By | kincaid_n |
| File Modified | 2012-01-10 |
| File Created | 2012-01-10 |
© 2024 OMB.report | Privacy Policy