January 27, 2012
NOTE TO THE REVIEWER OF: |
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
|
FROM: |
Brandon Kopp Research Psychologist Office of Survey Methods Research
|
SUBJECT: |
Submission of Materials for the Pretesting of proposed changes for the CE Quarterly 2013 |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The total estimated respondent burden hours for this study is 40 hours.
If there are any questions regarding this project, please contact Brandon Kopp at
202-691-7514.
Introduction and Purpose
The Bureau of Labor Statistics has completed several projects with the goal of improving the Consumer Expenditure Quarterly Interview Survey (CEQ). One of the main goals of these improvements is to keep the market basket of goods and services, which provides item weights for the Consumer Price Index (CPI), current. Given the rapid changes in the marketplace, changes need to be made to the CEQ to ensure that the data accurately reflect the state of consumer buying habits. Based on input from CPI commodity analysts and economists, a few areas of the survey were identified as needing attention to maintain the most up-to-date market basket possible. In a parallel effort to improve CEQ data quality, economists from the Branch of Production and Control (P&C) identified changes that need to be addressed for reasons of respondent burden, outdated questions, and general instrument issues.
In addition to these recommendations, independent teams within the Bureau of Labor Statistics (BLS) evaluated products and services which are becoming more common and made recommendations for changes to the questionnaire to improve reporting for these areas.
2. Research Design
The purpose of this study is to ensure that proposed changes to the 2013 CE Quarterly instrument are clear and understandable to respondents, and are interpreted as intended. To accomplish this goal, question variations will be tested using cognitive interviewing. The questions presented to each participant will vary, with each set covering multiple topics. Participants will be assigned topics based on their experience with and knowledge of the subject matter. A complete list of the changes to be tested is included in Appendix A. The changes are currently in draft form and will likely be revised and retested before being finalized. A final list of question wordings will be sent to OMB with the final report.
As noted, cognitive interviewing will be used to gain insight into the topics addressed in the changes. Cognitive interviews provide an in-depth understanding of the respondent’s thought processes and reactions to the questions. These interviews will be done one-on-one in the Office of Survey Methods Research (OSMR) laboratory, and will be audiotaped. Interviews will be conducted by staff from the OSMR who are experienced in conducting these types of interviews. The testing protocol is included in Appendix B.
The study will follow an iterative design, with question wording being modified as problems and improvements are identified. After the initial draft of question wording, each question will be tested using cognitive interviews, and revised, if necessary. If the question has been revised, another round of cognitive interviewing will be conducted. If further revisions are deemed necessary, additional rounds of cognitive interviewing will be done until the results indicate no major problems remain.
3. Participants
Participants will be recruited from the OSMR participant database. Screening questions will aid recruitment of participants with desired target knowledge. Efforts will be made to select participants with varying levels of education, income, and occupation, based on self-reported information provided during the initial recruitment process. Experience with the topic being studied will be considered as part of the scheduling process and this information will be used to assign topics to participants.
4. Burden Hours
The number of participants is dependent upon the results and the point at which diminishing returns is reached and no further information is being gained from additional subjects. Given that, our goal is to obtain no more than 40 participants in the OSMR lab. We anticipate that each session will average 45 minutes and last no longer than one hour, for a total of 40 burden hours. Participants will be compensated $40 for their time.
5. Data Confidentiality
Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national Consumer Expenditure interview survey. Participants will be given a consent form to read and sign (See Appendix C). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.
Appendix A: List of Changes to be Tested
Each of the following proposed changes to the CEQ will be tested using cognitive interviews. The goal is to identify the associations participants have with various terms, how they interpret question meanings, and to identify the most effective wording for questions and response choices. Below is a list of draft changes to be tested.
Large Changes
Add question about whether respondent has received the Earned Income Tax Credit
Allow flexible reference period for school meals
Revise asset and liability questions
Combine Items
Combine “golf courses” and “country clubs” with other sports club entry fees AND combine social organizations with civic, service, and fraternal organizations
Combine AAA and OnStar into single question
Combine bank services and safe deposit boxes into a single question
Combine dishes and glasses into single question
Combine Internet service item codes into single question AND Reword Internet expenditure question to cover Internet at hotels and other locations
Combine PDAs, digital book readers, and tablets into single question
Combine phones and accessories with answering machines into single question
Combine prepaid cellular and regular cellular into single question
Combine residential phone service and Voice-over-Internet Protocol (VoIP) into single question
Combine rental and purchase questions for medical related items
Combine rental of sports equipment and fees for playing sports into single question
Combine smoke detectors and other household appliances into single question
Combine sweaters, shirts, blouses, and tops into single question
Combine uncommon, similar items in the auto repair section
Combine water and sewer into single question
Combine childcare codes in Misc. expenses and reword “Nursery school” to “preschool” in educational expenses
Combine response options for question about who pays health insurance premiums
Reword current questions
Reword camping and hunting and fishing equipment to include “including all guns”
Reword “garage rental” to “parking garage”
Reword “trash and garbage collection” to “garbage and recycling collection”
Reword audio/video items to include those in vehicles
Reword Medical Expenditure Reimbursement Section Intro to exclude other types of accounts
Reword Medical Expenditures Section Intro to include mention of “co-pays” and “out-of-pocket expenses”
Reword other vehicles to include “including boats and planes”
Replace word “convalescent” with “rehabilitation” and “rehabilitative”
Replace word “paperbacks” with “digital books” in OTHRBKRF
Appendix B: Cognitive Testing Protocol
The protocol includes the question that will be tested, additional questions that provide context for the tested question, and debriefing questions. The interviewer may ask unscripted follow-up questions if a participant’s answer suggests additional, pertinent information can be gained.
Large Changes
Add question about whether respondent has received the Earned Income Tax Credit
During the last 12 months, did ^YOU_ANYMEM claim an Earned Income Tax Credit on your federal income tax return? 1. Yes 2. No |
|
Allow flexible reference period for school meals
Since the first of the reference month, not including the current month, have you or any members of your household purchased any meals at school for preschool through high school age children? 1. Yes 2. No
What are the names of all household members who purchased meals at school? * Enter line numbers for all that apply. [enter value] _____________
Since the first of ^REF_MONTH, not including ^CUR_MONTH, what has been the usual expense for the meals ^NAME purchased at school? [enter value] _____________
SCHLMLPD * Specify time period 1. Days 2. Weeks 3. Months 4. Percent 5. Half school year 6. School Year 7. Other, specify |
|
Revise asset and liability questions
The next few questions are about credit, loans, and financial assets. We know people aren't used to discussing their debt and financial assets, but we use this information to get a picture of how spending relates to changes in debt and assets. Be assured that, like all other information you have provided, these answers will be kept strictly confidential.
As of TODAY- [Do you/does your household] have a balance on any major credit cards including store cards and gas cards?
What is the total amount owed on all cards? _________ What was the total amount paid in finance, interest, and late charges for all cards in ^LAST_MONTH? _________ As of TODAY - [Do you/does your household] have any student loans?
What is the total amount owed on all student loans? _________ What was the total amount paid in finance, interest, and late charges for all student loans in ^LAST_MONTH? _________ As of TODAY - [Do you/does your household] have any other debt such as medical loans or personal loans?
What is the total amount owed on all other loans? _________ What was the total amount paid in finance, interest, and late charges for all other loans in ^LAST_MONTH? _________ As of TODAY – What is the total value of any checking, savings, money market mutual fund accounts, and certificates of deposit or CDs [you/your household] [have/has]? _________ As of TODAY – What is the total value of all retirement accounts such as 401(k)s, IRAs, and Thrift Savings Plans that [you/your household] [own/owns]? _________ Do you [or any members of this household] have a defined benefit retirement plan?
As of TODAY – What is the total value of all directly-held stock and stock funds [not in retirement accounts] that [you/your household] [own/owns]? _________ What is the total value of all directly-held bonds and bond funds [not in retirement accounts] that [you/your household] [own/owns]? _________ As of TODAY – What is the total value of all other financial assets, such as whole life insurance, annuities, trusts, and royalties that [you/your household] [own/owns]? _______ |
IF HH Size > 1
|
Combine Items
Combine “golf courses” and “country clubs” with other sports club entry fees AND combine social organizations with civic, service, and fraternal organizations
Have you had any membership costs or other expenses related to any of the following? 1. Subscriptions to newspapers, magazines, or periodicals. Include online subscriptions. 2. Books purchased from a book club 3. Season tickets to theater, concert series, opera, other musical series, or amusement parks 4. Season tickets to sporting events 5. Encyclopedias or other reference books 6. Golf courses, country clubs, health clubs, fitness centers, swimming pools, weight loss centers, or other sports and recreational organizations 7. Vacation clubs 8. Civic, service, fraternal, and other social organizations 9. Credit card membership fees 10. Shopping memberships including warehouse clubs like Sam’s Club and discount memberships like Amazon Prime 11. Services that use Global Positioning System or GPS, such as OnStar, not already reported 12. Direct or online dating services |
IF REPORTS
|
Combine AAA and OnStar into single question
Since the first of ^REF_MONTH, not including ^CUR_MONTH, have ^YOU_ANYMEM had any expenses for automobile service clubs such as AAA or services such as OnStar or LoJack? 1. Yes 2. No
How much? [enter value] _____________ |
IF REPORTS
|
Combine bank services and safe deposit boxes into a single question
Have ^YOU_ANYMEM paid any charges or fees for bank services such as overdraft fees, ATM fees, other account service charges, or safe deposit box rental from financial institutions such as banks, credit unions, and finance companies? 1. Yes 2. No
What is the usual MONTHLY charge? [enter value] _____________ |
|
Combine dishes and glasses into single question
Have you or any members of your household purchased any -- 15. Lamps or other lighting fixtures? 16. Other household decorative items? 17. Closet storage items? 18. Travel item including luggage? 19. Dishes, glasses, or serving pieces? 20. Stainless, silver, or other flatware? 21. Non-electric cookware? |
|
Combine internet service item codes into single question AND Reword Internet expenditure question to cover Internet at hotels and other locations
Since the first of the reference month, have you or any members of your household had any expenses for:
1. Cable or satellite TV, not already reported? 2. Satellite radio services, not already reported? 3. Internet service at home not already reported or at places such as hotels and airports? 99. None/No more entries |
IF REPORTS
|
Combine PDAs, digital book readers, and tablets into single question
Since the first of the reference month, have you or any members of your household purchased or rented any of the following items? 1. Small electrical kitchen appliances 2. Electrical personal care appliances 3. Smoke detectors 4. Electric floor cleaning equipment 5. Sewing machines 6. GPS devices, calculators, and fax machines 7. Digital book readers and tablets 8. Computers, computer systems, or related hardware 9. Computer software including computer games |
|
Combine phones and accessories with answering machines into single question
Since the first of the reference month, have you or any members of your household purchased or rented any of the following items? 13. Video game hardware or accessories 14. Video games 15. Telephones or accessories including answering machines 16. Photographic equipment 17. Musical instruments, supplies, or accessories 18. Lawn mowing machinery or other yard equipment 19. Power tools 20. Non-power tools 21. Window air conditioners 22. Portable cooling or heating equipment 23. Televisions, all types |
|
Combine prepaid cellular and regular cellular into single question
What types of telephone services did the bill include? 1. Residential Service including VoIP 2. Mobile/Cellular Service including prepaid |
|
Combine residential phone service and VoIP into single question
Since the first of the reference month, have you or any members of your household received any bills for telephone services, including cellular and Voice Over IP? Do not include bills for telephones used entirely for business purposes. 1. Yes 2. No
What types of telephone services did the bill include? 1. Residential Service including VoIP 2. Mobile/Cellular Service |
|
Combine rental and purchase questions for medical related items
Since
the first of the reference month, have you or any members of your
household made any payments for the following? |
|
Combine rental of sports equipment and fees for playing sports into single question
Did ^YOU_ANYMEM pay any fees to play sports, exercise, or rent any sports equipment ^NotCountingAnySpeqp? 1. Yes 2. No
How much did ^YOU_ANYMEM pay? / What costs for playing sports or renting sports equipment won’t be reimbursed? [enter value] _____________________ |
IF REPORTS
|
Combine smoke detectors and other household appliances into single question
Since the first of the reference month, have you or any members of your household purchased or rented any of the following items? 1. Small electrical kitchen appliances 2. Electrical personal care appliances 3. Electrical floor cleaning equipment 4. Other household appliances including smoke detectors 5. Sewing machines 6. GPS devices, calculators, and fax machines |
|
Combine sweaters, shirts, blouses, and tops into single question
Since the first of the reference month, have you or has any member of your household purchased any of the following items, either for members of your household or for someone outside your household? 1. Coats, jackets, or furs? 2. Sport coats or tailored jackets? 3. Suits? 4. Vests? 5. Shirts, sweaters, blouses, or tops? 6. Pants, jeans, or shorts? 7. Dresses? 8. Skirts?
What was the total cost of these purchases? [enter value] _____________ |
IF REPORTS
|
Combine uncommon, similar items in the auto repair section
Since the first of the reference month, have you or any members of your household had expenses for any of the following? 1. Oil change, lubrication, or oil filter 2. Other regular vehicle maintenance or servicing such as tune-ups or alignment 3. Battery purchase or installation 4. Tire purchase and installation 5. Tire repair 6. Body work or painting 7. Any vehicle or engine repairs 8. Vehicle accessories or customizing 9. Other vehicle services, parts, or equipment 10. Vehicle cleaning services including car washes 99. None/No more entries |
IF REPORTS
|
Combine water and sewer into single question
Since the first of the reference month, have you or any members of your household received any bills for any of the following utilities, fuels, or services? Do not include bills for properties used entirely for business. 1. Electricity 2. Natural or utility gas 3. Fuel oil 4. Bottled or tank gas 5. Other fuels including wood 6. Piped-in water and sewerage maintenance 7. Garbage and recycling collection 8. Water softening service 9. Septic tank cleaning 99. None/No more entries |
|
Combine childcare codes in Misc. expenses and reword “Nursery school” to “preschool” in educational expenses
Since
the first of the reference month, have you or any members of your
household paid for - |
IF REPORTS
|
Combine response options for question about who pays health insurance premiums
***Do
you have a health insurance plan? |
|
Reword current questions
Reword camping and hunting and fishing equipment to include “including all guns”
Since the first of the reference month, have you or any members of your household purchased or rented any of the following items? 31. General sports equipment (exclude athletic shoes for sports-related use, such as football, baseball, soccer, or bowling 32. Health and exercise equipment 33. Camping equipment 34. Hunting and fishing equipment including all guns 35. Winter sports equipment 36. Water sports equipment |
|
Reword “garage rental” to “parking garage”
Since the first of ^REF_MONTH, have ^YOU_ANYMEM had expenses for – Parking, such as parking garages, parking lot fees, or parking meters? Do not include expenses that are part of your property ownership or rental costs, a business expense, or expenses that will be totally reimbursed. 1. Yes 2. No
How much was paid, not including any payments made this month? [enter value] _____________ |
IF REPORTS
|
Reword “trash and garbage collection” to “garbage and recycling collection”
Since the first of the reference month, have you or any members of your household received any bills for any of the following utilities, fuels, or services? Do not include bills for properties used entirely for business. 1. Electricity 2. Natural or utility gas 3. Fuel oil 4. Bottled or tank gas 5. Other fuels including wood 6. Piped-in water and sewerage maintenance 7. Garbage and recycling collection 8. Water softening service 9. Septic tank cleaning 99. None/No more entries |
|
Reword audio/video items to include those in vehicles
Have ^YOU_ANYMEM purchased or rented any -- * Read each item on list 22. Televisions, all types including those installed in vehicles 23. DVD players, VCRs, DVRs, or video cameras 24. Satellite dishes, receivers, or accessories 25. Handheld personal music players 26. Stereos, radios, speakers, and sound components including those installed in vehicles 27. Other sound or video equipment including accessories |
IF REPORTS
IF NO REPORTS
|
Reword Medical Expenditure Reimbursement Section Intro to exclude other types of accounts
Now I am going to ask some questions about your reimbursements.
By reimbursements, I mean any money received for any money received for any members of your household from an insurance company, medical care provider, or non-household member for medical expenses which you previously paid or will pay.
Do not include reimbursements from any consumer driven health plans such as Flexible Spending Accounts (FSA), Health Reimbursement Accounts (HRA), Health Savings Accounts (HSA), High Deductible Health Plans (HDHP), or Medical Savings Accounts (MSA). |
|
Reword Medical Expenditures Section Intro to include mention of “co-pays” and “out-of-pocket expenses”
Now I am going to ask some questions about medical payments and reimbursements. I will begin with your payments.
By payments I mean any co-pays and out-of-pocket expenses. Include all payments, even those for persons who are outside your household. |
|
Reword other vehicles to include “including boats and planes”
^CU_DoDoes ^YOU_YRCU own any other types of vehicles including boats and planes which are not used entirely for business? * IF YES - Do you own any - 2. Motor home 3. Trailer-type camper 4. Other attachable-type camper 5. Motorcycle, motor scooter, or moped (motorized bicycle) 6. Boat, with motor 7. Boat, without a motor 8. Trailer other than a camper type, such as for a boat or cycle 9. Private plane 10. Any other vehicle 95. Continue list |
|
Replace word “convalescent” with “rehabilitation” and “rehabilitative”
Since the first of the reference month, have you or any members of your household made any payments for the following? 1. Eye examinations, treatment, or surgery 2. Purchase of eye glasses or contact lenses 3. Dental care 4. Hospital room or hospital services 5. Services by medical professionals other than physicians 6. Physician services 7. Lab tests or x-rays 8. Care in rehabilitation centers or nursing homes 9. Care for invalids, persons undergoing rehabilitation, handicapped, or elderly persons in the home 10. Adult day care centers 11. Other medical care and services |
|
Replace word “paperbacks” with “digital books” in OTHRBKRF
Since the first the reference month, have you or any members of your household ....
Bought any books, including digital books, which were not purchased through a book club? Do not include school books or reference books. 1. Yes 2. No
What was the total expense? [enter value] ________
How much of the total amount was spent this month? [enter value] ________ |
|
Appendix C: Consent Form
The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.
The BLS, its employees, agents, and partner statistical agencies will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.
During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.
We estimate it will take you an average of 45 minutes to participate in this research (ranging from 30 minutes to 60 minutes).
Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.
Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires February 29, 2012.
------------------------------------------------------------------------------------------------------------
I have read and understand the statements above. I consent to participate in this study.
___________________________________ ___________________________
Participant's signature Date
___________________________________
Participant's printed name
___________________________________
Researcher's signature
OMB Control Number: 1220-0141
Expiration Date: 02/29/12
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
Attachment D: Introductory Material
Hi! Thank you for coming in today.
I am …... [This is my colleague ( ) who will be taking notes for us today]
Have you participated in any of our studies before? (if yes, Which ones?)
Consent Form/Permission to audiotape
Explanation:
We are going to be working with some questions from the Consumer Expenditure Survey, an ongoing survey that provides information about the buying habits of American consumers. This information is used for a lot of things but its primary purpose is to help calculate inflation. Are you familiar with the inflation rate?
What we are going to do today is go through a few questions from the survey and get your reactions to them. The purpose of this session is to help us find out more about how people respond to these questions. Basically, we’re trying to find out what you think a question is asking and how hard it is to answer. We are not here to evaluate you, we are looking to improve the questions, so there are no wrong answers. All the information you give us will be kept completely confidential, and will be used to improve the survey questions.
Any questions before we begin?
File Type | application/msword |
File Title | October 14, 2003 |
Last Modified By | kopp_b |
File Modified | 2012-01-31 |
File Created | 2012-01-18 |