July 8, 2016
NOTE TO THE REVIEWER OF: |
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
|
FROM: |
Erica Yu Research Psychologist Office of Survey Methods Research
|
SUBJECT: |
Submission of Materials for “Testing instructions for organizing electronic records for the redesigned CE interview” |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The total estimated respondent burden hours for this study is 49 hours.
If there are any questions regarding this project, please contact Erica Yu at 202-691-7924.
Introduction
The approved Gemini redesign proposal for the Consumer Expenditure (CE) Survey calls for two separate interviews as well as individual diaries using a single sample of consumer units. After the first interview collects reasonably easy to recall expenditure categories and the individual diaries collect small, frequent, and personal expenditure categories, the second interview will ask respondents to collect financial records related to select expenditure categories to use when answering the interview questions. This redesign creates a new opportunity for CE interviewers during Visit 1 to encourage respondents to collect records in preparation for Visit 2.
Records relevant to Visit 2 include household bills, account statements, bank statements, checkbooks, and other financial documentation that can provide accurate, detailed, expenditure information that is typically difficult for respondents to recall. It is expected that much of this information will be readily available to use during the interview in paper form; however, we also expect that many respondents may have only electronic versions of some records. Electronic records impose additional challenges to the records collection process if respondents are to be able to easily access them during the interview: the records need to be downloaded and saved to a central, organized location before the interview, or the respondent must use interview time to log into accounts and find the requested information. Can this process be optimized to reduce the amount of time spent locating information on records before and during the interview?
A previous project, completed in spring 2016, collected information about how people typically use electronic records, including for what expenses people use electronic records, how people access those records, and how respondents in a CE interview might use records. That study found that CE respondents are likely to have electronic records for most of the expenses asked about in the interview, and that the most common method for accessing those records was to log in to online accounts, rather than to access records stored locally on the respondent’s computer. This finding supports the need for further research on designing guidance for respondents on how to efficiently organize their electronic records for use during the CE interview. The purpose of this study is to test these instructions with participants.
Methodology
The guidance to respondents on how to collect and organize electronic records will be tested using a mixture of in-lab testing and at-home records collection. Participants will be asked to visit the lab two times, between 3 and 14 days apart. Participants will be asked at the first visit to collect and organize electronic records in advance of the second visit. A brief description of the methodology is below and the full protocol is available in Attachment A.
Visit 1. At the first visit, participants will be introduced to the CE survey and the interviewer will explain the expenses we are asking about and what kinds of records we are asking the participant to gather. Instructions will be given on how to save and organize electronic records in preparation for an interview. As this is pretesting, we expect that modifications may be made during the course of the study based on initial interviews. Although the goals of the testing, and overall design, will remain the same, findings from preliminary results may be used to improve the guidance to participants about collecting electronic records. It will be emphasized to the participant that we do not need to collect the information on the records (i.e., they do not need to report the expense information to us); rather, we are interested in the process of gathering the records. If a second visit has not been scheduled, one will be scheduled at that time.
At-Home Records Collection. Participants will be asked to take home a Records Checklist (Attachment B), which will include guidance on collecting and organizing electronic records, and to use that guidance to collect records in advance of the second visit. The Checklist will include space for the participant to take notes on how they collected each record and how they organized the files. The participants will be instructed to create an electronic folder for their records and to label them. The interviewer will encourage the participant to take screenshots or photos of the final organization of their electronic records; sharing the image with the interviewer (e.g., showing the photo on their phone screen during Visit 2) will be voluntary. The interviewer will call the participant approximately three days after Visit 1 to answer any questions.
Visit 2. The participant will return to the lab between 3 and 14 days after Visit 1, bringing the Checklist with them. Participants will be debriefed about their records collection experience. Specifically, they will be asked how they went about accessing records, how they organized the records, and whether they encountered any problems while saving the records. They will also be asked for any recommendations for improving the records collection instructions.
Following the debriefing about their experience, participants will be asked to complete several electronic records exercises in the lab with the interviewer. The interviewer will provide participants with three scenarios and participants will be asked to guide the interviewer through the process of saving and organizing the records (Attachment C). These scenarios have been designed to test unique circumstances that present challenges in the file organization process:
No “file” can be saved from the online account: In these circumstances, the expense information is shown on the screen but there is no simple way for participants to save the original record. Participants may skip saving the record, or record incomplete information.
Multiple records for a single item category: Participants may purchase a single type of item multiple times during the reference period, and have multiple records (e.g., recurring monthly bills, multiple doctor visits). These records will have to be organized such that the participant can locate each record and all records.
A record that includes multiple expenses across categories: Participants may purchase multiple items spanning categories at the same time, resulting in a record that crosses categories. Participants will have to organize the record such that it can be used at multiple points during the interview.
The interviewer will note how the participant handles these situations and then ask follow-up questions about the process. The content of the scenarios may be modified if preliminary results show that participants are having difficulty understanding the task or, alternatively, if the tasks are so simple as to not be informative.
Participants
Twenty participants will be recruited from a participant database maintained by OSMR. Efforts will be made to select participants with varying levels of education, income, occupation, and household size. Participants will be told during the initial recruitment that they must
1) have and be familiar with electronic records;
2) be available for two sessions, approximately 3-14 days apart; and
3) be willing to complete the at-home records collection task, including noting the steps they take to access and organize the records.
Participants will be informed of the total estimated amount of time we are asking them to spend on the study (2 hours and 20 minutes). The recruiter will attempt to schedule both sessions at that time. If the participant can only schedule the first meeting at the time of initial recruitment, the second interview will be set up at the end of the first interview.
Burden Hours
The total burden hours for this study will be 49. Twenty participants will be recruited to participate in this study. We anticipate a moderate acceptance rate for the initial recruitment contact with potential participants (approximately 75%); we estimate that we will need to contact approximately 27 people in order to get 20 interviews. We expect these initial recruitment calls to last 5 minutes, for a total of 2.25 burden hours (27 participants x 5 minutes). The screening protocol is included in Attachment D.
The first visit will last up to 20 minutes, for a total of 6.67 burden hours (20 participants x 20 minutes).
We expect that participants in this study will spend as long as 60 minutes collecting records at home, for a total of 20 burden hours (20 participants x 60 minutes).
The second interview will last up to one hour, for a total of 20 burden hours (20 participants x 60 minutes).
|
N |
Minutes/ Participant |
Total Burden Hours |
Initial Recruitment |
27 |
5 |
2.25 |
Visit 1 |
20 |
20 |
6.67 |
Records Collection At Home |
20 |
60 |
20 |
Visit 2 |
20 |
60 |
20 |
|
TOTAL |
2 hr. 25 min. |
48.92 hours |
Payment to Participants
Each participant will be compensated a total of $80 for their participation in this study. Because it is vital that the participant returns for the second interview, we will compensate the participant $30 for their first visit and $50 at the time of their second visit. We believe that offering a greater incentive for the second visit (while still maintaining an average of $40 per visit) will be helpful in limiting the number of breakoffs between the first and second study. If the participant does not come back for the second interview, we do not get their Checklist and we do not get their feedback; both key to the success of this study. A total of $1600 will be spent on payments to participants.
Data Confidentiality
Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national Consumer Expenditure survey. Participants will be given a consent form to read and sign (see Attachment E). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.
Attachments
Attachment A – Interview protocol
Attachment B – Records Checklist
Attachment C – In-lab simulations
Attachment D – Screening protocol
Attachment E – Consent form
Attachment A: Interview Protocol
This protocol includes the script that will be used for both lab visits. The interviewer may ask unscripted follow-up questions (e.g., “Can you tell me more about that?”) if a participant’s answer suggests additional, pertinent information can be gained.
Visit 1
Hi! Thank you for coming in today.
I have a couple of colleagues in the next room that will be observing and taking notes.
Our recruiter told you that this study involves two visits – one today and one a few days later, right?
In this study, we’re going to be helping out with the Consumer Expenditure Survey Interview. This survey collects information about how US households spend their money, and the data are used to calculate how much average Americans are spending and what the rate of inflation is.
They need your help with coming up with instructions to give respondents. The survey needs people to collect their online records – like bills, receipts, and account statements – all in to one place and organize them so that they could be used for the Consumer Expenditure Interview. People might need help on that, so they want to provide instructions on collecting electronic records.
Our goal here is just to test-drive those instructions and see what they’re like. I’ll show you some instructions, ask you to follow them, and then get your feedback on whether the instructions were clear, whether they were easy or difficult to follow, and whether there’s anything we can do to improve them.
I’m not interested in your expenses like how much money you spent, I just want you to go through the motions as if you were going to participate in the Consumer Expenditure Survey Interview.
It’s important to note that this is not a test of your ability. If the instructions are difficult to follow, that’s something we need to work on and we need to hear it from you.
Today I’m going to give you the instructions that we’ve been talking about. Then, I’ll ask you to go home and try them out on your own – collect your electronic records and organize them. That part at home will probably take between 30-60 minutes. Then, you’ll come back in and we’ll talk about what it was like. How does that sound?
[Consent form]
Great. Let’s talk about the Consumer Expenditure Interview for a moment. Here is an example of the kind of question that they would want you to be able to answer. They want a description of the item, how much it cost, and in what month it was paid for. I want you to be looking for records that have this kind of information on them.
Since the 1st of May, did you or anyone in your household have any expenses for electricity? |
||
|
Company name |
Pepco |
Report 1 |
Amount |
$71.06 |
Month |
May |
|
Report 2 |
Amount |
$91.79 |
Month |
June |
|
Report 3 |
Amount |
$103.62 |
Month |
August |
Now let’s look at the Checklist. You’ll be taking this home with you today and using this to guide the collection of electronic records at home.
Here you can see the instructions for what we are asking you to do: for each of these items, mark down whether you had that expense in the past three months. If you did, then mark down whether you have a paper record or an electronic record – an electronic record is anything like a bill, a receipt, or an account statement that is stored electronically, not on paper. It might be in your email, in an account where you go to a company’s website and then log in, or maybe just in your checking account or credit card statement.
The goal here is to save that record so that you can use it at a later time, during an interview. Here you can see are step-by-step instructions on how to go about saving the information. Follow these, but if you’re not sure how to handle something, just give me a call and I’ll try to help.
Part of saving the record is making sure that you’ll be able to find it again. Make sure the name of the file helps you to identify what the record is for. If you need to change it, you can follow these instructions.
Looking at this checklist again, take a look at these columns. Here you’ll see that we’re asking you to jot down some notes about how you went about saving each file. This is the most important part – fill this out for each expense.
When you have saved records for all of the expenses on the list, you should have a folder with many files in it. Take a look at this folder and see if you need to make any changes – maybe you want to make subfolders or you want to change the file naming system. Anything you do now could save you time later on. If you make changes, note that down, too.
If you can, then please take a screenshot of the folder, or a photo with your phone. If you could email that to yourself or show me the photo that would help me to know what kind of folder you ended up with. This is optional, but it would be very helpful.
How comfortable do you feel with the task of collecting electronic records?
[Extremely comfortable, very comfortable, somewhat comfortable, somewhat uncomfortable, very uncomfortable, extremely uncomfortable]
Visit 2
At-home records collection debrief, looking at the Checklist they bring in (15 minutes)
Tell me about what it was like to collect the electronic records.
Was there anything about the process that was unexpected?
Do you feel like the instructions and Checklist covered everything you needed to know, or what were they missing?
Did you happen to have a photo or screenshot of what your final records collection looked like?
Did you ever have to look up records from someone else in your household?
About how long would you say you spent on collecting records?
Let’s take one of your expenses [pick one] – tell me what you did to save this record.
Great. That was really helpful. We’ll be taking a look at all of the information you recorded here – we’ll combine it with the data from the other participants who are coming in to get a sense of what it’s like to collect records.
In-lab exercises (25 minutes)
Now I’d like us to go through a couple of exercises together. Since I couldn’t look over your shoulder while you were at home, I’m going to have you do it here! I’ll ask you to imagine that what you see on the screen is one of your own expenses, and you need to save it.
What you’ll see on the screen are just some mock-ups. They are really basic versions of websites, so they might not act exactly like what you’re used to, but hopefully they’ll come close. And they are fake accounts and they don’t have your real expense information, or anyone’s real information.
On this screen here, you can see there is a list of all the expenses that this imaginary “you” has. We’re going to go through a short Checklist, and I’d like for you to come to this page, find the expense you want, click on it to go to it, and save it.
Scenario 1 – No “file” can be saved from the online account:
At your apartment, you pay your own internet and phone through CastCompany, but do not have to pay for electricity, gas, water, or any other utilities. Please save the records that you would need for the interview.
Scenario 2 – Multiple records for a single item category:
You went to the eye doctor for a vision exam and got a new pair of glasses. Please look save the records that you would need for the interview.
Scenario 3 – A record that includes multiple expenses across categories:
You have made several online purchases from Rainforest.com. Please save the records that you would need for the interview.
Attachment B: Records Checklist
The Checklist will be printed on legal size paper, with information on both sides. One side will have the Checklist and space for participants to record their record collection process, with large boxes for handwritten text entry (to be extended when the Checklist is printed on legal size paper for the study). The other side of the paper will have instructions about how to gather electronic records and what information to note for the research study.
Collecting Records for Your Interview
Thank you!
Thank you for agreeing to take part in this research study! Your participation will help us to design the survey and our materials before they are sent out to thousands of people across the country. We value your honest feedback – positive and negative – to help us make improvements.
Prepare for your interview
Collect records to help you remember details about the expenses listed on this sheet. Records include things like:
Bills Statements
Emails Receipts
Try to collect records with the most amount of information on them. Only collect bank or credit card statements if you can’t access the original record.
We will not collect your records – we will not ask you for any expense information. You may decline to answer questions or share information for any reason. Any information you do report will remain confidential and will not be shared.
Your next appointment is on:
_______________, August ____ 2016
at ____: _______ am / pm
Collect your electronic records
C
reate
a new electronic folder and rename it “Records Interview”.
Take a look at the other side of this sheet to see a list of expenses.
For each expense that you have had (since May 1, 2016), download the record.
F
or
online accounts:
If you can, click a link or icon to download the record.
If that’s not offered, go to Tools > File > Save as, or use your mouse to right-click anywhere on the screen for a similar menu
Save the webpage to the “Records Interview” folder you just created. Make sure “Webpage, complete (*htm; *.html)” is selected.
Give the file a name that will help you to easily find it during the interview – like, “Electricity June” or “Creditcard July”.
For all other types of records:
If you have any questions about how to save other kinds of records, call the study manager, Erica, at (202) 691 - 7924
Take notes on the process of collecting records
We need you to take detailed notes on the steps that you took to collect your records. Please note what steps you took, whether you had any problems, and anything else that might help us to improve the instructions.
Did you have this expense?
Write in Yes or No
Do you have a record?
Mark the options that apply to you
Description of record(s)
Write down the type of record. Provide as much detail as you can.
Where did you go to find this record?
Write down if you went to a company website, your bank’s website, offline software, or anywhere else. If you had to ask another member of household, note that here.
What steps did you take to save the record?
Write down everything you did after you located the record. Include the name that you gave the file.
Any problems?
Note down anything that didn’t go as you expected or that caused you frustration.
Records Checklist
CE Item |
Did you have this expense? |
Do you have a record? |
Description of record(s) (bill, statement, etc) |
Where did you go to find this record? |
What steps did you take to save the record? |
Any problems? |
||
No Record |
Electronic |
Paper |
||||||
EXAMPLE |
Yes |
|
X |
|
Account statement |
The company website |
Pulled up the statement. Clicked the icon to download PDF. Named it “CarInsuranceJune”. |
The statement is quarterly, not monthly. |
Mortgage, total amount paid
|
|
|
|
|
|
|
|
|
Mortgage, of total amount, interest paid
|
|
|
|
|
|
|
|
|
Electricity
|
|
|
|
|
|
|
|
|
Internet connection
|
|
|
|
|
|
|
|
|
Physician, dental, and eye care services
|
|
|
|
|
|
|
|
|
Health insurance, amount spent on premiums, including payroll deductions |
|
|
|
|
|
|
|
|
Attachment C: In-lab Simulations
Attachment D: Screening Protocol
We’re recruiting for a new research study. The purpose of this study is to improve the instructions we give to respondents about how to collect electronic records. By “records” we mean things like receipts, bills, and account statements. We don’t need to ask you about your expenses, just about how you access and organize them.
As part of this study, you will come in to the lab two times. The first visit to the lab will be a short visit where the researcher will explain more about the study. Then, she will ask you to take some materials home with you. At home, you’ll need to spend about 30-60 minutes collecting records, following her instructions. Once you’ve done that, you’ll come back in to the lab, where you’ll share feedback about your experience collecting records and answer questions. Altogether, you’ll need to spend about 2 hours on this study. You will be paid $80 – the first time you come in, you will get a check for $30 and the second time you come in you will get a check for $50.
Can you agree to that?
[If yes] Ok, then we have a few questions to check your eligibility.
Do you have electronic records – that is, online bills, receipts, emails, account statements, that sort of thing?
Yes
No *Screen out
About how many would you say you use, or could access, if you wanted to?
0-3 *Screen out
4-9
10 or more
Are you willing to take detailed notes on the process of gathering the records?
Yes
No *Screen out
Can you come in twice within a two week period?
Yes
No *Screen out
Attachment E: Consent Form
The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.
The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.
During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.
(Visit 1) We estimate it will take an average of 20 minutes to participate in this research today and up to 1 hour to gather electronic records at home.
(Visit 2) We estimate it will take an average of 60 minutes to participate in this research today.
Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.
Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires on April 30, 2018.
------------------------------------------------------------------------------------------------------------
I have read and understand the statements above. I consent to participate in this study.
___________________________________ ___________________________
Participant's signature Date
___________________________________
Participant's printed name
___________________________________
Researcher's signature
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
Page
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | October 14, 2003 |
Author | Yu, Erica - BLS |
File Modified | 0000-00-00 |
File Created | 2021-01-27 |