Purpose:
Researchers in the Census Bureau’s Economic Statistical Methods Division (ESMD) and Economy-Wide Statistics Division (EWD), and with RTI International, will conduct participant debriefing interviews in support of the 2023 Annual Integrated Economic Survey (AIES). These interviews will cover three main topics: response process, communications materials, and instrument performance. Participants will represent both respondents and nonrespondents to the 2023 AIES.
Not all questions in this protocol will pertain to all participants, and some modules are optional depending on the length of the interview and firm characteristics. Numbered questions are priorities, and bulleted questions underneath are optional probes.
Research Questions: The research will be guided by the following research questions:
How are respondents reporting to the AIES?
What are respondents' overall impressions of the survey?
What is the ideal length and timing of the field period for respondents?
What are the unique reporting needs of companies with locations outside of the 50 U.S. states?
What are the barriers to reporting (non-respondents)?
What is respondents’ feedback on the content and accessibility of respondent communications?
What support materials are respondents using when reporting to the AIES?
Are the content selection tool and summary document sufficiently supporting response?
What are respondents’ impressions of letters and emails?
What is respondents’ feedback on instrument performance and response burden?
Informed Consent: Respondents will be asked to complete a consent form electronically before the time of the interview.
Materials Needed:
Electronically signed consent form
Letters: AIES-L1, AIES-L3, AIES-L4L, AIES-ECSL1
Emails: AIES Initial Email; AIES Due Date Reminder Email; AIES Postcard
Websites: AIES Interactive Content Selector Tool; AIES Front Page webpage; AIES Information for Respondents webpage; AIES FAQs webpage
Respondent recruitment and firm information, including: Contact info; response status; establishments in Puerto Rico; establishments in manufacturing; single/multi-units; item missingess; pilot participation flag; number of establishments; number of industries
Method: We will conduct the interviews by Microsoft Teams (or phone).
Expected length of interview: 1 hour (60 minutes) maximum
General probes that may be used throughout the interview:
Were these data easy to access?
What else can you tell me about this?
Can you tell me more about that?
How confident are you in that response?
What looked unclear or is confusing here?
If necessary: You should have received a link to a consent form from our recruiter via email. Did you have a chance to review and sign that yet?
[IF YES] Did you have any questions about any of the information presented there?
[IF NO] Please open up the link now and review the form, then sign it. Take your time and let me know if you have any questions.
Thank you so much for agreeing to talk with me today!
As part of the roll out of the new Annual Integrated Economic Survey, we are following up with some companies to learn more about the processes you may or may not have used to complete the survey.
I am working with the Census Bureau to make sure that the Annual Integrated Economic Survey is performing as expected and to get feedback about ways to improve the performance of the survey instruments. I’m talking with you today because you are identified as the person who provided response to the AIES this year. When I refer to “the AIES survey” during our conversation I’m talking about the Annual Integrated Economic Survey that you responded to about your business within the past few weeks.
My job is to improve the AIES survey. I didn’t write the survey questions, so you do not have to hold back when telling me your feedback. Please be candid and frank in your responses. Our interview is being conducted under the authority of Title 13, which means that your responses are confidential, and neither your name nor the name or identifying information about your company will be included in any of our findings.
Do you have any questions before we get started?
I’d like to record our session today so that when I go to analyze the results of these interviews, I can use the recording to pick up on anything I may have missed in my notes. Do I have your permission to record our session today?
[Turn on recording]
[State respondent ID number and date, repeat consent on recording: Do you agree to participate in the interview today? Is it okay if I record our conversation?]
Universe: All participants
What is your role in the company?
What is your role in the process for responding to Census Bureau surveys in general? How long have you been in this role?
How do you find the data you need to answer Census Bureau surveys?
Do you work with anyone else to get the data you need?
Do you access any reports to get the data you need?
How easy or difficult is it for you to find all the data you need for Census Bureau surveys? Why is that?
How easy or difficult is it for you to enter the data into the survey once you have the answers you need? Why is that?
Tell me a little bit about your business. What types of goods or services does this business provide?
Universe: All Respondents (optional)
The Census Bureau usually allows four to six weeks from the initial email about the survey to the due date to complete the survey. What do you think about this length of time given for completing this survey?
How easy or difficult is it to respond with the current time frame?
How would it impact your response if we extended or shortened this length of time?
Were you aware that due date time extensions were available for the AIES?
Universe: All respondents
What differences, if any, did you notice between this AIES survey and other Census Bureau surveys you have taken in the past?
In what ways was your approach the same or different for the AIES versus other annual Census Bureau survey(s)?
Did anything change in the way you retrieved the data you needed to answer the AIES compared to other annual Census Bureau surveys?
Did anything change in the way you entered the data you needed to answer the AIES compared to other annual Census Bureau surveys?
What was your overall impression of the AIES?
How easy or difficult was it to answer the survey?
Did some topics or sections take more or less time than others? Which ones? Why?
Which was the most difficult to answer questions for: the company level, the location level, or the industry level, or were they equally difficult? Why?
Did you have any confusion about which parts of the company to include when answering the AIES?
Overall, how easy or challenging was it to submit your response to this survey?
Based on your experience this year, are there ways you might prepare for the AIES in the future?
Universe: All respondents (optional)
How long do you think it took you to complete the whole survey, including any time to retrieve the data you needed?
Compared to other Census Bureau surveys you may have completed in the past, how easy or difficult was it to answer the AIES? Why?
Compared to other Census Bureau surveys you may have received, how similar or different was it to answer the AIES? Why?
Universe: All respondents with at least one location in Puerto Rico
In preparing for our conversation today, I noticed that your company has at least one location in Puerto Rico.
How easy or difficult was it to report some but not all data for your locations in Puerto Rico? For example, we ask for revenue by locations in Puerto Rico, but we don’t ask for capital expenditures including locations in Puerto Rico.
Are your records for Puerto Rican locations the same or different than those in the 50 states?
Do you use the same steps to pull data for Puerto Rican locations as you do for those in the 50 states?
Do you use different steps to report data for locations in Puerto Rico?
How accessible is the information we request on the AIES for your locations in Puerto Rico?
Do you track the same information for your locations in Puerto Rico as you do for those in the 50 states?
If we asked all the questions on AIES to include all of your locations in Puerto Rico, how would that impact your response to the AIES? That is, would you have to access different systems, reach out to different people, or have other steps to report all of the questions on the AIES for your locations in Puerto Rico?
How do language barriers impact how you respond for your Puerto Rican locations, if at all?
When you reach out to your locations in Puerto Rico, do you encounter any language barriers? What do you do about it, or how do you handle language barriers?
In what ways could the Census Bureau mitigate or minimize language barriers in getting data for our survey from your company?
When you reach out to your locations in Puerto Rico, do you encounter any cultural differences that impact your ability to get requested data? What do you do about it, or how do you handle cultural differences?
Universe: All nonrespondents
Tell me a little about yourself…
Whose job is it typically to respond to Census Bureau surveys?
What recent changes to your job or at your company might impact how you respond to Census Bureau surveys, if any?
What challenges do you specifically face with answering survey questions?
What do you think the information we ask for in this survey is used for?
We hear from businesses that sometimes our communications simply do not make it to the right person at the business. Do you usually receive the letters and emails that the Census Bureau sends out about surveys, or is it someone else at your company?
How do you get your work-related mail? What is the process, and how much time does it usually take?
What are some ways that communications do not reach you in a timely way? That is, are there ways that we could address our letters or flag our emails to ensure that they get to you in enough time to respond to our requests?
If our communications ever went to the wrong person, how would you be notified, if at all?
In preparing for our time together today, I noticed that you did not respond to our request for the Annual Integrated Economic Survey. So far, we would have sent letters and emails about the survey to your company. Can you tell me about that?
Are there things about your company that pose a challenge for you as you complete the AIES?
What should we keep in mind about response for companies like yours?
Were you able to look through the survey? What were your thoughts?
Are there other surveys from us you’ve found challenging?
What, if anything, could we at the Census Bureau do to support you in responding to the AIES?
While I’ve got you on the phone, I’d like to run a few letters past you for your feedback. These are about the AIES.
Interviewer
Note: Enter the AIES-L4L into the chat.
Take
a look at this letter – not all companies receive this one.
Tell me what it is all about.
What is the first thing you notice about the letter?
What is the purpose of this letter?
What stands out about this letter?
What would you do if you got this letter?
Interviewer
Note: Enter the AIES-ECSL1 into the chat.
This
last one is different because it would not be sent to you, it would
be sent to the most senior financial officer we had on file for
your company. I want to stress that we have not sent this letter
out, and we are not going to send this letter. We are just looking
for feedback on a hypothetical letter like this one being sent to
the most senior financial officer we can identify. In this
hypothetical situation, we would have reached out to this company
multiple times by email, mail, and phone, and gotten no
response.
Tell me what it is all about.
What is the first thing you notice about the letter?
What is the purpose of this letter?
What stands out about this letter?
How do you feel about this letter being sent to the most senior financial officer we could identify at your company? What might happen if we sent a letter like this to the senior financial officer?
Interviewer note: Select ONE of Modules 7 – 10.
Universe: All participants (optional)
Interviewer
note: make the participant a presenter in Teams. Now,
let’s take a look at a website for the AIES. I’m going
to put a link in the chat of our meeting, please click on it.
https://www.census.gov/aies/questionspreview/
If you feel comfortable, please share your screen with me. You
can do this by clicking on the “Share” button next to the
red “Leave” button, it has an upward pointing arrow on
it.
Take a look at this page – please tell me what this page is all about. What is this?
Did you know about this page before our time together today?
Did you use this tool before today? If so, please tell me about that.
Please think about what your company does or makes. Now, select the industry or industries that best represent your company using this tool.
Did you find it easy or difficult to select your industry/industries?
How did you know what to select?
How confident were you in making selections?
Now that you’ve made selections, let’s preview questions for your business at the company, industry, and establishment or location levels.
What are you looking at on your screen? What do you notice?
Where would you go to find the question for “What were the total sales, shipments, receipts, or revenue in 2022?” by location?
We’ll move on to the next task of exporting the questions for your business at the company, industry, and establishment level to a spreadsheet file. Please show me how you would go about doing that.
Did you find it easy or difficult to export your survey questions to a spreadsheet file?
What would you do with this download once you had it?
Would this be helpful for you in completing the AIES? If so, how would it be helpful to you?
Universe: All participants (optional)
Interviewer note: make the participant a presenter in Teams. Now, let’s take a look at a few website pages for the AIES. I’m going to put a link in the chat of our meeting, please click on it. If you feel comfortable, please share your screen with me. You can do this by clicking on the “Share” button next to the red “Leave” button, it has an upward pointing arrow on it.
Interviewer
note: Enter the following URL in the
chat:
https://www.census.gov/programs-surveys/aies.html
Take
a look at this page – please tell me in your own words what it
is all about.
Did you see this page before our time together today?
What stands out on this page for you?
What is the most important information on this page?
What, if anything, do you expect to see that is not on this page?
When might you visit this page?
Interviewer
note: Enter the following URL in the
chat:
https://www.census.gov/programs-surveys/aies/information.html
Now,
let’s take a look at this page – please tell me in your
own words what this page is all about.
Did you see this page before our time together today?
What stands out on this page for you?
What is the most important information on this page?
What, if anything, do you expect to see that is not on this page?
When might you visit this page?
Interviewer
note: Enter the following URL in the
chat:
https://www.census.gov/programs-surveys/aies/faq.html
Ok,
last one – let’s take a look at this page. Please tell
me in your own words what this page is all about.
Did you see this page before our time together today?
What stands out on this page for you?
What, if anything, do you expect to see that is not on this page?
What is the most important information on this page?
When might you visit this page?
Of all of the pages we just looked at, which one do you think would be most helpful in completing the survey, and why?
Do these pages make sense to you?
Is the information on these pages what you would expect to see?
Can you find the information you need to complete the survey?
Universe: All participants (optional)
Interviewer note: make the participant a presenter in Teams. Now, let’s take a look at a few of the emails we sent out for the AIES. I’m going to put a PDF document in the chat of our meeting, please open it. If you feel comfortable, please share your screen with me. You can do this by clicking on the “Share” button next to the red “Leave” button, it has an upward pointing arrow on it.
Interviewer
note: Enter the AIES Postcard into the chat.
Ok,
last one – this is a postcard that we sent to some businesses
but not all. Take a look at it – tell me what this all about.
Do you remember receiving this post card?
If so, did you find receiving this information useful?
Did you notice the QR code in the corner?
What is the likelihood that you would use your mobile device to access the QR code?
Do you generally interact with QR codes when you see them or do you generally ignore them? Why?
What would you expect this QR to point to – that is, what information would you want this to open up to?
Interviewer
note: Enter the AIES Advanced Email into the chat.
Take
a look at this email – we sent this to your company in late
February. Tell me what it is all about.
Do you remember seeing this email before?
If did remember receiving: What did you do after receiving this email?
If did remember receiving: Did it come directly to you, or was it forwarded to you?
What is the first thing you notice about the email?
Would you/did you find it useful to receive this communication in advance of the survey opening?
Did you visit the websites noted and use the tools there to begin preparing in advance of the survey opening?
What, if anything, would you change about this email, having completed the survey?
Interviewer
note: Enter the AEIS Due Date Reminder Email into the chat.
Now,
take a look at this email – tell me what this is all about.
Do you remember seeing this email before? Did you receive a paper copy of it?
If did remember receiving: What did you do after receiving this email?
If did remember receiving: Did it come directly to you, or was it forwarded to you?
What is the first thing you notice about the email?
What is the purpose of this email?
What stands out about this email?
Universe: All participants (optional)
Interviewer note: make the participant a presenter in Teams. Now, let’s take a look at a few of the emails we sent out for the AIES. I’m going to put a PDF document in the chat of our meeting, please open it. If you feel comfortable, please share your screen with me. You can do this by clicking on the “Share” button next to the red “Leave” button, it has an upward pointing arrow on it.
Interviewer
Note: Enter the AIES-L1 into the chat.
Take
a look at this letter – tell me what it is all about.
Do you remember seeing this letter before? Did you receive a paper copy of it?
If did remember receiving: What did you do after receiving this letter?
What is the first thing you notice about the letter?
What is the purpose of this letter?
What stands out about this letter?
What, if anything, would you change about this letter, having completed the survey?
Interviewer
Note: Enter the AIES-L2 into the chat.
Take
a look at this letter now – tell me what it is all about.
Do you remember seeing this letter before? Did you receive a paper copy of it?
If did remember receiving: What did you do after receiving this letter?
What is the first thing you notice about the letter?
What is the purpose of this letter?
What stands out about this letter?
What, if anything, would you change about this letter, having completed the survey?
Interviewer
Note: Enter the AIES-L4L into the chat.
Now,
take a look at this letter – not all companies receive this
one. Tell me what it is all about.
What is the first thing you notice about the letter?
What is the purpose of this letter?
What stands out about this letter?
What would you do if you got this letter?
Interviewer Note: Enter
the AIES-ECSL1 into the chat.
This
last one is different because it would not be sent to you, it would
be sent to the most senior financial officer we had on file for your
company. I want to stress that we have not sent this letter out,
and we are not going to send this letter. We are just looking for
feedback on a hypothetical letter like this one being sent to the
most senior financial officer we can identify. In this hypothetical
situation, we would have reached out to this company multiple times
by email, mail, and phone, and gotten no response.
Tell me what
it is all about.
What is the first thing you notice about the letter?
What is the purpose of this letter?
What stands out about this letter?
How do you feel about this letter being sent to the most senior financial officer we could identify at your company?
What information should be in the letter that is not in there now? Anything else you would need to know or needs to be included in the letter?
In your opinion, what would be the outcome if we sent a letter like this to your company?
Universe: All multi-unit respondents
Think about when you entered your data into the AIES instrument. There were a few points in the survey where you could download an Excel file or you could fill in the data using an online spreadsheet. How did you decide how to respond?
Did you use just one way to respond, or did you use both?
While completing the survey, did you ever flip between the two methods?
Did anyone else at your company use either the Excel file or the online spreadsheet?
What feedback do you have on your chosen response mode? How did it go?
Think about when you entered your data into the AIES instrument. There were a few points in the survey where you could run error checking on your submission. Do you remember running the error checking?
Did running error checking help you to submit your data?
What feedback, if any, do you have on error checking on this survey?
What do you recall about the process of running the error checking?
Universe: All respondents with missing data
Interviewer note: Review the missingness flags for the three key variables: annual payroll, March 12 employment, and Quarter 1 employment. Then, ask questions as appropriate.
I took a look at your AIES submission, thank you for responding. I noticed that you did not provide data for [TOPIC] for your [UNIT(S)]. Can you tell me more about that?
Were the data left off intentionally, and if so, why?
If not intentional, what happened?
Sometimes, when we have missing responses on our surveys, we may have people from the Census Bureau call to collect those data by phone. What do you think about providing missing data over the phone?
How easy or difficult might that be for you?
Universe: All respondents (optional)
Now, let’s take a look at a few screenshots from the AIES. I’m going to pull up the screenshot and share my screen – please let me know when you can see it.
Interviewer note: Display Screenshot 1: Step 2, Reporting Period. Take a look at this screen. In this question, we are asking about the reporting period for this survey. What is your first impression of this screen?
What do you expect will happen on this screen when you answer this question?
What do you think about the layout of this screen?
What do you think about the size and thickness of the font on this screen?
Interviewer note: Display Screenshot 2: Step 2, Fiscal Year Selected. Now, look at this screen. Is it clear what has happened here?
Describe for me what has happened on this screen. What would you do next?
What do you expect will happen if you change your selected answer?
Interviewer note: Display Screenshot 3: Step 1 FAQ Modal Window. Ok, now take a look at this screen. This is what the screen looks like when a user clicks on the FAQ link at the top. What do you notice first here?
What about the font – is it too small, too big, or the right size?
What stands out to you?
Interviewer note: Display Screenshot 4: Step 2: Grid Format. Ok, last one. Take a look at this screen. This is a question from the company-level collection part of the survey. What do you notice here?
How do these questions relate to each other, if at all?
How do you anticipate this screen will work? What do you expect to see?
What are the horizontal lines trying to communicate to you?
Look about halfway down at that double line – why is that there? What is that double line trying to communicate to you?
Universe: All participants
That’s all the questions I have for you today! Do you have any other comments, questions, or suggestions for us?
Thank you so much for your time today.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Melissa A Cidade (CENSUS/EWD FED) |
| File Modified | 0000-00-00 |
| File Created | 2025-08-12 |