2023 AIES Nonresponse Debriefings

Attachment I - 2023 AIES Nonresponse Debriefings.docx

Annual Integrated Economic Survey

2023 AIES Nonresponse Debriefings

OMB: 0607-1024

Document [docx]
Download: docx | pdf



Attachment I


Department of Commerce

United States Census Bureau

OMB Information Collection Request

Annual Integrated Economic Survey

OMB Control Number 0607-1024



2023 AIES Nonresponse Debriefings












Findings and Recommendations from Nonresponse Debriefings for the 2023 Annual Integrated Economic Survey



Prepared for:

Lisa Donaldson, Economy-Wide Statistics Division (EWD)



Prepared by:

Rachel Sloan, ESMD

Krysten Mesner, EWD



Office of Economic Planning & Innovation

Economic Programs Directorate [for external release]

U.S. Census Bureau [for external release]



09/09/2024



The Census Bureau has reviewed the report for unauthorized disclosure of confidential information and has approved the disclosure avoidance practices applied. (Approval ID: CBDRB-FY24-ESMD001-013).



Table of Contents

Tables




Executive Summary



Researchers in the Data Collection and Methodology Research Branch (DCMRB) of the Economic Statistical Methods Division (ESMD) and the Economy-Wide Statistics Division (EWD) conducted debriefing interviews with 32 participants to understand barriers to responding to the Annual Integrated Economic Survey (AIES) instrument, and what non-respondents would need to reduce burden and make it easier to respond to the survey.

Findings included:

  • Respondents often experienced technical difficulties, which caused some of them to not complete the survey.

  • Companies’ records often do not match what the survey is asking for.

  • The survey is very long, which makes it difficult to complete when businesses have other priorities or are short staffed.

Research Objectives



The purpose of the debriefing interviews was to:

  • Evaluate how the instrument performed.

  • Determine which parts of the survey were highly burdensome.

  • Provide recommendations for further research that is needed to improve the AIES.

Research Methodology



The objective of these interviews was to discover issues that debriefing participants experienced when attempting to complete the AIES. Interviews took 10-55 minutes, depending on how much time the participant was willing to speak. The interviews followed a semi-structured interview protocol, found in Appendix A.



PARTICIPANTS

Thirty-two participants took part in the debriefing interviews for the Annual Integrated Economic Survey. This included a variety of companies representing single unit firms (SUs), multiunit firms (MUs), and industries.



Table 1. Companies by sector. Note, MUs were classified into multiple sectors.

Sector Code

Number of Companies*

Description

31-33

6

Manufacturing

42

10

Wholesale Trade

44-45

6

Retail Trade

48-49

6

Transportation and Warehousing

52

7

Finance and Insurance

54

7

Professional, Scientific, and Technical Services

55

11

Management of Companies and Enterprises

56

4

Administrative and Support and Waste Management and Remediation Services

62

7

Health Care and Social Assistance

81

4

Other Services (except Public Administration)

N/A

5

All Other Sectors


*Companies were associated with multiple NAICS.




Table 2. Companies by Number of Establishments.

Establishments

Number of Companies

1

4

2-20

21

21+

7


Twenty-eight of the total participants were recruited from multi-unit establishments, with the majority being smaller companies with 2-20 establishments. Four participants were recruited from single-unit business establishments. Participants held positions such as:

  • President, Vice President, Controller, CFO, Accountant, Human Resources Manager, Finance Director, etc.


The industries in which these participants represented varied and were reflective of the types of establishments reporting to the survey. These included:

  • Management of Companies and Enterprises, Wholesale Trade, Finance and Insurance, Professional, Scientific, and Technical Services, and Health Care and Social Assistance.


Researchers recruited participants using a file which included companies who had not yet completed the AIES. Participants were recruited via email. The debriefings were conducted virtually, by phone and Microsoft Teams. Twelve of the debriefing participants had actually responded to the AIES by the time of their interview, though their response was received after the April 30, 2024 due date. Some of these participants were prompted to answer the AIES because they had received the recruitment email for this research. Of the seventeen participants who had not completed the AIES, fourteen said they intended to complete the survey eventually, while three said they would not be able to complete it. Three participants were not sure if they had completed the AIES already. Respondents are often receiving multiple surveys from the federal government.



LIMITATIONS


As in any study, findings may be limited by the small, non-statistically representative sample used. Since this study was voluntary, those who participated may be more motivated to complete surveys than those who chose not to participate, and so are not necessarily representative of the population. Some respondents had completed or attempted the AIES before the debriefing interview and did not remember details of survey. Additionally, some participants had been very frustrated by the AIES and were only willing to do a short interview.



FINDINGS



Technical Difficulties



Most of the non-respondents had attempted to complete the survey but had experienced technical difficulties with the AIES instrument.

  • Many respondents complained that the spreadsheet was confusing and not user friendly, particularly because they needed to scroll horizontally to view the entire spreadsheet.

    • “I was having trouble opening the spreadsheet. I’m not computer literate. I’m one year away from retirement and it makes me want to retire now!”

    • “Every time I’d go in, but it would always say there was an error. I’d look at it, my colleague would look at it, and there was nothing wrong with it. You’d have to keep scrolling to the right, go box by box, but it kept saying ‘error fails’. I tried two more times, but I couldn’t figure it out…I thought I put everything in the fields and it kept giving an error, so I quit.”

    • “I’m not even positive what I need to input, so that makes that hard. We have a bunch of stores, so I have to write down which row is which store when I’m entering and I have to scroll.”

    • “it’s not readily easy to say this goes all the way across and forever to the right…there’s a lot of information. The way the programmers put it together is well understood by them, but for someone who isn’t familiar or who only does this once a year it’s not as intuitive. It’d be nicer if it had a table of contents instead of just one big long mess of something that goes to the right – like you could click on balance sheet, income statement, payroll, general bank info like facilities, etc. – like different topics. It would be more user friendly.”

    • “The accounting person in my group tried to do this but the questions were very confusing. We did the Economic Census. I am the practice administrator and I am responsible for all finance, staffing, everything in this business with the exception of treating patients is my ultimate responsibility. I’ve done this for 40 years and it’s stumped me that I had challenges with this.”

  • Several respondents attempted to complete the survey but found that their information was not being saved.

  • “All of the instructions are in there but then trying to figure it out…it was difficult to get the first numbers saved. I couldn’t quite figure that out.”

  • “When I started it there was a computer error where they erased all of my info. And then I got an email from Census that there was a glitch and they needed a few days to fix it. And then I went on vacation. And all of that impacted my ability to respond.”

  • “If you get out of it you can lose your information. You have to save every time you leave the survey or lines you’ve entered won’t save.”

  • “It doesn’t leave me logged in long enough and I have to go look and then I have to go log back in every single time. That is frustrating. It’s quadrupling the time it takes me to do this.”



Recommendations:

  1. Conduct additional research to improve instrument usability.

  2. Fix errors in instrument that cause data to not be saved.

  3. Make clear what specific correction is required for the error to be resolved. Jump participants to the question, and use plain language to describe the error.



Matching Records to Survey



Respondents often had difficulty completing the AIES because they do not keep their records in the format requested by the Census Bureau.



Reporting by Location



Many respondents found reporting by location to be either extremely difficult or impossible, as their business records are not kept this way. Having to report by location drastically increased respondent burden.

  • “Data comes from a lot of different places. We have various teams and business units so it’s very difficult. Especially by location. It’s not a very easy task to do. The records are all consolidated, it’s not to that level of detail that the census asks for. It takes a while to get this info and compile it.”

  • “We have a lot of plants and the survey makes me report for each of them and I have to report the same thing over and over. We don’t run like that. I have to do this like 28 times, the same thing over and over. Assets aren’t recorded at the plant level, they are at the corporate level. It’s cumbersome, it’s a pain... AIES was way more difficult than other surveys…This was way more difficult because of the locations. The others aren’t as cumbersome…I couldn’t finish it, even when I had people to help. I don’t want to get fined, but what can I do, I tried!”

  • “If you are asking as a whole, it would be easier to answer – like with ballpark percentages. But when you’re asking in that detailed level, it’s very difficult to answer. And it would make sense for the Census Bureau to send this to the individual companies for completion instead of to just the parent company who has to reach out to all the other companies for their data.”

  • “One office has 4 providers the others have 2 and 2. I was told to try to divide these evenly, and that is not correct.”





Recommendations:

  1. Conduct more research on how to reduce respondent burden.

  2. Increase flexibility with regard to the content and formatting of the files the AIES instrument will accept.

  3. Limit the number of questions requesting “detailed” data.

  4. Make clear the ability to consolidate establishment information.



Reference Period



Some respondents also had difficulty because the survey’s reference period does not match their company’s records. Converting the records into the survey’s reference period is very burdensome.

  • “I never completed …It was a little difficult because the way our accountants give me information is not broken down by quarter. If I remember correctly, they were asking for quarterly stuff and I get it on a monthly basis and that became a problem.”

  • “I can give you annual numbers, but not quarterly and I think that’s what they are asking.

  • “It’s hard to find the data from the right time period. Might have to combine data from multiple fiscal years.”



Recommendation:

Conduct more research on how to reduce respondent burden.



Other Reporting Issues



Other issues included the AIES asking about inventory and revenue in a way that respondents could not easily report, and asking for information that respondents do not track.

  • “Some inventory questions for AIES are confusing. We do not do their accounting this way that it’s asked on the survey. We’ve had to have internal meetings to discuss how to report. Our inventory is based on the gap measure. We have to put 0 for some of the questions because they don’t value their inventory this way.”

  • “The biggest challenges are having the data available in the way that it’s requested. A lot of times I would love if the Census was already in a format of government reporting. I would love if the government agencies got together and get it from others. I find that it’s a lot of the same information that we already give to other government agencies. It seems like government agencies don’t communicate with each other despite already having the data.”

  • “Unfortunately, We don’t have an easy way to get some of the information, especially with the longer surveys that are more involved so it takes us a lot of time. I have to go to multiple systems, IT department, payroll I need to go to HR department, etc.. The person I had doing the surveys earlier this year didn’t have access to that so I had to involve the HR department. It’s extremely time consuming for us.”

  • “We’re having to go to multiple systems on some of the sales data they want categorized and it’s a customized thing that I need to have our IT put together for us. And it’s a very frustrating process. Your survey requests do not match how we record data in our systems.”



Recommendations:

  1. Conduct more research on how to reduce respondent burden.

  2. Allow for formatting flexibility within uploaded files.

  3. Consider allowing respondents to report inventory and revenue in a manner that better matches how their own records are maintained.



Length of Survey



The length of the survey increased respondent burden and frustrated the respondents. Some respondents felt that AIES was significantly longer than other surveys. Several respondents were not able to complete the survey on time because the survey was so long.

  • “I’ve done other surveys that were a lot easier and quicker, and the length of time is a thing in today’s world, and you lose someone if it’s going to take 2 hours to complete. If they could come up with a better type of survey and more user friendly, I think it would have a better response.”

  • “I have to answer for 6-7 different companies” “Its lengthy cumbersome but understandable why we need it. Sales and expenses took the longest.”

  • “AIES was harder than other surveys, there were a lot more questions… It did take a lot of time. picked it up and put it down a lot, probably took 4 hours to do, a lot of it was sending emails to others and waiting for response.”

  • “Still working on it. Its so long and intense takes a lot of time… It's cumbersome and has to do a lot of research. This is an overall issue with the survey.”

  • “It took me a while to complete and that’s why it was late this time. This survey was a lot of detail. They combined a couple of surveys for this survey. It was challenging to complete because we don’t just have one company – we have 6-7 companies and we have to do it for each company.”

  • “Ask about ranges instead of exact numbers and share information with other federal agencies so we aren’t answering the same questions for different agencies.”



Recommendations:

  1. Conduct more research on how to reduce respondent burden.

  2. Reduce the number of questions on the AIES, particularly those that are especially detailed.



Company Issues



Many respondents were not able to complete the survey on time due to internal company issues.



Staffing Issues



Several companies were not able to submit the survey on time because they did not have adequate staff to complete the survey.

  • “Things have been difficult, we terminated many staff, and the accounting department is shrinking.”

  • “A payroll person would have done part of the work on this, but one quit and one was fired so I’m having to do all of the work.”

  • “We are short staffed and it’s not a priority for me to enter information into government surveys. Doing my actual job is the priority and I’m very far behind on everything.”

  • “The person who used to complete the Census surveys is no longer with the agency, and it’s now my role to complete them and I’m new in this role as of about 3 months ago.”



Recommendation:

Conduct more research on how to reduce respondent burden.





Prioritizing Other Work



Businesses often have to prioritize other work, and so are not able to complete the survey on time. When asked if there was a more convenient time of year to complete the AIES, some respondents said that summer or fall would be a better time. One respondent pointed out that the current AIES deadline is too close to tax season.

  • “We have not had time because of the merger. This time of year is also busy season.”

  • “Also going through audit and tax stuff… still trying to get 2024 budget finalized.”

  • “Slows down again in November and December. It’s funny that they want this done April 30, any company this size is doing taxes. In January and February, year end numbers wouldn’t be done by then. The timing is terrible. If it was November 30 or December 15 we would have data for the prior year and tax returns done.”

  • “Anytime post April 30 to end of calendar year is post busy season and would be an ideal time for doing surveys.”

  • “June and July are the best time. March is a very busy time because we have back to back year end and Q1 reporting.”

  • “Summer is probably best time period for surveys. January has the year end close, and we do audits. March and May are for getting info together for Medicaid and Medicare.”

  • “Slowest time is Q4. Q1 is wrapping up year end stuff, Q2 is preparing tax stuff. After 9/15 is slowest because extensions are done.”

  • “September-October would be a better time to do surveys.”

  • “Immediately after March is the end of the fiscal year, but summer is usually okay for doing surveys.”

Recommendation:

Conduct further research into which time of year would be less burdensome for respondents. Consider changing the survey due date to not interfere with tax season or end of quarter reporting.



Issues with Contacting Respondents



Some respondents did not initially receive the messages about the AIES, either because the message was sent to spam, or it was sent to someone who previously responded to Census Bureau surveys at the company who is no longer in that role or at that company.

  • “Emails were sent to my predecessor.”

  • “I never received any letters. The emails went through my spam folder, and I periodically look through that and that’s how I found them.”

  • “I think that our office has moved, and our mail is forwarded, I don’t know how to correct it with government agencies that we hear from every now and then. I don’t know how to get them to change the address. “

  • “One way – if it looks too much like an ad…we get a lot of “you must respond to such and such” and in tiny print it will say ‘not an official document’ – it’s not your fault, but we do get those. I wonder if there were letters sent and I didn’t get it because someone thought that maybe it was a scammy postal mail letter. As for email, I might have listed census.gov as subscriptions for the future. But, to be honest, I don’t understand our spam filter because sometimes things go through just fine and sometimes it catches everything. But I think our IT Department is trying to keep as much spam out as possible.”

  • “If the person who receives it moves on, then you may send the email to her email address which will not be active anymore. And then Census won’t receive a response and then would likely notify us to let us know that it’s due. Ideally it would be nice if you sent it to two different people as a back-up.”

  • “I did not receive any letters in the mail. I did receive emails but they were in my spam box which I found because I check my spam box periodically.”

  • “It would be helpful if the Census Bureau had an updated point of contact. We actually created an operations email and it would be helpful to streamline that to that general email as opposed to going to a particular person.”



Recommendation:

Consider ways for respondents to report that a new contact person will be responsible for surveys.



Companies No Longer in Business



A few respondents received surveys for companies that were no longer in business. They did not have a way to indicate this on the survey, and were still counted as non-respondents.


  • “I received a letter for the Annual Integrated Economic Survey…but that company merged with another company in 2022 so I had nothing to report for calendar year 2023 for that company. We were a smaller agency merged into a bigger agency. I contacted the Census Bureau and they said that they were going to mark it as merged into a bigger business. I have merged onto that and did it for the bigger company. I thought I completed it the way they told me to. I had someone on the phone and I did what they told me to.”

  • “Maybe one of the first questions should be: does this business still exist or are you still in business? For the company that the letter was addressed to, I would have said no, but the survey did not allow me to do that. I could put some information in, but there were required fields and it would not let me submit. If I could have had the option to say no from the get go, then I could have skipped everything else and submitted if that was an option.”



Recommendation:

Consider ways for respondents to report that the company is no longer operating, without needing to answer other questions or being considered delinquent.





Communications



More than half of the non-respondent debriefing interviews participants were asked what they think the information in the AIES is used for and the purpose of it. Additionally, these participants were asked to read and provide feedback on two draft letters, the Office of General Counsel (OGC) “Light” Letter (AIES-L4L) and the Experimental “Dear CEO” Letter (AIES-ECSL1). These letters can be found in Appendix C and Appendix D of this report. This section details participants’ understanding of the purpose of the AIES as well as feedback obtained on the two letters tested during the nonrespondent debriefing interviews.



Purpose of the AIES



When participants were asked what they think the information on the AIES is used for and the purpose of it, we received a variety of responses, ranging from no understanding to minimal understanding.

  • Many participants had no understanding of what the information on the AIES is used for and the purpose of it.

    • “I have no idea. I’d have to go back and look. It might say so in the letter.”

    • “I couldn’t even wager a guess.”

    • “Not a clue. Maybe just tracking the way like different costs are moving over time. A lot of the time I’m not analyzing it, I’m just trying to get the information in and just move on.”

  • Some participants had minimal understanding of how AIES data is used and the purpose of it. Of note, these participants often express low confidence when articulating their understanding.

    • “I haven’t really put much thought into it. Whatever the government is keeping the information on…I don’t know. I’m sure it’s policy information and general direction of the economy.”

    • “I would imagine it’s to judge the direction of the economy and the industry as a whole.”

    • “I honestly don’t know. I assume some of the payroll information specifically on number of employees at certain time of year helps gauge employment at a certain time of year, maybe sales information is used to compare one year to another year, increase in services and goods in a category, etc. That’s off the top of my head. I honestly don’t know specifically what they’re used for.”





Recommendation:

Given the amount of time and level of effort required to complete the AIES, it would be helpful for respondents to understand the purpose of the survey. Additional research on effectively communicating the purpose of the survey to respondents is warranted.








Office of General Counsel (OGC) “Light” Letter (AIES-L4L)



Participants who read and provided feedback on the Office of General Counsel (OGC) “Light” Letter (AIES-L4L) were asked questions pertaining to the following: their initial observations/impressions, the purpose of the letter, and what they would do if they received this letter.

When describing their initial observations/impressions and what stood out to them about the letter, participants often noted that response to the survey is required by law as well as the penalties or consequences for not responding. Some participants mentioned that the items in bold caught their attention.

  • “That it’s threatening me. You’re failing with the law, we’re going to penalize you, and like I don’t have anything better to do than respond to a mandatory request.”

  • “It said it was a requirement of the federal government, there are consequences for not doing it. And then the online reporting being triggering for me because it was so not user friendly and frustrating.”

  • “I like the fact you’re saying – the response is required by law and that there is the potential for penalties, but you’re highlighting that this isn’t what you’re trying to do, but rather that the goal is to get the information.”

  • “Required by law being highlighted. If it wasn’t required, we wouldn’t do it. Reiterating it’s required by law is what really matters.”

  • “The required by law. That to me makes me want to make sure I do it.”

  • “There’s just a lot of words. I don’t know what’s important on there. To me everything kind of seems [important]. I don’t know how to explain it, I guess, but to me the first thing that pops out is the required by law because it’s in bold and all caps.”

  • “I would say that it’s the bold writing – Failure to provide mandatory response. All these bolded things – required by law, penalties up to five thousand dollars – that’s what I see here.”



The purpose of this letter was abundantly clear to participants. Participants overwhelmingly understood that the purpose of the letter is to inform them that their response to the survey is past due and to remind them that they need to complete the survey because it is required by law.

When asked what they would do if they received this letter, participants mostly indicated that it would elicit action on their part. Many participants indicated a desire to comply with the survey request, however, re-iterated time and staffing constraints. Some said the following:

  • “I would sign in and start answering the survey which is exactly what I did.”

  • “If I get the letter, I just put the due date on top (on paper copy) and put on my desk. If it’s easier and I have time, I just do right away.”

  • “I’d probably work on the survey until 2-3 hours past our normal day to get it done because I have no people to help.”

  • “If I got this and it hadn’t been finished, then I would get right back to it.”

  • “I would just continue to try to work on the survey and try to get it done as soon as possible or contact someone if needed.”

  • “I would follow the instructions at the bottom there – register and see what I had to do to get it completed and I probably would have ended up calling somebody for help.”

  • “In all honesty, I get these letters and they pile up on my desk until I can find someone or the time to respond to them.”

  • “Respond fairly quickly.”

  • “I guess I would just try to comply with it as much as possible.”





Recommendation:

Additional communications research, including testing the wording of messages, is recommended. It would be helpful to better understand how to effectively communicate the importance of responding to the survey as well as potential motivators to response for AIES respondents.



Experimental “Dear CEO” Letter (AIES-ECSL1)



Participants who read and provided feedback on the Experimental “Dear CEO” Letter (AIES-ECSL1) were asked questions pertaining to the following: their initial observations/impressions, the purpose of the letter, and how they felt about and what might happen if we sent this letter to the most senior financial officer we could identify at their company.



When describing their initial observations/impressions and what stood out to them about the letter, participants often mentioned that the tone of this letter was gentler than the previous letter they were shown. Of note, many of the participants interviewed are the most senior financial officer at their company and several participants had additional suggestions. Some feedback on first impressions obtained from participants included:

  • “It’s less threatening and sounds more professional.”

  • “Past Due on the top so immediate attention is required.”

  • “It’s less serious than the first one. I expected it to be worst but it doesn’t call out penalties and it feels less serious.”

  • “It’s better written than the other one because you don’t come out of the gates swinging with your threats.”

  • “I like this letter better than the other letter to be fair. It was easier to read. The other letter was more like we’re going to fine you if you don’t do it, but this one is like you should do a better job of designating someone to do this because nobody is responding.”

  • “I am the most senior financial officer at my company. I get the emails and the mail, but sometimes emails kick out things to spam. Mail sometimes gets lost and not to the right person. The friendly nudge reminder is nicer. It seems very similar to the other letter you showed me. The senior financial person should be getting this at most companies, but maybe a President could get this. Sending to a nonfinancial person who effectively manages that person would be helpful.”

  • “I am the most senior financial person at this company. This one seems a little better than the last one. I just think that there seems to be a disconnect of how. I think the government thinks I can just go about and get this information and plug into a survey. Just designate a person – but when it’s HR and payroll – it’s multiple people and someone needs to coordinate and then get into a format and submit it. From reading this letter it gives me the impression that just designating a person is an easy thing to do, but it seems lost on the government how much effort is required on our part and how much coordination is required.”

  • “It feels like a nice like – hey you’re late and someone needs to be identified to do this report. It tells you where you can get the information that you’re looking for on there – and it’s quite nice. For someone new, it’s helpful to say where you would find this information.”

  • “It’s still worded friendly. Since I’ve seen the previous letter which is a past due letter and, if I think about this from your perspective and what the Bureau is trying to accomplish and I look at it from my perspective. if the first letter is going to contain the information about fines then I would want this letter to say the same thing OR the first letter should just be about – our goal is to work with you to gather the information – the senior financial person should know that there are potential fines even though that isn’t the goal. I know that my CFO would have some understanding because he started as an accounting manager in the company and worked his way up as well so he is familiar and knows that these surveys are out there. I don’t know if at some companies there are CFOs who may never have seen a survey and don’t know the level of effort required to complete them. You’re letting them know that response is required by law but if they aren’t familiar with then they may not know what the consequences are for being non-compliant. It’s almost like to me the first past due letter is going to the person you expect to be doing the submission – that one if you’re only going to have the consequences of the fines and penalties and letter then I would put that in the CFO letter because you’re escalating. The second letter is an escalation letter saying we’ve done everything we can but we’re not getting a response.”

  • “That you’re giving us an opportunity to complete the Census before you go into the threatening part. This is not as threatening as the other letter you showed me first. It’s a much nicer letter. It’s a more proactive letter. When you ask for someone’s cooperation like this… in a more proactive approach, you get a better response.”





The purpose of this letter was abundantly clear to participants. Participants overwhelmingly understood that the purpose of the letter is to inform them that their response to the survey is past due and to remind them that they need to complete the survey because it is required by law. In addition, some participants noted that the tone of this letter sounds more like a partnership or request for cooperation rather than a threat.



When asked how they would feel about and what might happen if we sent this letter to the most senior financial officer we could identify at their company, this letter was well received and the feedback was positive.

  • “This would be forwarded to the department head and they would send to the person responsible for completing the survey. I think it would be the right thing to do to send to the senior person if it hadn’t been complete.”

  • “They’d hand it to me and ask what needs to happen to get it done. I’d feel fine with this going to the senior financial officer at my company because it’s a factual letter. I will say that the first letter – it’s not the Census Bureau’s fault that I’m so short staffed, but I’d be losing a night with my family and I’d be very resentful. With the second letter I saw, I know I have to get it done but it does not have to be today.”

  • “It would go to the CEO and then they would contact my boss and my boss would come to me to ask why it hadn’t been done and then I’d have to give an explanation – I hadn’t seen it or that I’m in the process, etc. I’d be fine with it.”

  • “The most senior financial officer at my company is the CFO, who I report to directly. At my company, there will definitely be action taken because my CFO would come to me and say what’s going on? That’s why to me that this letter should indicate that you have tried to contact us through the various modes – email, USPS postal mail, phone, etc. – and not been successful because if someone escalates something to me it’s helpful to know that as part of the escalation that that person has tried X,Y,Z and they’re coming to me because they’re not getting anywhere. It’s helpful to know the full side of the story before going to staff. If my CFO gets this, he will come to me and say why am I getting this letter? From his perspective, he’d be like – ok, are you working on it? It will have more meaning as a past due notice for him if it tells him that you’ve tried multiple ways and times to reach out to us and you didn’t get a response. Whoever receives this needs to understand why he’s getting this – that we aren’t getting response from our communication attempts and that we need his help to get the survey completed.”

  • “It might be helpful to send this to a senior non-financial person like the President because the more senior financial person at a small company has probably already received the requests and may not be addressing them. It depends on the size of the company. President or CEO may be more successful if the financial person isn’t responsive.”

  • “I am the most senior financial officer at my company. It would be me and I would access the survey and would want to go in and close the case. I would want to make sure that our response is recorded – whatever it is.”

  • “I can tell you exactly what would happen at our company. The director of finance would get it and then give it to me.”

  • “I am the most senior financial officer at this company. I need to balance out my department’s priorities. If we have other priorities to run our business and supply our banks and vendors and ownership – we’re a family company – then that’s going to take a priority.”

  • “If you’re calling the most senior financial officer that would be the CFO and they wouldn’t want to look at a letter like this – whether they’re upset that someone hasn’t filled or they just don’t want to bother with this. There is an appropriate level of person to send this to, but I just don’t know if the most senior financial person would be the most appropriate. Like if there was a Director. I don’t know how much data you guys have on who’s filling out the survey and those things. Technically I started filling out some of these surveys when I was an associate and now I’m a manager. I think the previous manager got some sent to her and then I was filling them out.“

  • “I feel fine about the letter. There’s nothing alarming about it. And I think it would be received as it’s something that needs to be taken care of quickly.”



Recommendation:

Additional communications research, including testing the wording of messages, is recommended. It would be helpful to better understand how to effectively communicate the importance of responding to the survey as well as potential motivators to response for AIES respondents. Furthermore, it would be helpful to do additional research to better understand the most appropriate recipient of such a letter.











Appendix A


2023 AIES Participant Debriefing Protocol

Purpose:

Researchers in the Census Bureau’s Economic Statistical Methods Division (ESMD) and Economy-Wide Statistics Division (EWD), and with RTI International, will conduct participant debriefing interviews in support of the 2023 Annual Integrated Economic Survey (AIES). These interviews will cover three main topics: response process, communications materials, and instrument performance. Participants will represent both respondents and nonrespondents to the 2023 AIES.

Not all questions in this protocol will pertain to all participants, and some modules are optional depending on the length of the interview and firm characteristics. Numbered questions are priorities, and bulleted questions underneath are optional probes.



Research Questions: The research will be guided by the following research questions:

  1. How are respondents reporting to the AIES?

    1. What are respondents' overall impressions of the survey?

    2. What is the ideal length and timing of the field period for respondents?

    3. What are the unique reporting needs of companies with locations outside of the 50 U.S. states?

    4. What are the barriers to reporting (non-respondents)?

  2. What is respondents’ feedback on the content and accessibility of respondent communications?

    1. What support materials are respondents using when reporting to the AIES?

    2. Are the content selection tool and summary document sufficiently supporting response?

    3. What are respondents’ impressions of letters and emails?

  3. What is respondents’ feedback on instrument performance and response burden?

    1. What feedback do respondents have on updates to the instrument since the last round of research?

    2. What are the respondent-reported reasons for item nonresponse on the AIES?

    3. What are respondents’ impressions of screen layout, font, and other features of the online instrument?


Informed Consent: Respondents will be asked to complete a consent form electronically before the time of the interview.


Materials Needed:

  • Electronically signed consent form

  • Letters: AIES-L1, AIES-L3, AIES-L4L, AIES-ECSL1

  • Emails: AIES Initial Email; AIES Due Date Reminder Email; AIES Postcard

  • Websites: AIES Interactive Content Selector Tool; AIES Front Page webpage; AIES Information for Respondents webpage; AIES FAQs webpage

  • Respondent recruitment and firm information, including: Contact info; response status; establishments in Puerto Rico; establishments in manufacturing; single/multi-units; item missingess; pilot participation flag; number of establishments; number of industries



Method: We will conduct the interviews by Microsoft Teams (or phone).



Expected length of interview: 1 hour (60 minutes) maximum



General probes that may be used throughout the interview:

  • Were these data easy to access?

  • What else can you tell me about this?

  • Can you tell me more about that?

  • How confident are you in that response?

  • What looked unclear or is confusing here?



Introduction (5 minutes)

If necessary: You should have received a link to a consent form from our recruiter via email. Did you have a chance to review and sign that yet?

[IF YES] Did you have any questions about any of the information presented there?

[IF NO] Please open up the link now and review the form, then sign it. Take your time and let me know if you have any questions.

Thank you so much for agreeing to talk with me today!

As part of the roll out of the new Annual Integrated Economic Survey, we are following up with some companies to learn more about the processes you may or may not have used to complete the survey.

I am working with the Census Bureau to make sure that the Annual Integrated Economic Survey is performing as expected and to get feedback about ways to improve the performance of the survey instruments. I’m talking with you today because you are identified as the person who provided response to the AIES this year. When I refer to “the AIES survey” during our conversation I’m talking about the Annual Integrated Economic Survey that you responded to about your business within the past few weeks.

My job is to improve the AIES survey. I didn’t write the survey questions, so you do not have to hold back when telling me your feedback. Please be candid and frank in your responses. Our interview is being conducted under the authority of Title 13, which means that your responses are confidential, and neither your name nor the name or identifying information about your company will be included in any of our findings.

Do you have any questions before we get started?

I’d like to record our session today so that when I go to analyze the results of these interviews, I can use the recording to pick up on anything I may have missed in my notes. Do I have your permission to record our session today?

[Turn on recording]

[State respondent ID number and date, repeat consent on recording: Do you agree to participate in the interview today? Is it okay if I record our conversation?]

Topic 1: Response Process



Module 1: Warm up (5 minutes)

Universe: All participants

  1. What is your role in the company?

  2. What is your role in the process for responding to Census Bureau surveys in general? How long have you been in this role?

    • How do you find the data you need to answer Census Bureau surveys?

      1. Do you work with anyone else to get the data you need?

      2. Do you access any reports to get the data you need?

    • How easy or difficult is it for you to find all the data you need for Census Bureau surveys? Why is that?

    • How easy or difficult is it for you to enter the data into the survey once you have the answers you need? Why is that?

  3. Tell me a little bit about your business. What types of goods or services does this business provide?




Module 2: Nonrespondents (35 minutes)

Universe: All nonrespondents

  1. Tell me a little about yourself…

    • Whose job is it typically to respond to Census Bureau surveys?

    • What recent changes to your job or at your company might impact how you respond to Census Bureau surveys, if any?

    • What challenges do you specifically face with answering survey questions?

    • What do you think the information we ask for in this survey is used for?

  2. We hear from businesses that sometimes our communications simply do not make it to the right person at the business. Do you usually receive the letters and emails that the Census Bureau sends out about surveys, or is it someone else at your company?

    • How do you get your work-related mail? What is the process, and how much time does it usually take?

    • What are some ways that communications do not reach you in a timely way? That is, are there ways that we could address our letters or flag our emails to ensure that they get to you in enough time to respond to our requests?

    • If our communications ever went to the wrong person, how would you be notified, if at all?

  3. In preparing for our time together today, I noticed that you did not respond to our request for the Annual Integrated Economic Survey. So far, we would have sent letters and emails about the survey to your company. Can you tell me about that?

    • Are there things about your company that pose a challenge for you as you complete the AIES?

      1. What should we keep in mind about response for companies like yours?

    • Were you able to look through the survey? What were your thoughts?

      1. Are there other surveys from us you’ve found challenging?

    • What, if anything, could we at the Census Bureau do to support you in responding to the AIES?

  4. While I’ve got you on the phone, I’d like to run a few letters past you for your feedback. These are about the AIES.

    • Interviewer Note: Enter the AIES-L4L into the chat.
      Take a look at this letter – not all companies receive this one. Tell me what it is all about.

      • What is the first thing you notice about the letter?

      • What is the purpose of this letter?

      • What stands out about this letter?

      • What would you do if you got this letter?

    • Interviewer Note: Enter the AIES-ECSL1 into the chat.
      This last one is different because it would not be sent to you, it would be sent to the most senior financial officer we had on file for your company. I want to stress that we have not sent this letter out, and we are not going to send this letter. We are just looking for feedback on a hypothetical letter like this one being sent to the most senior financial officer we can identify. In this hypothetical situation, we would have reached out to this company multiple times by email, mail, and phone, and gotten no response.
      Tell me what it is all about.

      • What is the first thing you notice about the letter?

      • What is the purpose of this letter?

      • What stands out about this letter?

      • How do you feel about this letter being sent to the most senior financial officer we could identify at your company? What might happen if we sent a letter like this to the senior financial officer?



Wrap up/Debriefing (5 minutes)

Universe: All participants

  1. That’s all the questions I have for you today! Do you have any other comments, questions, or suggestions for us?

Thank you so much for your time today.


Appendix B

Recruitment Email for 2023 AIES Participant Debriefing



Subject: Action Requested: Meeting with U.S. Census Bureau



Hello,

I hope this message finds you well. I am a survey researcher at the U.S. Census Bureau in the Economy-Wide Statistics Division. You are listed as the contact for your company, and we are reaching out to gather feedback about the Annual Integrated Economic Survey (AIES).  The meeting should take less than 45 minutes to complete.  Below is a link to select a date and time to meet with us. No advance preparation required.

Follow this link to Schedule a Meeting:
Schedule a Meetng

Or copy and paste the URL below into your internet browser:
https://research.rm.census.gov/jfe/form/SV_0dKujegfYUyQ89o?Q_DL=qFpmlUD3wXUxWaW_0dKujegfYUyQ89o_CGC_1g7vjOVTm6K3uur&Q_CHL=email

If you have any questions or concerns, please feel free to contact me via e-mail or phone. Your participation in this research is voluntary and invaluable.

Thanks in advance for your consideration.

Krysten Mesner, Statistician
Economy-Wide Statistics Division, Office of the Division Chief
U.S. Census Bureau
301-763-9852













Appendix C: AIES-L4L – Office of General Council (OGC) “Light” Letter

N OTICE OF FAILURE TO PROVIDE MANDATORY RESPONSE

Our records indicate that we have not received your 2023 Annual Integrated Economic Survey response. If you have submitted your survey within the past few weeks, thank you for your participation. You may verify your filing status by logging into your account at https://portal.census.gov.

If you have not yet reported, we remind you that your response is REQUIRED BY LAW. Title 13, United States Code, Sections 131 and 182, authorizes this collection. Sections 224 and 225 require your response. Section 9 requires that we keep your answers STRICTLY CONFIDENTIAL.


Although Title 13 (Section 224) and the Sentencing Reform Act of 1984 (18, U.S.C. 3559 and 3571) allow for possible prosecution of responsible officials and penalties up to $5,000 (and still require response), the U.S. Census Bureau prefers to work cooperatively with businesses like yours to ensure we gather and distribute reliable statistics about the U.S. economy. We ask for your help in meeting that goal, recognizing that we are relying on your valuable time and effort to comply with this request.


Online reporting is easy, with a secure account setup necessary to begin the reporting process. For assistance with completing this survey or for a list of the surveys that are covered under this new collection, please visit census.gov/aies/information or call our customer helpline at 1-800-681-3012, Monday through Friday, 8:00 a.m. to 8:00 p.m. Eastern time. For a preview of the questions in this survey, visit census.gov/aies/questionspreview/.


Authentication Code:

Due Date:


Your timely response will save tax dollars by reducing the need for additional mailings and telephone calls. Please report within 10 days, following these steps:


  1. Register OR sign in at https://portal.census.gov

  2. Add your authentication code OR locate this report under “My Surveys”

  3. Report by clicking on “REPORT NOW”


After reporting, we invite you to review the useful data and analysis tools available to aid and inform businesses such as yours by visiting the Census Bureau’s website at https://www.census.gov/aies/information.


Thank you in advance for your time and participation.


Sincerely,


Nick Orsini

Associate Director for Economic Programs

U.S. Census Bureau

The Office of Management and Budget (OMB) approval number for the Annual Integrated Economic Survey is 0607-1024.

Appendix D: AIES-ECSL1 – Experimental “Dear CEO” Letter





PAST DUE NOTICE


Our records indicate that we have not received your company’s 2023 Annual Integrated Economic Survey. This survey is mandatory and requires your immediate attention.


This survey includes questions about your company and all the locations for your company. Most questions can be answered using your general ledger, but some are about payroll and human resources. Please designate a person at your company to be responsible for answering this survey and complete the survey as soon as possible. This designated person will use the information below to access the survey.


Authentication Code: XXXXXX

Shape1

DRAFT –

FOR RESEARCH ONLY



  1. Register OR sign in at https://portal.census.gov

  2. Add your authentication code OR locate this report under “My Surveys”

  3. Report by clicking on “REPORT NOW.” You can return to your account over multiple sessions to complete the survey.




YOUR RESPONSE IS REQUIRED BY LAW and will be kept strictly CONFIDENTIAL. The U.S. Census Bureau is authorized to collect this information under Title 13, United States Code (Sections 131 and 182). The same law requires that you respond (Sections 224 and 225) and assures the confidentiality of the information you provide (Section 9). The Census Bureau is not permitted to publicly release your responses in a way that could identify your business, organization, or institution, and use your responses only to produce statistics.



For assistance with completing this survey or for a list of the surveys that are covered under this new collection, please visit census.gov/aies/information or call our customer helpline at 1-800-681-3012, Monday through Friday, 8:00 a.m. to 8:00 p.m. Eastern time. For a preview of the questions in this survey, visit census.gov/aies/questionspreview/.


Thank you in advance for your time and participation, and for helping the U.S. Census Bureau measure America’s people and economy.


Sincerely,



Lisa E. Donaldson The Office of Management and Budget

Chief, Economy Wide Statistics Division (OMB) approval number for the Annual Integrated

U.S. Census Economic Survey is 0607-1024

Appendix E

About the Data Collection Methodology and Research (DCMR) Branch

The Data Collection Methodology and Research (DCMR) Branch in the Economic Statistical Methods Division assists economic survey program areas and other governmental agencies with research associated with the behavioral aspects of survey response and data collection. The mission of DCMR is to improve data quality in surveys while reducing survey nonresponse and respondent burden. This mission is achieved by:

  • Conducting expert reviews, cognitive pretesting, site visits and usability testing, along with post-collection evaluation methods, to assess the effectiveness and efficiency of the data collection instruments and associated materials.

  • Conducting early-stage scoping interviews to assist with the development of survey content (concepts, specifications, question wording and instructions, etc.) by getting early feedback on it from respondents.

  • Assisting program areas with the development and use of nonresponse reduction methods and contact strategies.

  • And conducting empirical research to help better understand behavioral aspects of survey response, with the aim of identifying areas for further improvement as well as evaluating the effectiveness of qualitative research.

For more information on how DCMR can assist your economic survey program area or agency, please visit the DCMR intranet site or contact the branch chief, Amy Anderson Riemer.



File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorKrysten Mesner (CENSUS/EWD FED)
File Modified0000-00-00
File Created2026-01-22
© 2026 OMB.report | Privacy Policy