Various Economic Area Pretesting Activities

2

MEETING AGENDA

1. 20-30 minutes – Introductions, Purpose of the Research, and Background Information.

2. 40-60 minutes – Task Analysis

017 Economic Census Electronic Data Collection Task Analysis Exploratory Interviews

Introduction

• Introduction of Census employees

• Introduction of company employee(s)

• Audio taping

  • Permission to audio-tape discussion? Have R sign consent form.

  • Before we get started: I'd like to audio tape this interview, so I don't have to rely on my memory later. This session is confidential. Only persons connected with this project will have direct access to your tape. If that's all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

• How long have you been with the company?

• What is your title/role?

• What are your major responsibilities?

• What are your government reporting responsibilities?

• What other government surveys are you responsible for?

• Give the respondent their reporting calendar (if available).

Purpose of the research

• Develop a way to collect economic census data through the Web that meets the needs of the respondents.

• Identify ways to collect the data easier and faster, if possible.

• Ensure that whatever tool we develop will not add burden.

Background Information

• What surveys have you reported electronically to?

• Our records show that your company has been reporting on the Web to the XYZ surveys. Do you know about these? Have you had any involvement with them?

• What has been your experience with using either our Web or software instruments to gather and report your information?

• Have you reported through the Web for other government surveys?

• If so, were there any features from those Web surveys that you particularly liked? Anything that you felt worked well?

• What have been some of the positive aspects of reporting on the Web?

• What have been some of the negative aspects of reporting on the Web?

• Can you name some of the features of the Web survey(s) that didn’t work well?

• Are there things that could be done to address the problem you encountered with these Web surveys?

Task Analysis

I’d like to understand how our forms move around your company. If you remember your experiences with working on the Economic Census, please share them as we ask the following questions.

Data Gathering & Management

• In a perfect world, where our data collection was designed for your company alone, how would it function? What would it look like? Please be as specific as possible about what you would need and how it might look.

• What is the first thing that you would do, or did do, if you received a stack of forms for each of your locations asking for these pieces of information (show them a list of major data items that we would be requesting on an economic census form)?

• Did/would you need to work with others inside your company to obtain this information?

• How did/would you request this data? Do you pass along paper copies or PDFs? Spreadsheets? Did you provide any other information with the forms such as the instruction sheet associated with the form or any other instructions that you developed?

• Are you making requests for data from different people at the same time (i.e, person 1 is working on questions ABC at the same time that person 2 is working on questions XYZ)?

  • Is completion of any items dependent on getting responses to other items from other people? E.g., do you need to compile multiple responses, or manipulate them in some way? If so, which ones?

• Are any of the people that you request data from working with each other to gather this information?

• Is there a need to restrict the access to data for the different people gathering it? In other words, do you want to keep the information that person A is gathering confidential from person B?

System input

• How many different information systems will you need to access in order to gather these types of information together? (If more than one) Are they structured differently? Do you have access to all of the systems?

• What are the names of the information systems you use?

• How do you retrieve the various data items? That is, do you access summary documents for other purposes, perform database queries, etc.?

• Are there any major changes anticipated for the near future in your information systems? Would these changes have any effects on how you/others would gather this information? How so?

• Are there any ways that we could automate your process of gathering the data?

Reconciling locations

• When you get survey forms assigned to all of your locations how do you go about reconciling our list of locations to the locations that you have?

• How do you identify locations that are new? How do you handle locations that have closed?

• Is there a tool or feature that could be developed to assist you with this task?

Exporting/Importing/Formatting Data

• What was/is the most effective way for you to provide data for multiple locations on a variety of topics?

• Are there any tools or file formats that you frequently use to provide data either externally or internally, for example XBRL, XML, Spreadsheets, etc.?

• Some of our instruments offer a spreadsheet option for gathering and reporting data. Would this be useful for gathering this type of information?

• What would a useful spreadsheet look like?

• What information would you need to have included on the spreadsheet to help you collect the requested information?

• What would be the best way for you to access this spreadsheet?

• When using a spreadsheet, what would be the best way to submit that information back to us?

• Would you need any additional support information to assist you in working with the spreadsheet (i.e., a copy of the form/questions, instructions)?

• How would you get the data onto the spreadsheets?

• (if they have responded to the COS, ASM, or Economic Census) Did you key your data directly into the form or did you input the data some other way? Did you experience any issues?

• If they start discussing issues that they experienced with importing, ask them the following probes:

  • Why did you choose to import your data?

  • What was your overall impression of the importing feature?

  • What were some of the positive aspects of importing?

  • What were some of the negative aspects of importing?

  • Did importing save you any time or resources?

  • Did you use the instructions to help figure out the importing process? If so, were they helpful?

  • Do you have any recommendations for how we could improve the importing functionality?

  • If we moved these surveys to the Web, would you still need importing to be a part of the process? How would you envision importing to work within a Web environment? Are there any changes/improvements that you would recommend?

Help

• Have you ever accessed additional information or help within a web survey instrument?

• What is the best way that a system could display this information?

• Where would you go to find this information?

• What would it look like?

• What types of information would you want to have built into a web survey?

Editing

Most electronic surveys include messages that notify you when there is an issue with the information that you have entered. For a survey like the economic census, would you want there to be checks on the data?

• What sort of checks would be the most useful?

• When would you like the checks to occur?

• How would you like to see that information communicated to you?

• Are there any other features that could be included in a Web survey to help you check the data that you are providing?

Reviewing the Data

• Do you typically do a review of the data before it is submitted?

• How do you go about doing this?

• Is anyone involved besides yourself in the review? If so, how do you provide the data to them?

• Are there any tools that could be built into a Web instrument that could assist you with any review that you conduct at this point?

Other:

• Do you want to keep copies of what you submitted for your records?

• In what format would you want these copies to be?

• Are there any other features in Web surveys or data collection instruments that we should ensure are incorporated into a future instrument?

Thank respondent(s) for his/her (their) assistance.

4

2017 Economic Census Electronic Data Collection

Respondent Debriefing - DRAFT

12/18/13