Respondent Debriefings for SURVEYOR Electronic Data Collection Tool in 2007 Economic Census

Surveyor Research

Phase 3

Surveyor Debriefing

Meeting Agenda

Section 1 – Introduction

Section 2 – Respondent Background Information

Section 3 – Overall Software Probes

Section 4 – Welcome View

Section 5 – Inbox View

Section 6– Form View

Section 7– Workbook View

Section 8– Errors / Warnings View

Section 9– Exporting / Importing

Section 10– Wrap-Up

Section 1 - Introduction

Introduction of meeting members

• Introduction of Census employees

• Introduction of company employee(s)

• Audio taping

• Permission to audio-tape discussion? Have R sign consent form.

Before we get started I'd like to audio tape this interview, so I don't have to rely on my memory later. This session is confidential. Only persons connected with this project will have direct access to your tape. If that's all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

Section 2 – Respondent Background Information

• How long have you been with this company?

• What is your title/role?

• What are your major responsibilities?

• What are your government reporting responsibilities?

• Can you remember the names of other government surveys that you are responsible for?

Past Electronic Reporting experience

• Besides reporting electronically to the Economic Census, what other surveys have you reported electronically to? Have you ever used Surveyor in the past?

• How long have you been reporting electronically to other surveys?

Section 3 – Surveyor

• What made you choose to report electronically to the 2007 Economic Census?

• Do you recall receiving a handout labeled “Electronic Reporting is Easy as 1-2-3” when you got the notice about the Economic Census? If so, what was your overall reaction to the flyer? Did you find it useful? Do you have any recommendation for improving it?

• For past Surveyor users - Did you notice any changes in Surveyor from the last time you had used it? What was your initial reaction to these changes?

  • What were some of the major differences that you noticed?

  • Were there any changes that were positive? Negative?

• Overall, what were the best features of the software?

• What were some of the features that you struggle with the most?

• In general, what have been some of the positive aspects of reporting electronically?

• What have been some of the negative aspects of reporting electronically?

Section 4. Welcome View

• Did you read the information in the Welcome View?

• If so, did you read it when you first opened the software?

• Did you ever return to this screen later on?

• Did you view any of the other documents mentioned on the Welcome screen – Reporting Guidance, Tip Sheet, Help, or census.gov/bhs? If so, were they helpful?

• If not, what sort of information would you expect from these documents?

• Do you think that the information presented on this screen is useful? Is it too much information or not enough for a Welcome screen?

• Is there any information that is missing from the screen?

• Do you have any other recommendations for how we could update the look or content of this screen?

Section 5. Inbox View

• What do you think is the purpose of the inbox? What do you expect to be able to do using this screen?

• Is the information presented in the inbox useful for you? What are the most useful things in the inbox? Why?

• Is there anything that is not helpful to you? Is there anything that is missing?

• Is the status column useful to you? What does “In Progress” mean to you? We currently have three status options (In Progress, Not Started, and Submitted) what other status designations would be helpful to you?

• Are the labels for each column descriptive? Are they initially presented in a useful order? If not, what order should they be in?

• At the top of the screen there are several options (Check for Software Updates, Add Location(s), Export to Spreadsheet, Import from Spreadsheet, Review All Forms, Submit Responses, and Exit). What do you think that each of these options will allow you to do? Are there any other buttons that should be at the top of the screen? Are there any that aren’t necessary?

• Below that list of buttons there is another list of buttons or tabs. These are called Welcome, Inbox, Form, Workbook, and Errors / Warnings. Did you have any problems using the tabs? What do you think about the titles we used for each tab? Do you have any recommendations for changing the titles? Do you have any other recommendations for how we could have had you navigate between each of the different screens in the software?

• Are there any features that have improved from the old version to the new version? Are there any features that you liked in the old version that aren’t in the new version?

• Did you manipulate any of the columns on this screen (moving, hiding, unhiding, or sorting)? Did you know that these functions were available? If not, how could we make it more obvious to you?

• Did you add any locations from the inbox tab? How did that process go for you? Do you remember having any issues with it? Do you have any recommendations for revising this functionality?

• Do you have any other comments or suggestions about the layout or content in this screen?

Section 6. Form View

• One of the changes to this screen is the panel above the question(s). This panel displays the mailing label address for the location that this form is associated with. Was this information helpful to you as you were completing the form? Was it initially obvious what this information was? Is there any other information that you would need to see about a location as you are completing a form that we should add to this part of the screen?

• Are there any features that have improved from the old version to the new version? Are there any features that you liked in the old version that aren’t in the new version?

• We made some modifications to the way that the errors and warnings are listed at the bottom of the screen. Did you use this part of the screen? Did you click on the Review Form button? Do you have any recommendations for how we could improve this part of the screen?

• Do you have any other comments or suggestions about the layout or content of this screen?

Section 7. Workbook View

• Did you use this view? If so, how often did you use it? Describe how you used it?

• If you didn’t use this view, why not? Did you try using it? What problems did you have with it?

• Did you use the All Locations option on the left side of the screen? If so, how did you use it? If not, why not? In your own words, describe what this option is giving you.

• What do the tabs that run along the bottom represent? Did you use them? Did you find them helpful?

• Did you notice that certain columns were repeated on each of the tabs at the bottom of the screen? Was this helpful to you at all?

• The cells under the first three columns are shaded. What does this shading mean to you?

• How did you move between the cells within this view (moving up and down, from side to side)? Is there any other way that you would prefer to navigate within this screen?

• Was it clear what each of the rows and columns represented?

• Are the labels for each column descriptive? Are they initially presented in a useful order?

• Did you manipulate any of the columns on this screen (moving, hiding, unhiding, or sorting)? Did you know that these functions were available? If not, how could we make it more obvious to you? Are there any other column options that we could have offered that would have been helpful to you?

• Did you toggle between this view and the form view at all? Was that useful to you?

• Did you ever cut and copy values in this view? Did you ever cut and copy a value to more than one cell? How did this function work for you? If you didn’t use it, would you have needed an option like this? If so, how could we have made that option clear to you?

• Did you notice the Sum information at the very bottom of the screen? Did you use this at all? Was it helpful? If you didn’t see it, did you know that you were able to highlight several fields and get a Sum? How could we have made this option more obvious to you?

• For items that required one of several options (1=Yes, 2=No), how did you input your information? Did you notice the drop down box in the cell? If so, was it helpful to you? Did you use it at all?

• Is there only one way to sort for all of the Workbook View or should you be able to sort All Locations and each of the other Form Types separately? Do you want to sort each tab of a Form Type separately?

• Do you have any other comments or suggestions about the layout or content of this screen?

Section 8. Errors / Warnings View

• This view was new, was this helpful to you at all? Did you use it? If so, how?

• After you saw an error/warning on this screen, what did you do? Where did you go (form view or workbook view)?

• Did you manipulate any of the columns on this screen (moving, hiding, unhiding, or sorting)? Did you know that these functions were available? If not, how could we make it more obvious to you? Are there any other column options that we could have offered that would have been helpful to you?

• Do you have any other comments or suggestions about the layout or content of this screen?

• In your own words, what is the difference between an error and a warning?

• Were you able to resolve all the errors associated with your data? Did you have any problems/issues resolving the errors? How did you feel about the number of errors in Surveyor?

• Approximately what percentage of our edits could we classify as errors without causing you problems? Are there any items that should only have warnings and not errors? Why?

Section 9. Exporting / Importing

• Did you use the exporting / importing option? If no, skip to Wrap Up section.

• Why did you choose to export and import your data?

• What was your overall impression of these features?

• What were some of the positive aspects of exporting/importing?

• What were some of the negative aspects of exporting/importing?

• How did you pull the data together before importing?

• How did you get the data from your records into the spreadsheet? Did you have the data in a spreadsheet already? If so, did you match the data from our spreadsheet to your spreadsheet? How did you do this? Did you cut and paste? Did you use an Excel function (such as V-lookup)? What fields were you matching? Were you using store number, CFN, any address fields, EIN, other?

• Did you have any problems using the tabs at the bottom of the spreadsheet? Was it helpful to have the items broken out into different tabs?

• We repeated the location address on each tab. Was this helpful to you?

• Did exporting/importing save you any time or resources?

• Did you use the instructions to help figure out the importing process? If so, were they helpful? Which instructions did you use?

• Did you notice extra information listed in the column headings? Was this information useful?

• Did you experience any problems when you were importing? Did you have any errors?

• Did you add any locations directly to the spreadsheet? Was this a problem at all during the import process?

• Do you have any recommendations for how we could improve the importing functionality?

Section 10– Wrap-Up

• Did you experience any bugs or errors when using the software? Can you describe what happened and what you did to resolve it?

• Did you experience any problems printing? How was this fixed?

• Did you need to call in for any support or help? If so, what did you call in about? What sort of questions/problems did you have?

• After initially downloading the software, did you ever go to retrieve any updated versions of the software? How did you know to do this? How did that process go?

• Do you have any final thoughts or comments about any of the material that we looked at today?

• If you could summarize the major changes that we should take back to our programmer, what would they be?

Thank you very much for your time and valuable comments!

6

Surveyor Debriefing Protocol – On-site Visits

June 20, 2008