Protocol

Respondent Debriefings – 2018 Census Test GQ eResponse Users

INTERVIEWER’S NAME: ______________________

DATE: ____ / ____ / 2018

START TIME: ______: ______ AM / PM

GQ RESPONSE STATUS: ____ Successful submission ____ Unsuccessful ____ Did not attempt

NUMBER OF GQ PARTICIPANTS: _____

1. Introduction

• Thank you for taking the time to meet today.

• I am [name] of the U.S. Census Bureau.

• (If observers are present/on phone) Joining us today [is/are] [observers’ name/s].

• [I’m/we’re] meeting with people like yourself who [used/attempted to use] our new eResponse system during the 2018 Group Quarters Enumeration.

• We want to learn about your experiences with using the system, what might have worked well and what may not have worked so well, so that we can look for ways to improve the system before the 2020 Census.

• Today I will ask you to tell me about your experiences with eResponse, and I will ask you questions about some specific features as we go along.

2. Interview consent

• Before we start, I would like you to read over the document in front of you.

• This document explains a little bit about this interview and provides information about your rights as a participant.

• Our session today is completely confidential, so all information you provide is protected and we won’t use your name in any report.

• Your participation in this study is completely voluntary. You may decline to answer any particular question.

• This form also asks for your permission to have this session audio recorded. That way, I can focus on what you’re saying rather than having to concentrate on taking notes or having to rely on my memory later.

• Please ask me any questions you may have about this document.

• Once you have finished reading the document, please sign it.

• (After Rs and interviewer sign consent form) I will now turn on the audio recorder.

3. Respondent background

First I’d like to learn a little about you and your organization.

• Can you briefly tell me what your organization does?

• What is your understanding of what the Census Bureau means by a “group quarters?”

• (If needed) Please describe the group quarters aspect of your organization.

For each GQ participant:

• What is your role in your organization?

• What are your primary responsibilities?

For all participants:

• Have you ever responded to any surveys on behalf of your organization?

• (If yes) What were they about?

• Who were they from?

• What was your experience like?

• Does your organization have any policies regarding surveys?

• What operating system do your computers use?

4. Response process

Please walk me through the steps you took from start to finish.

(Present screenshots if needed) I have screen shots from the system in case we need to refer to them.

(If multiple GQ participants) Please tell me what role everyone played in the process of responding to the Group Quarters Enumeration, if any.

• Are there any others who helped in preparing your response who are not present?

• How many others helped in some way?

• How did they help?

Did you have any issues with using the system?

• (If yes) Please tell me about that.

• How did you handle it?

• Were you able to resolve or work around the problem?

• Any other issues? (Repeat above probes as needed)

• Did you call anyone for help, either within your organization or at the Census Bureau?

  • (If yes) Whom did you call? How did they help?

Did you have any issues with using the system related to your organization’s IT security policies?

In your opinion, what worked well with the system, if anything?

• Anything else?

What would you change about the system, if anything?

• Anything else?

How many times did you access the system?

At any point, did you get logged out because of inactivity?

• In your opinion, is 15 minutes an appropriate amount of time?

Thank you for those comments. Now I’d like to go back to the beginning and ask you about some specific parts of the process that we may not have already discussed.

5. Log-in and account set-up

Let’s start with how you accessed the system.

• What browser did you use? (Internet Explorer, Firefox, Chrome, etc.)

• Did you have any problems getting to the website?

• How was your experience in logging into the web site?

What did you think of the log-in process?

• Did you have any problems entering the User ID?

• Did you have any problems entering the pin from the letter?

What did you think of being asked to create security questions?

• Did you have any issues with creating security questions?

• (If logged in >1 time) Did you have to answer the security questions any time you logged in later on? If so, did you have any issues with answering them?

Did you request a new pin at any point?

• (If yes) Did you have any issues with obtaining a new pin?

Thinking about the letter we sent, what is your impression of it?

• Do you recall any information from the letter?

• Did you find anything in the letter useful? Interesting?

• Based on the letter, did you have any expectations about the reporting process, such as how it works?

• Did you have any concerns?

• Is there any information you think should be in the letter?

• What did you do with the letter after you logged in the first time – keep it, discard it, etc.?

6. Main Menu

Now let’s talk about the next screen.

What did you think of this screen when you first saw it?

How did you use this screen?

Did you have any problem figuring out how to use this screen?

What did you think of the information presented on this screen?

There are two sections on this screen – Please tell me in your own words what they mean to you.

What do these terms mean to you?

First table:

• Facility

• Action

• Overall Status

• Point of Contact

• Instructions

Second table:

• Individual Group Quarters

• Census ID

• GQ Name

• Status

• Data Upload Error Count

In your opinion, is there a difference between “facility” and “individual group quarters”?

What does the icon below “Instructions” mean to you?

• Do you recall whether you clicked on that icon?

• (If yes) What happened when you clicked on it?

  • Did you look at that document?

  • (If yes) How did you use it?

    • In your experience with using the system, was this document useful or not useful?

    • What did you find useful about it, if anything?

    • What did you find NOT useful, if anything?

  • Did you save it?

  • Did you print it?

Do you recall whether you clicked on the “My Census IDs” button?

• (If yes) What did you think of this feature?

Do you recall seeing the links along the bottom of the screen (“Burden Statement,” etc.)?

• Did you click on any of those links?

• (If yes) What did you think of the pages that they opened?

  • Did you find them useful or not useful?

• (If no) What do you think you would find if you clicked on each one?

Do you have any suggestions for changing anything about this screen?

• The way it works?

• The way it looks?

• The information presented?

What did you do next?

7. Export Wizard

What did you do on this page?

What did you think of the information provided on this page?

• Was there any information you found useful?

• Was there any information you found NOT useful?

• Was there any information you found confusing?

Did you download the Sample Template, and if so, did you find it helpful?

Did you have any problems downloading (either of) the templates?

• (If yes) What problems did you have?

• How did you handle them?

Do you have any suggestions for changing anything about this screen?

• The way it works?

• The way it looks?

• The information presented?

What did you do next?

8. Spreadsheet template and data sources

Please tell me how you started using the spreadsheet template.

How did you figure out what information was being requested?

• Did you use the instructions in the spreadsheet?

• What is your opinion of the way instructions are presented in the spreadsheet?

• Is there anything about the spreadsheet that you would change?

How did you go about getting data into the spreadsheet template?

Did you have any problems using the spreadsheet template?

Please tell me about the process you went through to get the information that we requested.

• Did you have access to all the information that was requested?

  • (If no) What information did you have access to?

  • What information did you NOT have access to yourself?

• Did you have to access more than one system or other data source?

  • How many different data sources?

• What specific records did you access?

  • (If needed) What do you call them?

  • What data do they contain?

  • How are the records structured?

• Did you have to work with other people to get any information?

  • How were they involved?

  • Did the need to get help from others pose a challenge in any way?

• Did you have to get permission from someone in a position of authority to get access to any particular information or systems?

• Did you have to modify your source data in any way to get them into the format required by the spreadsheet template?

  • What did you have to do?

  • Did you have any problems with the desired data format in the template?

• Were you able to enter all the requested data into the template?

  • (If no) What data were you not able to enter?

The spreadsheet has two tabs. Can you tell me in your own words what each of those tabs is for?

• (If needed) In your opinion, what is the difference between them?

The Group Quarters Enumeration requires that every resident be listed with the Census ID for the group quarters in which they reside.

• In your opinion, was this requirement clear?

• How did you go about assigning Census ID’s to every resident?

• Did you have any issues with assigning Census ID’s to individual residents?

• Does the Census ID correspond to any information in your records with regard to where people reside?

What spreadsheet application did you use? Which version?

Do you use spreadsheets in your usual work?

• (If yes) What do you use them for?

• When it comes to using spreadsheets, would you say you have a basic, intermediate, or advanced skill level?

  • Why do you say that?

9. Import Wizard

Please tell me the steps you went through to upload the template.

Did you have any problems with the upload process?

Did you receive any feedback from the system after the upload?

• (If yes) Do you recall what messages you received?

• Did you receive any feedback indicating there were issues with the file you uploaded?

• In your opinion, was the feedback clear or not clear as far as describing the problems and what was needed to fix them?

• Did you try to fix anything and upload again?

  • (If yes) What did you do?

  • Did you receive any other error messages?

What did you do after you uploaded the file?

• Do you recall seeing the “Submit” button?

• In your opinion, was it clear that you had to click “Submit”?

Did you receive any feedback that the submission was successful?

• Did you print or save the confirmation message?

10. Wrap-up

I just have a couple of final questions to wrap up.

Overall, how satisfied are you with your experience using eResponse?

• Why do you say that?

In a perfect world, what kind of system for collection information about your residents would work best for you?

• How would it work?

• What kind of information would you want to have before you started using the system?

Is there anything else we haven’t discussed that you would like me to know about?

Those are all the questions I have. Thanks very much for your time today.