Various Demographic Pretesting Activities

Protocol in Spanish

Date_______________________; Participant #________; Experimenter:_______________

General Introduction: American Community Survey Internet Instrument

Buenos días/Buenas tardes/Buenas noches. Gracias por su tiempo hoy. Me llamo XX y trabajo aquí para el grupo de factores humanos y usabilidad (Give an explanation if necessary). Yo voy a trabajar con usted hoy. Si usted tiene un teléfono celular, apáguelo o ajústelo a vibrar (o silencio). [Hoy] vamos a evaluar el diseño de la encuesta de la comunidad estadounidense al tener que llenarla. Su experiencia con la encuesta es una parte indispensable de nuestro trabajo. No escribí la encuesta, así que por favor comparta conmigo sus reacciones y sentimientos positivos y negativos. Esta sesión debe durar una hora más o menos. Los diseñadores de esta encuesta van a recibir sus comentarios para que se pueda mejorarla.

Thank you for your time today. My name is XX and I work here with the Human Factors and Usability group. I will be working with you today. If you have a cell phone, please turn it off or put it in vibrate. We will be evaluating the design of the online American Community Survey by having you complete it. Your experience with the survey is an essential part of our work. I did not create the survey, so please share both your positive and negative reactions to it. We are not evaluating you or your skills, but rather you are helping us see how well the survey works. The entire session should last about an hour. Your comments and feedback will be given to the developers of the survey and may be used to improve it.

Primero, me gustaría pedirle que lea y firme el formulario de consentimiento. Este documento le explica el propósito de esta sesión de investigación académica y sus derechos como participante. También le informe que nos gustaría hacer una grabación de video de esta sesión con su permiso. Solamente nosotros que estamos afiliados con este proyecto van a mirar el video o cualquier otro dato recopilado y la vamos a utilizar solamente para fines de investigaciones académicas. También podemos utilizar partes del video para ilustrarle puntos claves sobre la encuesta al equipo de diseño de la Red. Adicionalmente, pueden estar mirando esta sesión los observadores del equipo de investigación de este proyecto en otra sala.

First, I would like to ask you to read and sign this consent form. It explains the purpose of today’s session and informs you of your rights as a participant. It also tells you that we would like to videotape the session, with your permission. Only those of us connected with the project will review the tape and any other data collected during the session, and it will be used solely for research purposes. We may also use clips from the tape to illustrate key points about the survey to the Web design team. In addition, there may also be observers from the project team observing this session in another room.

Hand the participant the consent form; give time to read and sign; sign own name and date if you have not already done so.

Start the tape.

Me gustaría que usted me diga acerca de sus pensamientos sobre las pantallas como usted las ve. En otras palabras, me gustaría que usted pensara en voz alta mientras usted navega las páginas de web. Si usted espera ver una pieza de información y no se encuentra, avíseme acerca de eso. Si usted espera que algo acontece y no acontece, avíseme acerca de eso también. Finalmente, si algo le parece confuso en la pantalla, avíseme acera de eso de nuevo.

I would like you to tell me your thoughts about the screens as you look at them. In other words, I would like you to ``think aloud'' as you go through the Web pages. If you expect to see a piece of information and it isn’t there, tell me. If you expect something to happen and it doesn’t happen, tell me. Let me know if you don’t know what to do next. Finally, if anything is confusing on the screen, let me know that too.

Pull up http://www.elnuevodia.com in Firefox.

Antes de empezar, vamos a practicar el acto de pensar en voz alta ya que no es algo que se hace normalmente mientras se trabaja en línea. Finja que usted tiene algunos minutos libres en su escritorio mientras está en su trabajo o en casa y describa su proceso de pensamiento mientras usted trata de encontrar algo interesante para leer.

Before we get started, let's practice thinking aloud, since it's not something that you would normally do while working online. Pretend that you have a minute or two to kill at your desk at work or at home and talk me through your thought process as you try to find something interesting to read.

Ok, eso es exactamente lo que me gustaría que hiciera durante esta sesión. Si usted se pone silente durante la sesión, yo podría recordarle a hablar conmigo. No es para romper su proceso de pensamiento sino para recordarle a seguir hablando conmigo. Por favor enfóquese en verbalizar lo que está pensando mientras completa la encuesta.

Ok, that’s exactly what I would like for you to do throughout the session. If at any time during the session you get quiet, I may remind you to talk to me. This is not to interrupt your thought process, but simply to remind you to keep talking to me. Please focus on verbalizing what you are thinking as you complete the survey.

Tiene usted algunas preguntas sobre la técnica de pensar en voz alta que acabamos de practicar?

Do you have any questions about the think aloud technique that we just practiced?

Mientras complete la encuesta, nosotros vamos a grabar los movimientos de sus ojos con nuestro monitor de rastrear ojos para tener un registro de que parte de la pantalla mira usted y vamos a grabar los movimientos del ratón para ver que tan bien interactúa usted con la encuesta.

While you are completing the survey, we will record the movements of your eyes with our eye-tracking monitor to get a record of where you are looking on the screen and we will record your mouse movements to see how you are interacting with the survey.

Ahora le voy a calibrar los ojos para el proceso de rastrear ojos.

Now I am going to calibrate your eyes for the eye-tracking.

Do Calibration

Ahora que tenemos sus ojos calibrados, estamos listos para empezar. Por favor responde a la encuesta en línea como si estuviera en casa. Usted podría contestar las preguntas de la encuesta como le aplican a usted en su vida verdadera. Aunque las materiales le dan una dirección del Internet o URL, para tener acceso a la encuesta, usted no va a necesitar entrarla porque nuestro programa de prueba le va abrir la encuesta.

Now that we have your eyes calibrated, we are ready to begin. Please respond to the survey online as you would at home. You may answer the survey questions as they apply to you in your real life. Although the materials will give you an internet address, or URL, to enter to access the survey, you will not need to enter that because our testing software will open the survey for you.

Por favor, imagine que recibió este paquete por correo postal en su casa. Éstos son las materiales de correo que va a necesitar para empezar la encuesta. Por favor, imagine que recibió este paquete por correo postal en su casa. Si usted recibiría la encuesta en su casa, las materiales de correo tendrían su dirección verdadera. Ya que no podemos replicar eso en este ambiente de laboratorio, todos los participantes van a utilizar la misma dirección. A los efectos de este estudio, vamos a simular que su dirección es (A typical address in Puerto Rico).

Please imagine that you received this packet in the mail at your home. These are the mailing materials you will need to start the survey. If you were to receive the survey at your home, the mailing materials would have your real address. Since we cannot replicate that for the lab setting, all participants will use the same address. For the purposes of this study, please pretend that your address is 198 Young Rd in Anytown MD.

Voy al otro cuarto para llevar a cabo una prueba de sonido. Mientras hago eso, por favor tome un momento para completar este cuestionario corto. [Hand Participant questionnaire on Computer experience and demographics]

I am going to go around to the other room to do a sound check. While I am doing that, please take a moment to complete this questionnaire. [Hand Participant questionnaire on Computer experience and demographics]

Voy a salir de este cuarto para el otro pero todavía podríamos ser capaces de comunicarnos por micrófonos y altavoces. Tiene usted alguna pregunta antes de empezar.

I’m going to leave but we will still be able to communicate through a series of microphones and speakers. Do you have any questions before we begin?

Leave room. Once in control room do a sound check and Start the eye-tracking software: Tobii Studio. The mouse tracing software will start when Studio opens Internet Explorer.

Encourage R to think aloud while completing the survey. Ask probe questions about what they are thinking if they are having trouble with any part of the survey.

Overall Probe: Make a note if a person left a page with a blank answer. If it had a soft edit, ask them whether the edit pursueded them to answer it.

PIN Screen: Seemed to read it [ ] Wrote PIN down [ ]

Write down the PIN in case the Participant does not.

STOP AFTER ROSTER CHECK (Final List of eligible people)!!!!!!!!!!

Check Awareness or use of progress indicator

I’m going to pause you for a second:

About how much longer do you think the survey will take? How did you come up with that answer?

Notes on other screens:

STOP PARTICIPANT AFTER MORTGAGE QUESTIONS

Second Check of awareness or use of progress indicator

I’m going to pause you for a second:

About how much longer do you think the survey will take? How did you come up with your

answer?

Let’s say you have to leave for an appointment, what would you do?

Observe: did they find “save & logout” button?

Observe: Use “save & logout” [ ] Close browser[ ] Other (explain)[ ]

Let’s select “save & logout” right now. Pretend you have left for your appointment and have returned. You want to reenter the survey. What would you do?

Observe: if they mention PIN, and if they remember it.

Thanks for that information, okay you can continue.

STOP AT PICK NEXT PERSON

The PICK NEXT PERSON comes before Person Questions;

PROBE: What do you think this question is asking you to do?

PRESUMMARY SCREEN (list of Names with Review Answers Links):

I’m going to pause you again:

Take a look at this screen, before making a selection on the screen, please tell me what you would do if you were taking this at home and why?

(if they chose submit) Why wouldn’t you select review first?

Vignette

Now let’s go to review (anyway) and see how that screen works for you.

Suppose you wanted to review the answers you gave for yourself, what would you do? Note what they say/do. If they don’t click the “review answers” link, have them do that.

Suppose you wanted to change the answer for place of birth from the United States to England, how would you do that? After they made the change, did they click return, or did you have to tell them?

When they got back to review and edit screen, did they notice the new line on the table for Citizenship with a [Blank] for the response? What did they do? IF they didn't notice it, point it out and ask them what they would do now.

This one only works if the person is born in the U.S. If they aren't born in U.S. have them edit housing question 4 (how many acres?) to more than 1 acre so the agricultural sales question is added to the review and edit screen.

Debriefing Questions

Show each relevant screen and allow the participant to look at each screen as you ask questions about it.

Debriefing:

What was your overall impression of the survey?

PROBE ABOUT ANY UNUSUAL RESPONSES.

If they mentioned that someone in their household uses more than one mode of transportation to get to work (such as bus and subway) and they chose one, ask why they chose that one.

MOVE ON TO THE POWERPOINT SLIDES

Slide 1:

¿Vio usted el indicador de progreso en el lado derecho de la pantalla?

Did you notice the progress indicator on the right side of the screen?

¿Qué piensa que significa el texto que está cubierto y escrito en gris?

What do you think the grayed out text indicates?

¿Qué piensa usted que le pregunta la pregunta abajo?

What do you think the question below is asking you?

Slide 2:

¿Por qué piensa usted que la pregunta en la parte superior de la página está escrita en gris?

Why do you think the question at the top of the page is grayed out?

¿Piensa usted que podría cambiar su respuesta para la pregunta que esta escrita en gris?

Did you think you could change the answer to the grayed out question?

Slide 3:

Suponga que usted tuviera un niño o una niña adolescente que ahora vive en un internado. ¿Cómo contestaría esta pregunta?

Suppose you had a high-school age student living away at boarding school right now. How would you answer this question?

Slide 4: Suponga que usted tuviera un niño o una niña custodia compartida quien normalmente vive con el/la otro/a padre pero está con usted en el día en que usted complete esta encuesta. ¿Incluiría este/a niño/a mientras completa esta encuesta?

Slide 4: Suppose you had a child in shared custody that usually lives with you but is with the other parent on the day you complete this survey. Would you include that child when you complete this survey?

Suponga que usted tuviera un niño o una niña custodia compartida quien normalmente vive con el/la otro/a padre pero está con usted en el día en que usted complete esta encuesta. ¿Incluiría este/a niño/a mientras completa esta encuesta?

Suppose you had a child in shared custody that usually lives with the other parent but is with you on the day you complete this survey. Would you include that child when you complete the survey?

Participant#:

RACE: American Indian or Alaska Native Asian Native Hawaiian or Other Pacific Islander White Asian Other

AGE RANGE: < 30 31-45 46-60 61+

GENDER: M F

EDUCATION: HS/GED Some Coll/AA Bachelor’s Some grad

DATE OF INTERVIEW:

INTERVIEWER:

10 | Page