Various Demographic Pretesting Activities

Cognitive Testing of ACS Group Quarters Supporting Materials

Version 1A – Director of Student Housing

November 15, 2010

Participant ID # __ __ __ __ __ __ __ (Interviewer’s initials / 3-digit sequential # / Location)

Date: ___ /____/2011

Interviewer: _______________________________________

Location (circle one): DC IL NC

Tested Forms: Group Quarters Contact Letter – Director of Student Housing (Version: 1-2011)

Frequently Asked Questions Brochure (Version: 11-9-2010)

START TIME: ____________ AM / PM

SECTION I: INTRODUCTION

PLACE THE CONSENT FORM IN FRONT OF PARTICIPANT

1. INFORMED CONSENT: Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant. In addition, the back page asks for your permission to have this session audio recorded. Please ask me any questions you have about this document. Once you have finished reading the document, please sign it.

WHILE RESPONDENT IS READING  OBSERVE AND DOCUMENT BELOW: TO WHAT EXTENT DOES THE RESPONDENT READ THE CONSENT FORM? PLEASE SELECT ONE A) GLANCE FRONT PAGE ONLY (quick look, does not read) B) GLANCE FRONT AND BACK (quick look, does not read) C) SCAN (quick read, does not read every sentence) D) READ E) READ ALOUD F) STUDY DOES THE RESPONDENT SHOW ANY HESITATION OR CONFUSION? YES -- WHAT? NO DOES THE RESPODENT SAY ANYTHING? YES -- WHAT? B) NO

PARTICIPANT READS AND SIGNS FORM. PROVIDE A COPY OF THE FORM TO THE PARTICIPANT

Here is a copy of this form for you to keep.

B. IF PARTICIPANT PROVIDES CONSENT TO HAVE THE SESSION AUDIO-TAPED:

I will now turn on the tape recorder.

TEST TAPE RECORDER AND TURN IT ON AGAIN AFTER TEST

Now, I would like to confirm on tape that it is fine with you to record this interview.

C. BRIEFLY EXPLAIN THE SURVEY AND PROCEDURES USING THE FOLLOWING SCRIPT:

• The Census Bureau counts the population in the U.S for the decennial census every 10 years, but the Census Bureau also conducts various other surveys such as the American Community Survey.

• They have hired us to conduct testing with administrators in colleges/universities, and health care facilities to see how well a new targeted introductory letter and the Frequently Asked Questions for Facility Administrators brochure work in gaining cooperation when contacted by the American Community Survey. We want to hear your thoughts and opinions on these materials. For example, we might ask you if you found something positive or negative about a document or we might ask if there were any words that seemed unclear or confusing.

• There are no right or wrong answers. We are simply trying to make sure the content of these materials is clear and easy for most people to understand.

• In a few minutes, we will ask you to read the materials as if you were selected to participate in the American Community Survey.

• Your participation in this review of the materials is very important because it will help the Census Bureau in revising these materials.

• Any questions before we begin?

D. NEXT HAND THE PACKAGE OF CONTACT MATERIAL TO THE PARTICIPANT AND ASK THEM TO EXAMINE THE CONTENTS.

• THE PACKAGE WILL CONTAIN A COPY OF THE APPROPRIATE INTRODUCTORY LETTER AND THE FREQUENTLY ASKED QUESTIONS (FAQ) FOR FACILITY ADMINISTRATORS BROCHURE.

• PAY ATTENTION TO WHICH OF THE TWO DOCUMENTS THE RESPONDENT SEEMS TO BE PAYING THE MOST ATTENTION AND INDICATE IT BELOW. THEN FOLLOW THE INSTRUCTIONS IN THE QUESTION ABOUT WHERE TO START THE INTERVIEW.  ALSO, NOTE ANY RESPONDENT BEHAVIOR THAT MIGHT HELP UNDERSTAND WHY THEY SELECT EITHER THE LETTER OR BROCHURE.

WHICH DOCUMENT DOES THE RESPONDENT FOCUS ON THE MOST?

• INTRODUCTORY LETTER (GO TO SECTION II-A)

• FAQ FOR FACILITY ADMINISTRATORS BROCHURE (GO TO SECTION II-B)

BEHAVIORAL OBSERVATIONS:

SECTION II – A: “DIRECTORS OF STUDENT HOUSING” LETTER (Version: 01-2011)

A. FIRST READING: SILENT READING BY PARTICIPANT

HAND THE LETTER TO THE PARTICIPANT

When an educational institution is selected to be in the survey, a letter like this one is mailed to that address in order to provide an explanation of the American Community Survey and what will be asked of you as the contact person during the visit to your facility.

Please read this silently to yourself and let me know when you are finished. I will then ask you some questions about the letter.

B. DEBRIEFING:

• What did you think this letter was about? Can you tell me more about the letter?

• After reading this letter, do you think you can explain to someone else what the American Community Survey is about?

• Please tell me what you would say in your own words.

• Does this letter ask you to do anything? What is that?

• Was there anything you found particularly good or positive about the letter?

• Was there anything about the letter that might encourage you to participate if your student housing unit were selected for this survey? What was that?

• Was there anything you found particularly bad or negative in the letter?

• Was there any part of the letter that caused you to become concerned? What was that/Why? IF YES: Do you remember where that was in the letter?

• What would you think if you received this letter in the mail?

• Do you know who is conducting the survey?

• Do you know how the American Community Survey helps this community?

• From this letter, do you think this facility has to participate?

• Was there anything in this letter you found confusing or difficult to understand?

• Is there anything in the letter that helped you understand why students selected for the survey should participate and for those students who do not complete the survey, why the survey information is still required?

C. SECOND READING/DEBRIEFING

IN THIS SECTION: THE PARTICIPANT READS ALOUD ONE PARAGRAPH AT A TIME INTERVIEWER OBSERVES PARTICIPANT INTERVIEWER DOCUMENTS OBSERVATIONS INTERVIEWER PROBES INTERVIEWER ASKS SCRIPTED QUESTIONS ALL STEPS ARE REPEATED FOR EACH PARAGRAPH AFTER ALL PARAGRAPHS, ADMINISTER “GENERAL PROBES”

• 1^st Paragraph: Please read the first paragraph aloud to me and then I will ask you some questions.

Your student housing facility has been randomly selected to participate in a very important national survey conducted by the U.S. Census Bureau called the American Community Survey (ACS). The ACS is an on-going monthly survey that tells us what the population looks like and how it lives. A sample of students who live or stay in student housing is included in the ACS each month. Data are used by federal, state, local, and tribal governments, as well as by businesses and private non-profit organizations. Participating in the ACS is important and also required by law (Title 13, United States Code), Sections 141, 193, 221, and 223).

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”).

INTERVIEWER PROBING

PROBE: Tell me, in your own words, about the American Community Survey?

PROBE: What were your initial thoughts when you read this paragraph?

PROBE: What did you think about when you read that participation is required by law?

PROBE: Had you heard of Title 13? Did you find its mention useful?

PROBE: Do you have any other comments regarding the first paragraph?

• 2^nd Paragraph: Please read the second paragraph aloud to me and then I will ask you s

  Each month, representatives for the ACS contact facilities, such as college residence halls, dormitories, and fraternity/sorority housing recognized by the college or university. In the next few weeks, one of our ACS field representatives (FRs) will contact you or someone in your office to discuss the survey and to schedule an appointment to visit your facility. They will:

  • answer any questions you might have about the ACS.

  • collect and verify basic information about your student housing facility such as the facility name and address.

  • ask for a list of students who are currently living or staying at the sampled student housing facility to establish a basis for their sample. According to the Family Education Rights and Privacy Act (FERPA) the Census Bureau is permitted to collect this information.

  • select a random sample of students to take part in the ACS.

  • interview the selected students.

  ome questions.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: What did you think about the description of how and when you would be contacted by an ACS Field Representative?

PROBE: When you read that the ACS Field Representative would collect basic information about your student housing facility what kinds of information did you think they would collect?

PROBE: When you read the letter, did you have any questions about the justification to collect this information from the Family Education Rights and Privacy Act (FERPA)?

PROBE: Did you have any concerns about providing a list of students currently living or staying in this facility?

PROBE: What do you think is meant by, “Select a random sample of students to take part in the ACS?”

• 3^rd Paragraph: Please read the third paragraph aloud to me and then I will ask you some questions.

Selected students will be asked to respond to the ACS and provide information about various topics, such as education and employment. All the information the U.S. Census Bureau collects is confidential by law.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”).

PROBE: What do you think is meant by “selected students?”

PROBE: What will selected students be asked to do?

INTERVIEWER PROBING [POINT TO STATEMENT]:

• Confidentiality: All the information the U.S. Census Bureau collects is confidential by law.

PROBE: What does this statement mean to you?

PROBE: What does the term “confidential” mean to you in this sentence?

• 4^th Paragraph: Please read the fourth paragraph aloud to me and then I will ask you some questions.

Your cooperation is essential to the success of the ACS, and the FR may need your assistance in collecting survey data from the sample of students. The FR will need access to the students chosen. For those students who do not complete the survey, colleges and universities can lawfully disclose directory information from student records to the Census Bureau without prior consent of the student, parents, or guardians as stated in the FERPA. Directory information includes a student’s name, date of birth, school address, and dates of attendance.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

INTERVIEWER PROBING [POINT TO STATEMENT]:

• Your cooperation is essential to the success of the ACS, and the FR may need your assistance in collecting survey data from the sample of students.

PROBE: Who do you think ‘FR’ refers to?

PROBE: When you read this question, what was your reaction?

PROBE: Had the letter provided enough information to this point to gain your cooperation? IF NOT: Tell me what was missing?

PROBE: What was your reaction to the sentences explaining that colleges and universities can lawfully disclose directory information from student records without consent of the student, parent, or guardian?

PROBE: Did these sentences make you feel uncomfortable? IF YES: Tell me why?

• 5^th Paragraph: Please read the fifth paragraph aloud to me and then I will ask you some questions.

The ACS data are used by federal agencies to determine the distribution of Title I funding and college tuition grant and loan programs. The information may also be used by local governments for budgeting, evaluating programs, and planning for community development projects, such as those for libraries and new schools. Other organizations use this information to provide services to the community and to plan or establish new facilities, buildings, and programs. Researchers and planners at your own institution almost certainly use ACS data themselves.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RS TO READ AND/OR

COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: What are your thoughts about how the information can be used?

PROBE: Do you feel this is the most appropriate location in the letter for this data use information or would you place it somewhere else? Why?

• 6^th and 7^th Paragraphs: Please read the remaining two paragraphs aloud to me and then I will ask you some questions.

The enclosed brochure provides answers to frequently asked questions about the ACS. If you want to learn more about the ACS, please visit the Census Bureau’s Web site at <http://www.census.gov/acs>.

If you have further questions, please call the Census Bureau’s Atlanta Regional Office toll-free number at

1-800-424-6974. Thank you in advance for your cooperation, and we look forward to working with you.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: Tell me your thoughts about how you can get more information?

PROBE: When you saw that this is signed by the U.S. Census Bureau Director, what was your reaction?

GENERAL PROBES ABOUT THE “DIRECTORS OF STUDENT HOUSING” LETTER (Version: 01-2011):

Now I have some general questions about this letter.

• Do the words, terms, and ideas used in this letter sound right or appropriate? Are there any that do not seem right or appropriate? Are there any terms that have a negative connotation in your language?

• Are there any words or ideas here that you found hard to understand? How would you change them?

• How about other Directors of Student Housing—do you think there are any words or ideas here that they might find hard to understand?

• Are there any parts of the letter that people like you may have difficulty with?

SECTION II – B: FREQUENTLY ASKED QUESTIONS BROCHURE (Version: 11-09-2010)

A. FIRST READING: SILENT READING BY PARTICIPANT

HAND THE BROCHURE TO THE PARTICIPANT

Now imagine you just received this brochure along with the letter.

This brochure is mailed to respondents, along with the letter you just read.

Please read this silently to yourself. However, I would like to know your reactions as you read the brochure. I would like for you to “think-aloud.” By this, I mean, tell me your thoughts and reactions as you read over the brochure.

When you are finished, I will then ask you some questions about the brochure. Please pretend you did not see the previous letter.

B. DEBRIEFING:

When you answer my questions, please think only about this brochure and try to ignore the letter:

• What did you think this about how the information was presented in this brochure?

• Can you tell me any sections or topics that you found helpful?

• Can you tell me any sections or topics that you did not find helpful?

• Do you still have questions that were not covered or described in enough detail in this brochure?

• Was there anything you found particularly bad/negative in the brochure?

POINT TO PARAGRAPH IN BROCHURE

P

How are residents at my facility selected to be in the sample?

After you provide a list of all residents currently staying at your facility, we randomly select residents to take part in the American Community Survey. One of the advantages of a random sample is that we can use it to measure the whole population without having to actually interview every resident at every facility. But in order for it to work properly, we cannot substitute sample residents–the sample has to be truly random.

lease look at this paragraph and read it aloud to me and then I will ask you some questions.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: What were your thoughts about the description of random sample? IF STILL NOT EVIDENT, ASK: In your own words, what do you think they are saying in the last sentence?

PROBE: What questions do you still have regarding the selection process after reading this paragraph?

POINT TO PARAGRAPH IN BROCHURE

P

Do the sampled group quarters and sampled residents have to answer the questions on the American Community Survey?

Yes. You are required by law to answer the American Community Survey. Two provisions of Title 13, United States Code, require your participation. Section 221 of Title 13 makes it mandatory for individuals to participate in the American Community Survey, and Section 223 of Title 13 makes it mandatory for building owners and others to provide access to individuals in order to conduct the American Community Survey group quarter interviews. The same law protects the confidentiality of the information you provide.

lease look at this paragraph and read it aloud to me and then I will ask you some questions.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: What did you think about this description of Title 13 and the requirement to participate in the American Community Survey?

PROBE: Who is required to participate? IF STILL UNANSWERED: Was it clear that individual participants as well as directors were required to participate? IF NO: Please explain.

PROBE: Do you have any concerns after reading this paragraph?

POINT TO PARAGRAPH IN BROCHURE

Please look at this paragraph and read it aloud to me and then I will ask you some questions.

Will the Census Bureau keep the group quarter facility and resident information confidential?

Yes. Your answers are confidential by law under Title 13, United States Code, Section 9. This law specifies that the Census Bureau can use the information provided by the facility and residents for statistical purposes only and cannot publish or release information that would identify any group quarters or individual.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: Can you tell me what this means in your own words? IF NOT ANSWERED: What do you think they mean by “statistical purposes?”

PROBE: Would this satisfy any concerns you might have about revealing resident information to the Census Bureau? IF NO: Please explain.

POINT TO PARAGRAPH IN BROCHURE

P

Who can see the answers this group quarters or its residents provide to the survey?

Your facility and resident responses will only be seen by Census Bureau personnel with a work-related need to know. The Census Bureau protects the confidentiality of the information. The police cannot see it and no court of law can see it. No one can see or use your specific answers to enforce any type of law or amend any kind of benefit.

If any Census Bureau employee were to share information about your facility or its residents, he or she would be subject to severe criminal sanctions imposed by Congress–up to 5 years imprisonment and/or up to a $250,000 fine (Title 13, United States Code, Section 214, as amended by Title 18, United States Code, Sections 3559 and 3571).

lease look at this paragraph and read it aloud to me and then I will ask you some questions.

WHILE RESPONDENT IS READING - OBSERVE AND DOCUMENT BELOW:

• PLACES WHERE TEXT/SYNTAX SEEMS DIFFICULT FOR RESPONDENT TO READ AND/OR COMPREHEND.

• ANY TERMS/CONCEPTS MENTIONED BY RESPONDENT AND THE ISSUES THEY HAVE WITH THOSE TERMS/CONCEPTS.

• ANY MISPRONUNCIATIONS. (THIS WILL SERVE AS A CLUE FOR DIFFICULT RESPONDENT UNKNOWN TERMS/CONCEPTS.)

PROBE IF APPROPRIATE: “I noticed when you read the paragraph …, you were laughing/ hesitating/pausing, what were you thinking at that point?”

PROBE: Can you tell me what this means in your own words?

PROBE: Would this satisfy any concerns you have about the use and sharing of information? Please explain.

SECTION III: CONCLUSION

1. FINAL COMMENTS: I just have a few final questions.

After reading the letter and the brochure, would you be likely to participate in the survey? Why/Why not?

Do you have any final comments or any questions?

B. CLOSING: I want to thank you very much for participating today. I will now give you $40 as a token of our appreciation. Please sign this receipt form verifying that you received the money.

TURN OFF THE TAPE RECORDER. HAND THE CASH INCENTIVE TO THE PARTICIPANT AND TAKE THE SIGNED INCENTIVE RECEIPT FORM.

TURN OFF THE TAPE RECORDER IF YOU HAVE A MICROPHONE TURN IT OFF TOO HAND PARTICIPANT THE CASH INCENTIVE HAVE PARTICIPANT SIGN RECEIPT FORM COLLECT CONSENT FORM WITH 2 SIGNATURES THANK PARTICIPANT – YOU ARE DONE!!

END TIME: ____________ AM / PM

20