Various Economic Area Pretesting Activities

Draft 5/1/2011

BE-577 Respondent Debriefing Protocol

Dave Tuttle and Kristin Stettler, RIRS

Research Questions to Address:

• Do the changes in layout adversely affect the response process, and the responses provided?

• What is the effect of placing instructions on pages adjacent to the questions they refer to?

• Do respondents have issues with comprehending the terminology used on the questionnaire?

• Is the requested information available in respondent’s records?

Expected Length of Interview: 90 minutes

Materials Needed:

• Copies of respondent’s BE-577 report

• Consent form

• Digital recorder

Introduction:

• People around the table: how long at company, position/title and responsibilities

• Explain purpose of meeting: to understand the process used to answer questions and their reaction to the new form design

• Explain that we are not testing the respondent – we only want information

• Structure of meeting: general and specific questions about the form itself, as well as the organization

• Permission to record discussion? Have R sign consent form.

Before the Questionnaire:

• What types of products or services does this company provide to its customers or clients?

• What do you usually do when you receive this questionnaire in the mail? (Sit on it, complete immediately, etc.) What do you do with it after you’ve finished? (Double-check numbers, reconcile with previous quarter, and / or look at balance sheet?)

• (For new Rs: ) How did you end up with the form? What were your impressions of it? How did you learn how to file the form?

Now let’s look at what you returned to the Department of Commerce/BEA recently. We’re only interested in some of the questions, not all of them.

Questions for each item of interest:

• What is this item asking for, in your opinion?

• How did you go about completing this item?

• Did you have access to all the information you needed, or did you have to get help from others?

• (If needed help) Whom did you contact? Where is s/he located in the company? What is her/his role?

• How did you communicate the request? How did s/he return the data to you?

• Did you attempt to validate the response they gave you in any way, for example by comparing it to other data you had?

• What records did you use to answer this question?

• Do you recall seeing the instructions associated with this item? Did you read them?

• (If read instructions) Were you looking for anything specific? Did you find what you were looking for? Were the instructions helpful or not helpful?

• Were there any terms that seemed strange or confusing to you?

• (If yes) Which ones? How can we clarify that?

Specific Probes for Items of Interest:

Overall impression of the redesigned form

• (If R completed older form in past) What did you think about the new look of the form? What are the changes that you noticed?

• In your opinion, what are the advantages of the new form, if any? What are the disadvantages?

• Do you prefer the new format, with more space and spread out over more pages, or do you prefer the older version which took up fewer pages?

• Do you prefer the new letter-size pages or the previous legal size pages?

• Do you prefer instructions in the back of the form, as with the older version, or do you prefer to have the instructions with the questions?

• (If R did not complete previous form) What is your overall impression of the form? What did you like about it, if anything? What did you not like?

Item 9

• What does “reporting status” mean to you?

Item 10

• What does “fully consolidated” mean to you?

• Do your recall whether you looked at the rules for consolidation on page 10 of the form?

• (if yes) How did those instructions affect how you reported in item 10, if at all?

Item 12

• What does “the foreign affiliate in this affiliate’s ownership chain that is directly owned by the US reporter” mean to you?

• What does [first/second] diagram mean to you? Did you find these diagrams helpful or not helpful in understanding the question, and why?

Item 13 – Instructions adjacent to question

• What do you think of the layout of this page?

• In your opinion, what are the advantages of this layout, if any? What are the disadvantages?

• Do you recall seeing instructions for which books to use for certain questions? What do these instructions mean to you?

Item 14-16 – TBD

Items 19-23 – TBD

Items 22-26 – Selected Annual Data

• What does this section mean to you?

• How often would you complete this section? When would you complete it?