QSS- Attachemt 6: Cognitive Testing Finance and Accomodation

2012 Company Visit

Services Indicator Branch

Findings and Recommendations

Visits done by:

Maria Mate

Rebecca Hutchinson

Brendan Brankin

Author: Maria Mate

Last updated: May 18, 2012

Background

In February 2012, we conducted 11 total interviews to evaluate the proposed questions on our new accommodations and finance forms. Nine were conducted in New York and one in Virginia. We also conducted one phone interview with the subject also in New York. Interviews were conducted with respondents whose roles within the companies included chief executive officers, chief financial officers, accountants, tax managers, and vice presidents. The interviews were 30-45 minutes in length.

These interviews consisted of gaining perspective on respondent’s interpretation of form questions, as well as their ability to report. The primary issues addressed during the course of the interview were item definitions, the burden it imposed on companies, and the ability to report on a quarterly basis. A copy of the protocol used in these interviews is attached in this document.

Findings and Recommendations

Cover Page and General Instructions

Respondent noted that the same information from the old form is also available in the new form. They also noticed the general instructions but most did not look at the contents. Mostly it appeared that the instructions were straightforward. A number of respondents complained about the due date and their constant need for filing extensions.

Recommendation: No recommendation.

Survey Coverage

For the most part, respondents did not have a problem understanding the information provided. One respondent asked for the meaning of the word “provide.”

Recommendation: No recommendation.

Sales, Receipts or Revenue

Finance

The following are the comments received from the respondents:

• Splitting realized gains from unrealized gains is difficult on a quarterly basis.

• Wondered if interest on inventory positions and holds on securities should be included.

• Company would exclude certain types of interest that were not explicitly listed as examples in the include section.

• Company noted that definitions are kind of light in this section.

• There was some confusion on interest and the examples of kinds of interest currently provided, if we want all interest we should just ask for “interest .“

• Wanted clarification on whether intracompany transactions were to be included.

• Would like to add management fees and performance fees to the Include section

Recommendation: Consider removing examples of interested earned from the include list.

Action: Removed earned interest examples from the include list. Bullet will simply say, “Earned interest.”

Accommodations

Most of the companies visited do not manage other hotels. One said that they do not have a problem reporting for managed revenue. Another company owned properties that were managed by another company and they were not concerned if that company was asked to report revenue from those locations. However, they would include revenue from those locations in their report as well. One company compared QSS and ARTS form and noticed QSS form did not have a ‘Deduct’ section.

Recommendation: Consider having the same instructions as ARTS.

Action: Deduct section will be added in the include and exclude list to match ARTS.

Source of Revenue

QSS-6A and QSS-6E

• Where would wholesale banking and participation loan fees go?

• With international banking, remittances are a big portion of business. Should income generated on those be included anywhere?

• Unsure on distinction between ‘individual’ and ‘business and government’. Company purchases individuals’ mortgages from other banks. Individuals pay fees to bank but a portion goes to the company as well.

• Initial setup to break out data will take much longer (hours). Once in place it should not take much longer than the old form.

QSS-7A and QSS-7E

• Completing A & B would require effort to split out. They asked for more clarification on the definition of individual; their current interpretation would put everything in B.

• D will be much larger than B because of interest.

• Company would like a definition of both individual and businesses.

• Are institutions considered individuals?

• PCE is not relevant for a brokerage.

• What is considered a business and what is considered a government?

• Initial setup to break out data will take much longer (hours). Once in place it should not take much longer than the old form.

QSS-8A and QSS-8E

• Are mutual funds businesses or individuals?

• Should non U.S. governments be counted as businesses or government?

Recommendation:

• Consider including a definition for class of customer, specifically for individuals.

• Consider adding “participations” and “remittances” to the definitions

Action:

• Brief definition of individuals will be included on the form.

Individual - Refers to a person or household spending, investing, or saving their own money on their own behalf. All money mangers in any form (such as fund managers or investment advisors) or investment funds should be treated as businesses.

• Participation will be added as an example in the definition of Loan Fees in QSS-6A and QSS-6E.

• Remittances will be added as an example in the definition of Other Fees QSS-6A and QSS-6E.

BSR-12 Cognitive Interview Protocol

Accommodations

Quarterly Services Staffs

January 24, 2012

Research Questions

• Will the respondents be able to respond to this survey on a quarterly basis?

• Will the sales, revenue, or receipt data be available on a quarterly basis?

• Will the respondents connect the QSS form to ARTS?

Expected Interview Length

1 hour

Materials Needed

• 2-3 copies of the industry-specific accommodations form

• Business cards

Introduction

• Survey purpose: to produce data on services economic activity within the U.S. on a quarterly basis.

• Meeting purpose:

  • To understand the process used to answer questions

  • To get feedback on the form – not testing you. Get info on interpretations, impressions, opinions.

  • No right or wrong answers. All feedback is useful.

• Meeting structure: General and specific questions about your business and the form itself. Some questions may seem odd/redundant because we’re not accountants, and because we’re meeting with various businesses that have different ways of keeping records.

General Probes

• In your own words, what was that question asking?

• What are you thinking about?

• Can you tell me more about that?

• You answered, “…” because…?

• I want to make sure I understand, can you explain that again?

• Can you tell me how you arrived at that answer? What did you include? What did you exclude? What records did you use?

• Reflect back on R’s answer and ask R to correct (“I want to make sure I have it right. I think you said, “…?”)

• Do you have any examples of your records we can view? (ask to see the ledgers)

Before the Questionnaire

• Tell me a little about your role in this business. How long have you been here, and what are your major responsibilities?

• Are you the person primarily responsible for government forms? Have you filled out government forms in the past? Are you responsible for any other forms?

• Generally speaking, how do government forms get completed?

• Do the forms come to you directly, or is there some sort of internal system or process that these forms go through before arriving at your desk?

Now I’m going to hand you a copy of the questionnaire. We’ll go through it page by page. Please write on it, and complete it as if we weren’t here. I’ll ask you questions when you’ve finished the page. Does this look like a form you have completed in the past? If so, do you recall how many times you have completed this form before? What years?

Cover Page and General Instructions

• Did you read the General Instructions?

• What is your reaction?

• Is there anything else you would like to be included?

Survey Coverage (QSS)

• Did you read the Survey Coverage?

• How easy or difficult was it for you to understand these instructions? Do these instructions hold any meaning for your company?

• Would you include or exclude any information in this instruction?

Sales, Receipts, or Revenue (ALL)

• Did you happen to read the include/exclude list? Is there anything on those lists that seems odd or out of place to you?

• Is there anything listed on the include list that you normally exclude? (Vice Versa)

• 5A - How easy or difficult would it be to report your sales, receipts, or revenue on a quarterly basis?

• How much time would this take you?

• 5B - Does the question make sense to you? (probe further here based on response)

• 5C - Are you willing to report sales, receipts, or revenue for hotels you manage?

• 5C - How easy or difficult would it be to report the sales, receipts, or revenue for the hotels you manage on a quarterly basis?

• What, if any, records would you need to reference in devising your response? When are those records typically available?

Wrap-up

• How much time would it take you to complete this report (actual time, not elapsed time)?

• Are you the person who would probably answer these questions, or would someone else answer them?

• Would the information requested on this form generally be available to you, or would information have to come from someplace else?

  • How would you go about getting that information?

  • Would you pass the form over to that department, or just get information from them over the phone/e-mail?

• Overall, how easy or difficult was it to complete this form?

• Is it clear that this form is associated with the Annual Retail Trade Survey (ARTS)? (probe further based on response)

Any other comments/suggestions?