Various Governments Area Pretesting Activities

12

2012 Census of Governments: Finance Component, Retirement

Round 2 Cognitive Interview Protocol

(This protocol is a guide – the questions presented here won’t necessarily be asked exactly as worded in the protocol or in this order. It is important to note that not all questions will be asked in every interview.)

Introduction

1. Purpose of visit:

Let me start by telling you a little about what we will be doing today. The Census of Governments is mailed out once every five years and collects comprehensive information about such topics as government revenue, expenditures, and financial assets.

We are in the process of re-designing the Census of Governments in order to better meet our data users’ needs. One of our goals is to assess how well the questions are able to capture the information we need. We need your feedback on how understandable the questions are and how you go about answering them.

2. Permission to audio-tape discussion? Have R sign consent form.

Before we get started, I'd like to audio tape this interview so I don't have to rely on my memory later. This session is confidential. Only persons connected with this project will have access to your tape. If that's all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

Background Information

4. Respondent Background

• Can you tell me a little about your role at this [pension system name]?

  • What are you responsible for?

• Can you tell me a little about the benefit plans that [pension system name] provides?

  • Are any of the plans defined benefit plans? Defined contribution plans? Both?

  • (If both) Are you able to differentiate between defined benefit funds and defined contribution funds?

  • (If both) Are you able to provide the same amount of details for plans?

• Does this plan also offer optional retiree healthcare benefits (OPEB)?

  • (If yes) Do you have the same amount of detail for this plan as you do for your other plans?

• How frequently do you report actuarial (liabilities) data in the annual report?

  • What is the lag time between the fiscal tear of your annual report and the actuarial information?

• Have you completed forms (i.e., surveys, censuses) from the Census Bureau for [government name]?

  • If so, which ones?

• Have you completed any other government forms (i.e., surveys, censuses)?

FORM NAVIGATION

• Watch how respondents enter responses.

• Note any confusion about the dot leaders when attached to “include/exclude” text.

• Notice if Rs read “include/exclude” text when it follows dot leaders to the text.

• Also note how respondents handle the bolded “include/exclude” text.

• Probe any difficulty with the form navigation.

INSTRUCTIONS

• Did you notice any of the instructions on the front of the form?

• Did you read any of them?

• What were they telling you?

• (If R did not notice) Could we do anything to make them stand out more?

PART 1: COVERAGE & ORG INFORMATION

Item 2: Type of system (New)

• What did you think this question was asking?

• How did you arrive at your answer?

• (Probe if R has trouble choosing or wants to mark more than one response option)

PART 3: PLAN INFORMATION

Items 6 & 7: Fiscal year ending date

• What did you think this question was asking?

• How did you arrive at your answer?

PART 4: MEMBERSHIP AND BENEFITS

Overall probes:

• What reference period were you using when answering these items?

• How is this information kept in your records? Are monthly figures in your CAFR, actuarial report, or both?

Item 8: Active and Inactive Members

• What did you think this item was asking?

• How did you arrive at your answer?

• What do “active” and “inactive” mean to you?

• Who are you including in these categories. (Rs tend to put retirees in the active category because they view them as active in the system)

• Is there another term you would use?

• (For inactive members) What does the term “vested” mean to you? “Non-vested”?

• How is this information kept in your records?

Item 9: Retirees and Beneficiaries

• What did you think this item was asking?

• How did you arrive at your answer?

• What does the term “retirees” mean to you? “Beneficiaries?”

• Is there another term you would use?

Item 10: Lump sum payments

• What did you think this item was asking?

• How did you come up with your answers? What were you including?

• How is this information kept in your records?

• What would you have to do to that information to report it in this question?

• What does the term “lump sum” mean to you?

• Is there another term you would use?

DROP:

• Do you consider Deferred Retirement Option Plan (DROP) payments to be lump-sum payments?

• Pay careful attention to how Rs account for DROP members

Withdrawals:

• What kinds of payments do you think you should include here?

• Make sure R reports items in this category correctly.

Lump-sum to survivors:

• (If R doesn’t indicate any payments in this category) Do you provide a death benefit to the families of retirees?

PART 5: RECEIPTS AND PAYMENTS

General probes for each item:

• What did you think this item was asking?

• How did you come up with your answers? What were you including?

• How is this information kept in your records?

• What would you have to do to that information to report it in this question?

Item 11: Receipts

Realized and unrealized gains

• What does the term “realized” mean to you (as in, “realized gains” and “realized losses?”)

• What does the term “unrealized” mean to you?

• Probe about how they decided how and where to report gains and losses – column header potentially confusing

Item 12: Payments

Withdrawals

• What kinds of withdrawals would fit into this category (make sure these withdrawals are recorded correctly – like rolling over into another account upon separation)

PART 6: CASH AND INVESTMENTS

General probes for each item:

• What did you think this item was asking?

• How did you come up with your answers? What were you including?

• How is this information kept in your records?

• What would you have to do to that information to report it in this question?

Item 13A: Cash and short-term investments

• What does the term “cash on hand” mean to you? Probe other sub-items.

• (If applicable)For 3f: What were you including in investments held in trust funds?

• What types of short-term investments for defined benefit/defined contribution/OPEB does this plan have?

Item 13B: Federal Government Securities

• What does the term “securities” mean to you? (If applicable) What were you including in your answer?

• What does the term “Federal treasury securities” mean to you? (If applicable) What were you including in your answer?

• What does the term “federally-sponsored agencies” mean to you? (If applicable) What were you including in your answer?

Item 13C: Corporate Bonds

• What does the term “corporate bonds” mean to you?

• What were you including in your answer? Make sure respondent is reporting Sallie Mae here.

• How easy or difficult would it be to include Sallie Mae in this question?

• Pay close attention to how they grapple with the include/exclude instructions.

Item 13D: Corporate Stocks

• What does the term “corporate stocks” mean to you?

Item 13E: Foreign and International Securities

• Is it feasible to be able to separate foreign stocks and foreign bonds?

Item 13F: Mortgages Held Directly

• What does “mortgages held directly” mean to you?

• (If applicable) What did you include in this category?

• Pay attention to how respondent grapples with layout

Item 13G: Other Non-governmental Securities

• What are you including in your response? ( Make sure R is excluding gov securities)

Item 13J: Other Investments

• Probe the “other - specify” responses – see if they belong in another category.

Items 13K and 13L: Mutual Funds and Hedge Funds

Mutual Funds:

• Pay attention to how Rs grapple with layout

• What do you think we mean by “mutual fund” here? (Make sure R isn’t reporting everything here when it should be in the other categories.

Hedge Funds:

• What does the term “hedge fund” mean to you? (If applicable) What were you including in your answer?

PART 7: ACTUARIAL INFORMATION ********FOR F-12 FORM ONLY********

General Probes for the section:

• What did you think this item was asking?

• How did you come up with your answer?

• How is this information kept in your records?

• What would you have to do to that information to report it in this question?

Item 15: Actuarial Accrued Liability (AAL)

• (If applicable) What were you including in your answer?

• How is this information kept in your records? How would you differentiate between active and inactive members?

• How would you provide information on how much AAL is for retirees?

Items 16 through 19: Employer Normal Cost and Covered Payroll

• Is it easy or difficult to report a percentage? (If diff) Is there some other way you would report this information?

Item 20: Annual Required Contribution (ARC)

• Is it easy or difficult to report a percentage? (If diff) Is there some other way you would report this information?

Item 21: Actuarial Cost Method

• Use general probes

Item 22: Investment Rate of Return or Discount Rate

• Is it easy or difficult to report a percentage? (If diff) Is there some other way you would report this information?

Item 23 and 24: COLA Adjustments

• Use general probes

• Make sure new two-part question is clear and Rs follow skip when applicable

Wrap Up

• What is your overall impression of it this form?

• What, if anything, did you like the most about the form? Least?

• Do you have any suggestions for how to make this form easier for you to fill out?