Cognitive Interviewing Report (Round 1)

Findings and Recommendations from
Cognitive Testing for the
Government Units Survey (GUS),
Round 1
Prepared for:
Debra Coaxum (GOVS)

Prepared by:
Rebecca L. Morrison (Response Improvement Research Staff)
Kristin Stettler (Response Improvement Research Staff)

Office of Economic Planning & Innovation
Economic Programs Directorate
U.S. Census Bureau

November 23, 2010

Table of Contents
Research Objectives ...................................................................................................................................... 4
Research Methodology ................................................................................................................................. 4
General Comments and Recommendations ................................................................................................. 5
Finding #1: “Your government” ............................................................................................................ 5
Finding #2: Time estimate ..................................................................................................................... 5
Finding #3: Appropriate respondent..................................................................................................... 5
Finding #4: Electronic Reporting ........................................................................................................... 6
Specific Comments and Recommendations ................................................................................................. 6
Finding #5: Dependent units ................................................................................................................. 6
Finding #6: Mailing Address .................................................................................................................. 7
Finding #7: Physical Address ................................................................................................................. 7
Finding #8: Fiscal Year ........................................................................................................................... 7
Finding #9: Year Formed ....................................................................................................................... 7
Finding #10: Homerule Charter............................................................................................................. 7
Finding #11: Debt .................................................................................................................................. 8
Finding #12: Licenses ............................................................................................................................ 8
Finding #13: Taxes ................................................................................................................................. 9
Finding #14: Retirement/Pension Plans................................................................................................ 9
Finding #15: Government Activity ...................................................................................................... 10
Finding #16: Public Services ................................................................................................................ 11
Finding #17: Judicial and Legal Activities ............................................................................................ 12
Finding #18: Finance Questions .......................................................................................................... 13
Finding #19: Other Comments ............................................................................................................ 14
About the Response Improvement Research Staff (RIRS) .......................................................................... 16
Appendix A: Cognitive Research Methodology .......................................................................................... 17
Appendix B: Cognitive Interview Protocol .................................................................................................. 18
General Probes:................................................................................................................................... 18
8 (homerule charter): .......................................................................................................................... 18
9 (debt):............................................................................................................................................... 18
10 (license/permit fees): ..................................................................................................................... 18
GUS Round 1 Findings & Recommendations
Page 2 of 25

11 (taxes):............................................................................................................................................ 18
31 (housing/redevelopment projects): ............................................................................................... 18
32 (natural resources): ........................................................................................................................ 18
33 (recreational/cultural facilities): .................................................................................................... 19
34 (highways, streets, etc.): ................................................................................................................ 19
37/38 (EMS/fire): ................................................................................................................................ 19
39 (regulation/inspection services): ................................................................................................... 19
44 (correctional facilities): .................................................................................................................. 19
Wrap Up: ............................................................................................................................................. 19
Appendix C: Questionnaire Used During Round 1 Testing ......................................................................... 20

GUS Round 1 Findings & Recommendations
Page 3 of 25

Research Objectives
•
•

Respondents’ ability to provide the requested data
Respondents’ understanding of the wording and terminology used on the questionnaire, and the form layout

Research Methodology
During the week of November 15, 2010, sixteen interviews were conducted in New Jersey and the metropolitan Atlanta
area. These interviews were split among the following types of governments:
Type of Government

Atlanta

New Jersey

Total

Counties

2

0

2

Cities/Towns

4

3

7

Special Districts

3

4

7

Total

9

7

16

Interviews were conducted using concurrent and retrospective think-aloud techniques, with follow-up probes. A further
discussion of cognitive interview methodology can be found in Appendix A. Appendix B contains a copy of the protocol
that was used during the interviews. A copy of the questionnaire that was used during testing can be found in Appendix
C.
This document presents the highlights of the testing, and makes draft recommendations for changing the questionnaire
for Round 2.

GUS Round 1 Findings & Recommendations
Page 4 of 25

General Comments and Recommendations
Finding #1: “Your government”
Some respondents, especially those representing special districts, had issues with the phrase “your government.” Those
respondents did not consider themselves to be governments, and suggested using some other phrase. Also, sometimes
the respondents bounced around, sometimes thinking about their local government, and sometimes about a higher
level government that offered the service being discussed. Sometimes it seemed the respondents were thinking about
whether “their citizens” had access to the services, rather than who was providing.
Recommendation:
o
o

Consider adding an instruction to 1, along the lines of “By ‘your government,’ we mean the unit of
government listed in the mailing address above.”
Consider using “your government/entity,” “your government/authority,” “your local government” or
some other phrase.

Resolution: An instruction will be added to 1. The instruction, which will appear in italics following the question,
will read “By ‘your government,’ we mean the unit of government listed in the mailing address above.”
Questions will continue to refer to “your government.”
Finding #2: Time estimate
Most respondents thought this form would not take a long time to complete. Estimates ranged from 15 minutes to one
hour, generally.
Recommendation: none.
Finding #3: Appropriate respondent
Respondents in Atlanta had the following job titles:
•
•
•
•
•
•
•
•
•

City clerk
Assistant city manager
Finance director
GIS coordinator/manager
Public affairs manager
County manager
County Support Services Agency Director
Executive Director
Authority Liaison

Respondents in New Jersey had the following job titles:
•
•
•

Secretary
Director
Office administrator
GUS Round 1 Findings & Recommendations
Page 5 of 25

•
•
•
•
•
•
•

Deputy clerk
HR coordinator
Chief Financial Officer (CFO)
Administrative manager
Commissioner
Clerk
Manager

For the most part, respondents we met with thought they would be completing the GUS, and that it would get routed to
them directly. Addressing the questionnaire to the highest elected official (e.g., mayor, chair of the county board) may
be appropriate, though one respondent noted that this could be problematic, as they are not the ones with the
knowledge required for completing the form. For special districts, the title of the person who should receive the
questionnaire is not so clear. General suggestions included “executive director” or “administrator.” For specific types of
special districts, there were more specific suggestions (e.g., “commissioner” for a fire district, or “district manager” for a
soil conservation district).
Recommendation: none.
Finding #4: Electronic Reporting
Respondents generally said they would be interested in reporting electronically for the GUS. Some mentioned the
following features would be especially useful:
•
•
•

Ability to do only one section at a time, rather than the whole thing all at once
Ability to keep a paper copy of the form
The system keeps her answers for next time, even if she gets kicked out due to inactivity.
Recommendation: Include these features when developing the electronic reporting system and emphasize them
when encouraging electronic response.

Specific Comments and Recommendations
Finding #5: Dependent units (2)
In Question 2, quite a few respondents were not sure what a “dependent” government was. Many of the respondents
appeared to answer appropriately. Cities and counties answered no, and the Atlanta special districts answered yes, and
marked the appropriate unit they were dependent upon. In New Jersey, it was more of a mixed bag. The special
districts weren’t sure how to respond, especially those that receive state funding or follow state regulations. For some
special districts, the dependency is not of a financial nature, but is rather of a “personnel” nature – the unit was
dependent upon counties to identify persons to serve on the board, so the respondent would mark “County.” Another
special district, however, in a similar situation, would mark “Other” rather than “County.” Confidence in units’ ability to
correctly answer yes/no is high, but we have more limited confidence in respondents’ ability to correctly identify the
government unit they are dependent upon.
Recommendation: Consider clarifying or adding a definition.
GUS Round 1 Findings & Recommendations
Page 6 of 25

Resolution: Christopher Pece will wordsmith the question to clarify what a dependent unit is.
Finding #6: Mailing Address (3)
Some of the addresses currently on file point to the homes of elected officials, or to generic mailboxes that the unit will
soon be dropping. Be advised that significant time and/or effort may be required in order to bring the master address
file up to date.
Recommendation: none.
Finding #7: Physical Address (4)
Generally, this question performed well. There were a couple of special circumstances worth noting, however. In the
first case, a city operated at two locations: city hall (located close to the city’s downtown core) and the city offices
(located elsewhere). In this case, the respondent wrote in the city office location, since that’s where the mayor’s office
is located. In the second case, the respondents noted that they operated out of 400 buildings located throughout the
county. In that case, they decided to make sure the address pointed to the building that housed most of the county
government offices, including the county manager and chair of the county board of commissioners.
Recommendation: none.
Finding #8: Fiscal Year (6)
In Georgia, local governments can set their own fiscal years, which may or may not coincide with the state’s fiscal year.
However, respondents did not appear to have any problems identifying their fiscal year end date. In New Jersey, there
was quite a variety of fiscal dates as well, and no problems of identification.
Recommendation: none.
Finding #9: Year Formed (7)
If respondents did not know the year off the top of their head, it was generally easily available. One respondent
consulted her identification badge, because it had the city’s seal (and date) on it. Other respondents looked at their
business cards. One or two respondents said they would have to ask someone else.
Recommendation: none.
Finding #10: Homerule Charter (8)
Some respondents were not sure what this was, so they left it blank. Others knew their government did not have one
and answered “No.” Others would ask an attorney for guidance. This was one item where we heard “I have no idea
what that is” from many respondents. Some hypothesized that if they had any sort of charter or resolution, it might fit
this definition. Several respondents checked yes and entered a date, but it was for the year formed, not a homerule
charter, per se.
Recommendation: Be advised that there may be misreporting or item nonresponse on this question.
Resolution: This question will be deleted, in favor of emphasizing key phrases within the debt, license/permit
fees, and taxes questions.

GUS Round 1 Findings & Recommendations
Page 7 of 25

Finding #11: Debt (9)
Generally, the debt question performed as expected. The Atlanta respondents were able to identify the types of debt
their government were authorized to issue, and mostly understood that the question asked about authority to issue
debt rather than actual debt. In NJ, several respondents weren’t sure about their authority, and felt more comfortable
answering for types of debt they knew they had issued in recent years.
•
•
•

•

•
•

Quite a few respondents did not understand the term “utilized debt,” and several other respondents did not
understand “long-term conduit debt.” Two respondents said No to all those they weren’t familiar with.
Some respondents would enlist the aid of a financial person in order to appropriately answer these questions.
To several respondents, C and D seemed similar and could be combined. If not, it may be helpful to clarify
“general obligation” – is it debt or something else? Also, should revenue bond be plural? Is “utilized debt” even
necessary? Could be combined with revenue bonds (E and F)?
One county suggested adding a new type of debt to this question that covers public-private partnership debt,
such as tax allocation districts (TADs) or TIPs (acronym unknown, but possibly some variation on Tax
Improvement or Tax Increment). Apparently, this type of debt is gaining popularity in places like Illinois and
Texas. In these cases, a developer agrees to develop property if the public finances some part of the
infrastructure (e.g., sidewalks, water).
One respondent couldn’t decide between Yes and No, because they can enter into debt, but only with voter
approval. Finally, said No to A-F and Yes to G, writing in “with voter approval”.
A soil conservation district finally decided to say No to all, since they can “incur” debt, but not “issue” debt.
Recommendation:
o
o
o

Consider adding definitions of “utilized debt” and “long-term conduit debt.”
Consider putting “authority to issue” in bold text and perhaps clarifying, for example (whether or not it
has been used).
Consider adding public-private partnership debt to the list.

Resolution: Public-private partnership debt will not be added to the list. Utilized debt may be deleted. Lines C
and D will be combined into one item. The phrase “authority to issue” will be in bold text.
Finding #12: Licenses (10)
• There was some confusion regarding the meaning of “amusement licenses.” Does this include temporary
carnivals that set up in parking lots? Does it include licenses for adult entertainment establishments?
• Are franchise fees supposed to be included in public utilities licenses? (Or do they go in the taxes section?)
• Several NJ respondents mentioned “mercantile fees” as an example of G. occupation and businesses license.
• Some of the write-in responses: massage therapists, exotic dancers, building permits, demolition/construction
permits, impact fee, game room license, bond financing fees (for joint development authorities), 251 chapter
permit fees, food handlers license, planning and zoning fees, mobile home licenses, junk yard licenses, dog
kennel licenses, dog licenses, sewer connection permits, fire marshal permit fees, pesticide application, fertilizer
application, peddler and solicitation fees.
Recommendation:
GUS Round 1 Findings & Recommendations
Page 8 of 25

o
o
o
o

Consider adding a definition of “amusement licenses,” or provide some examples.
Clarify where respondents should report items such as franchise fees.
Consider putting “authority to impose” in bold text text and perhaps clarifying, for example (whether or
not it has been used).
Prior to item 10, the section header should read “License and Permit Fees.”

Resolution: The phrase “authority to impose” will be in bold text. GOVS staff will write some clarification for the
“amusements license” line item. The phrase “franchise fees” will be added where appropriate.
Finding #13: Taxes (11)
• Should special purpose local-option sales taxes (SPLOST) be considered when answering Line B (general sales
and gross receipts taxes)?
• One Atlanta respondent focused on whether or not government collects them, not necessarily whether or not
the government is authorized to levy them. This issue came up in NJ as well (since towns collect the taxes for the
other governmental units).
• One soil conservation district in NJ did think they had the authority to levey taxes, but said No to all, since it’s
never been done in NJ.
• One NJ township respondent said she would call the NJ local government services office and ask them.
• Missing from the list: hotel/motel, rental cars, user fees, impact fees, fines & forfeitures.
Recommendation:
o
o
o

Consider clarifying where SPLOSTs and other such taxes should be reported.
Consider bolding the phrase “authority to levy” within the question to emphasize this key point.
Prior to item 11, change the section header to “Taxes.”

Resolution: The phrase “authority to levy” will be in bold text. Line B will be modified to read “General sales and
gross receipts taxes, including special purpose local option sales taxes.”
Finding #14: Retirement/Pension Plans (12-16)
• Overall, in the Atlanta interviews, respondents seemed to be able to answer the yes/no parts of the questions
without difficulty. Not many respondents listed any of the plans, and if they did, it was vague (e.g., “403(b),
VALIC,” “deferred comp.”). Conversely, in the New Jersey interviews, respondents were often quite confused
about whether they should include the pension plans run by the state in the earlier questions. They could list
the types of plans (PERS, DCRP), but not the specific company that administers the plans. Universally,
respondents did not provide address information for the plans (we didn’t ask for it!).
• “Operate” may be cause for concern. Some respondents noted, for example, that they provide or offer various
benefit plans, but that they’re not processing health insurance claims.
• None of the special districts interviewed in Atlanta had employees. All correctly answered “no” to each of the
retirement/pension plan questions.
• In New Jersey, it would have been much clearer to ask question 16 first, to determine whether the government
operates their own plan, or uses another government’s plan, then follow up with the details of the types of
plans offered.
GUS Round 1 Findings & Recommendations
Page 9 of 25

•

•

The definitions seemed a bit too long and confusing to many respondents. Is it necessary for employers to
contribute to both types of plans? No respondents in NJ had any hybrid plans, although one respondent said
Yes, because she thought it meant a “bridge” plan when an employee is switching positions.. Simplify the
definitions if possible.
OPEB seemed an unnecessary acronym and a confusing concept to respondents. Based on the list of examples,
it wasn’t clear to all respondents that these are ONLY offered POST-employment. Several NJ respondents
answered Yes and listed things that were offered to current employees, not retirees, such as life insurance.
Recommendation:
o Consider moving question 16 so that it appears first in this series (and reword as appropriate).
o Be advised that the type of information that respondents write into the box may be too vague for the
needs of the survey.
o Consider changing “operate” to “provide” or “offer.”
o Simplify definitions. Include examples of state/local plans in 13 (such as 457, 411H or 414H).
o Add “s” to end of Section header.
o Consider deleting OPEB and clarifying definition.
Resolution: The question series will be modified. The first question (new question 12) will ask, “Whose
retirement and pension plans do employees of your government participate in? Mark all that apply.” The
response options will be: “This government’s own plan,” “Another government’s plan,” and “No plans are
offered to employees (Skip to 17).” Question 13 will be the defined benefit question, question 14 will ask about
defined contribution plans, 15 will ask about hybrid plans and 16 will ask about other post-employment benefit
plans. The definition for 14 will be modified to include 457 plans. The definition for 16 will be modified so that
it contains 2-3 examples, all of which include the phrase “post-employment” at the beginning. There is still
some discussion about changing “operate” to “provide” or “offer.”

Finding #15: Government Activity (17-34)
• Several respondents in Georgia giggled or laughed when they got to the question about operating a liquor store.
One respondent commented, “Some days, maybe we should.”
• 17 (water supply utility) and 28 (water supply system) were very similar questions. Most respondents answered
the same way for both, even though they thought it was odd. One county thought it would be clearer to change
“supply” to “distribution” – that county buys water at wholesale from the local water authority, then
distributes/sells it at retail.
• 25: “Offer” may be a little vague. Also, one respondent mentioned that they don’t provide “collection” (must
drive to dump) but do provide “disposal”.
• 27-30 (water/sewer): A couple respondents answered No to sewage and water, then wrote in “N/A” for the
questions about separate systems and separate records. Several respondents commented that 29 and 30
seemed repetitive. Consider the intent of these questions and whether both are needed. Clarify if necessary.
• 31 (housing): Most Atlanta respondents thought this question asked about Section 8 housing. If a government
used community development block grants (CDBGs), even if they weren’t for housing, they typically answered
GUS Round 1 Findings & Recommendations
Page 10 of 25

•

•
•

•

yes to the question. In NJ, several respondents weren’t sure how much “support” was required before they
could say Yes. The length/complexity of this question made it difficult for respondents to parse.
32 (natural resources): Many seemed to get hung up on the word “regulate.” If governments were involved
with making sure developers complied with regulations, it wasn’t clear how they should answer (some answered
yes, others no). Also, the word “industries” seemed to confuse respondents and cause them to overthink the
question. Again, the length/complexity of this question made it difficult for respondents to parse. One
respondent suggested wording: “Does your government regulate anything to do with land or water?”
33 (recreational/cultural): This question appeared to perform as expected.
34 (roads): For the most part, this question appeared to perform as expected. It is worth noting the following
comments from 2 different special districts in Georgia, both of whom answered “no.”
o They do engineering studies for road projects, but they don’t own the roads and are not responsible for
their upkeep.
o They may choose to be involved in road projects, but they’re not responsible for them. (This was an
issue in NJ as well.)
Most respondents seemed to read the Exclude regulation of privately owned utilities instruction at the top.
However, if you wanted to ensure that this was handled consistently, you might consider turning this into a
Yes/No question and then instructing them to exclude.
Recommendation:
o

o
o
o

17 and 28: Decide which word or concept needs to be measured, then delete the other question. If 17 is
the appropriate concept, then keep it, but move it to 28. (In other words, keep all water/sewer
questions together.)
27-30: Add skips if No to both 27 and 28?
31: Consider separating the concepts of public housing and CDBGs into two separate questions.
32: Consider breaking this question into component pieces, or otherwise clarifying what is meant by
“regulate industries.”

Resolution: GOVS will decide whether to ask about “water supply utilities” or “water supply systems.” The
question will be kept in such a way as to keep water and sewer questions together. Skips will be used when
appropriate. GOVS will decide whether or not to separate public housing and CDBGs, and rework the questions
as necessary. They will also rework question 32, and may break it into multiple questions.
Finding #16: Public Services (35-41)
• Police/Fire/EMS:
o For the most part, respondents answered appropriately.
o Some saw the skip instruction, others didn’t. If respondents did not see the skip, they typically
answered “No” to the follow-up question about whether or not fire and EMS were separate, or wrote in
“N/A.”
o One situation worth noting: one Georgia county’s first responders are firefighters, who are also trained
EMTs. They provide first response care, but are not involved with transportation to hospitals – private
ambulance services do that. This county wasn’t sure if they should answer yes or no to the EMS
question.
GUS Round 1 Findings & Recommendations
Page 11 of 25

•

•

•
•

Regulation/inspection: This question generally performed as expected. Several noted that they were not
involved in regulating financial institutions. Examples included building inspections for certificates of occupancy,
fire inspections, electric inspections, construction regulations, etc.
Libraries: These questions were easily and correctly answered, across the board. Two situations are worth
noting:
o A Georgia county has a library focused on African-American history. The respondent considered this an
academic library, because of its scholarly focus, even though it’s not affiliated with a university. It is
unclear whether this respondent answered correctly or not.
o A township in New Jersey answered yes, even though it was a county library that was funded through a
special county library tax. Because the library was in their town, and the town owned the building and
assisted with maintenance, they included it.
Welfare: Most respondents thought of things like public housing, summer lunch programs, food stamps,
unemployment, etc.
In NJ, several respondents weren’t sure how to answer this series of questions, since they are not legally
required to provide the services, but choose to.
Recommendation: Consider changing the wording to: “Does your government provide…?”
Resolution: The recommendation was accepted. (Also reword 34 similarly, if possible.)

Finding #17: Judicial and Legal Activities (42-44)
• Many of the Atlanta respondents answered these questions with little difficulty. Some NJ respondents,
however, were not sure what they should be considering when answering 42. Several respondents interpreted
this to mean “Do you operate a court system?” or “Do you have a lawyer on staff?” or “Can you issue tickets?”
One respondent considered whether he should answer Yes since the Board can make rulings, fine people, etc.
(but answered No). One township respondent couldn’t decide how to answer, since the council pays for the
court, but doesn’t “perform”. One fire district answered Yes, since they enforce laws (like inspections, after
which they can issue fines, shut down).
• In 43, it was not clear what the intent of the question was. Since we already know the type of government,
what are we trying to learn in this question? Also, “municipality” and “city” or “town/township” were
considered interchangeable terms. One respondent suggested using one or the other.
• One county noted that the judicial system is about 1/3 of the budget, but there are only a few questions about it
on the questionnaire.
Recommendation:
o
o

The intent of 42 needs clarification. Reword or delete item 42 as necessary.
The intent of 43 needs clarification. Reword or delete item 43 as necessary. In item 43, consider
deleting “municipality” from the answer options.

Resolution: The correctional facilities question will move so that it appears before question 42. Question 42 will
be modified so that it reads “Does your government operate its own court system, including [examples to be
inserted by GOVS staff]?” Response options will be yes and no.
GUS Round 1 Findings & Recommendations
Page 12 of 25

Finding #18: Finance Questions (45-48)
• Respondents mentioned using one of several references to answer these questions: CAFR, Finance Director,
budget, W-2 or W-3, audit report, annual budget.
• Revenue: one respondent suggested using “collect” rather than “raise.” The same respondent thought it would
be a good idea to clarify that this number should include “all sources” of revenue, or others might only report
figures from the general fund. The issue of which funds to include was also raised by several respondents in NJ.
They weren’t sure if capital funds, utilities funds or others should be included (easier not to). Might ask for
separately, if want all. One respondent mentioned this is “operating revenue”.
• Expenditures: One respondent mentioned this is “operating expenses”.
• Debt: One respondent noted that the debt listed in the budget is just the principal and interest part, not the
total debt outstanding.
• One respondent said she would not want to estimate because “close only counts in horseshoes and hand
grenades.”
• One special district has no employees. Rather, all staff members are contract labor, and reported by 1099s. The
respondent would add up the value of the 1099s, and report that figure for payroll.
• The respondent at another special district made two passes at this question series. The first time, he attempted
to estimate the value of projects involved with each member of the district. The second time, he thought only
of what the special district had on its books.
• “Raise” didn’t sound right to many of the NJ respondents, who noted that you “raise” taxes, but not other types
of revenue, such as grants.
• The excludes in 48 were confusing to several respondents. Consider whether they are important/useful and can
be deleted. One respondent had a bit of difficulty since they had two types of employees, one of which is
reimbursed under a grant and their total included other things, like cell phones, insurance, etc.
• Several respondents weren’t sure if fringe benefits should be included or excluded (most excluded).
Recommendation:
o
o
o

o
o
o

In 45, change “raise” to “receive.” Consider modifying the instruction to read “Include all sources of
revenue, such as taxes, charges, state or Federal grants, etc. ..”
In 47, use the same list of examples as in 9, with an etc. at the end.
In 48, consider clarifying whether or not 1099 employees should be reported. This could be
accomplished by modifying the exclude instruction to read, for example, “Exclude the value of contract
labor, lump sum payments…”
Move 48 after 46, since it is related to expenditures and often comes from the same data source.
In 48, clarify instructions and whether fringe benefits should be included or excluded.
Be advised that question 48 may yield data that appears confusing, depending on the nature of the
government unit. Follow-up phone calls or e-mails may be necessary.

Resolution:
o

Christopher Pece will be consulted about whether or not all funds should be included in the revenue
question. If so, an instruction will be added.
GUS Round 1 Findings & Recommendations
Page 13 of 25

o
o

o

Question 45 will be modified to read “…did your government receive…”
The payroll question will move so that it follows the expenditure question. GOVS will decide whether
fringe benefits should be included or excluded. The exclude instruction will be modified to read
“Exclude the value of contract labor, lump sum payments…”
In the debt question, the examples will be the same as the sub-items in question 9.

Finding #19: Other Comments
The following changes should be made to the second iCADE draft (dated 11/16/2010):
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•

•
•
•
•

•
•

Change the title and subtitle to “2011 Government Units Survey | Census of Governments.”
On page 1, change “Please respond by March 4, 2011” to “Due Date: March 4, 2011.”
Section headers (and blue lines between sections) will be removed.
In 2, delete “Identify the relationship” and move the response options (e.g., county dependent, city dependent)
to the same line as the “yes” answer option. Delete “or” after “office”.
In 4, move the question mark so that it is after “mailing address.” Add a period to the parenthetical instruction.
In 5, put “Enter the web address” in italics.
In 8 (if this question is kept), put “Enter the year your homerule charter was adopted” in italics and delete the
colon.
In 9-11, modify the line numbers so that they are lower-case and bold (to match the line numbers in 41 and 44).
In 10, capitalize the first word of each line item only, not every word.
In 11, use either “tax” or “taxes” but not both. Capitalize the first word of each line item only, not every word.
In 12-16, the words “List these …plans below” should appear in plain text, not italics.
In 12, add the word “benefit” to the instruction so that it reads “List these defined benefit pension plans below.”
In 14, why does it refer to “system” rather than “plan”?
In 27 and 30, consistently use “sewage” or “sewer.”
In 29, correct “aystem” to “system.”
In 31:
o Change “and” to “or” so that it reads “…redevelopment projects or other…”
o Add a space between “activities” and “to.”
o Add more examples to the parenthetical instruction, and end the instruction with “etc.”
In 32, change the parenthetical instruction so that it reads “…fisheries, forestry, etc.”
In 33, change the parenthetical instruction so that it reads “…drama, celebrations, etc.”
In 34, add a space between “related” and “structures.”
In 39:
o Rephrase the question and instruction so that it reads “Does your government provide regulation or
inspection services? (Include building or construction regulation, licensing or regulation of financial
institutions, examination of professional occupations, etc.)”
o Adjust the line breaks of the question and parenthetical so that the text stretches further across the
page.
In 42, add “and legal” after judicial.
In 43, change “operations” to “activities.”
GUS Round 1 Findings & Recommendations
Page 14 of 25

•
•

In 46-48, delete “Expenditure,” “Debt,” and “Payroll.”
In the Thank you, change “Form” to “form”.

GUS Round 1 Findings & Recommendations
Page 15 of 25

About the Response Improvement Research Staff (RIRS)
The Response Improvement Research Staff (RIRS) in the Office of Economic Planning and Innovation (OEPI) assists
economic survey program areas and other governmental agencies with research associated with the behavioral aspects
of survey response and data collection. The mission of RIRS is to improve data quality in surveys while reducing survey
nonresponse and respondent burden. This mission is achieved by:
•

•
•

Conducting expert reviews, cognitive pretesting, site visits and usability testing, along with post-collection
evaluation methods, to assess the effectiveness and efficiency of the data collection instruments and associated
materials.
Assisting program areas with the development and use of nonresponse reduction methods and contact
strategies.
Conducting empirical research to help better understand behavioral aspects of survey response, with the aim of
identifying areas for further improvement as well as evaluating the effectiveness of qualitative research.

For more information on how RIRS can assist your economic survey program area or agency, please visit
http://cww.census.gov/econ/oepi/RIRS/ or contact the staff chief, Diane Willimack.

GUS Round 1 Findings & Recommendations
Page 16 of 25

Appendix A: Cognitive Research Methodology
Cognitive research is used in survey methodology “(a) to understand the thought processes used to answer survey
items, and (b) to use this knowledge to find better ways of constructing, formulating, and asking survey questions”
(Forsyth and Lessler, 1991). 1 Cognitive interviews traditionally focus on the four steps of Tourangeau’s (1984) cognitive
response model: comprehension, retrieval, judgment, and communication/reporting. 2 Comprehension refers to the
respondent’s interpretation and understanding of the question’s language, structure, and grammar. In order to answer
the question, a respondent must understand what information is being requested on the survey. Retrieval is the step
where relevant information is obtained, either from records or from memory. The next step, judgment, describes the
respondent’s evaluation of the completeness or relevance of the data obtained. It is here that estimates are made
based on partial or incomplete data. The last step, communication or reporting, deals with mapping the response to the
answer space provided and possibly altering the answer.
While Tourangeau’s model is suitable for household and social surveys, the establishment survey setting presents
additional factors that must be considered. First, instead of or in addition to a reliance on memory, establishment
surveys rely heavily on records and the information contained within them. Second, organizations tend to have
distributed knowledge. Some people are experts in one type of information, while others keep information about
something else. Third, competing priorities, both for the organization and the individual(s) completing the
questionnaire, mean that the survey sometimes does not receive the amount of attention that researchers and data
collectors would like. Finally, organizations regularly authorize only a few individuals to release data. If the data
provider is not authorized to release the data, an additional step must be added to the response process. Tourangeau’s
model was expanded by Sudman et al (2000) to account for these factors. 3

1

Forsyth, B.H. and Lessler, J.T. (1991). “Cognitive Laboratory Methods: A Taxonomy.” In Measurement Errors in Surveys, P.P. Biemer, R.M. Groves,
L.E. Lyberg, N.A. Mathiowitz, S. Sudman (eds). New York: John Wiley & Sons, Inc.

2
Tourangeau, R. (1984). “Cognitive Sciences and Survey Methods.” In Cognitive Aspects of Survey Methodology, T.B. Jabine, M.L. Straf, J.M. Tanur,
and R. Tourangeau (eds). Washington, DC: National Academy Press.

3 Sudman, S., Willimack, D.K., Nichols, E., and Mesenbourg, T.L. (2000). “Exploratory Research at the U.S. Census Bureau on the Survey Response
Process in Large Companies.” Paper prepared for presentation at the Second International Conference on Establishment Surveys, Buffalo, NY.

GUS Round 1 Findings & Recommendations
Page 17 of 25

Appendix B: Cognitive Interview Protocol
This is the protocol that was used during cognitive interviews in New Jersey and the metropolitan Atlanta area.
General Probes:
• What were you thinking when you were answering this question?
• How did you arrive at your answer?
• You reported ______. Can you tell me more about that?
8 (homerule charter):
• In your own words, what do you think this question is asking?
• How did you arrive at your answer?
• What does the term “homerule charter” mean to you?
9 (debt):
• In your own words, what did you think this question was asking?
• How did you arrive at your answers?
• What does the term “debt” mean to you in this question?
• What does the term “long-term conduit debt” mean to you?
• If respondent reports “other debt,” probe that type of debt (see if it could possibly fall into another category):
You said your government has the authority to issue _____ type of debt. Can you tell me more about that kind
of debt?
• If the respondent answers “No” to all items: Does your government have the authority to issue debt?
10 (license/permit fees):
• If the respondent appears to have difficulty with any of the terms, probe.
• If the respondent answers “No” to all items: Does your government have the authority to impose license and
permit fees?
11 (taxes):
• In your own words, what did you think this question was asking?
• How did you arrive at your answers?
• What does the term “taxes” mean to you?
• If the respondent reports some “other type of tax, probe.
• If the respondent doesn’t understand other terms, probe.
• If the respondent answers “No” to all items: Does your government have the authority to levy taxes?
31 (housing/redevelopment projects):
• In your own words, what did you think this question was asking?
• How did you arrive at your answer?
• Probe these activities. Find out if the government contracts out the services.
32 (natural resources):
• In your own words, what did you think this question was asking?
• How did you arrive at your answer?
• Probe these activities. Find out if the government contracts out the services.
• Probe any of the terms that respondents might not understand.
GUS Round 1 Findings & Recommendations
Page 18 of 25

33 (recreational/cultural facilities):
• In your own words, what did you think this question was asking?
• How did you arrive at your answer?
• Probe these activities. Find out if the government contracts out the services.
• Probe any of the terms that respondents might not understand.
34 (highways, streets, etc.):
• In your own words, what did you think this question was asking?
• How did you arrive at your answer?
• Probe these activities. Find out if the government contracts out the services.
• Probe any of the terms that respondents might not understand.
37/38 (EMS/fire):
If yes to both, find out if funding sources are separated.
39 (regulation/inspection services):
• In your own words, what did you think this question was asking?
• How did you arrive at your answer?
• Probe these activities. Find out if the government contracts out the services.
• Probe any of the terms that respondents might not understand.
44 (correctional facilities):
• Did you include or exclude any privately-owned correctional facilities as you were answering this question?
Wrap Up:
• Any questions particularly easy to answer? Particularly difficult?
• Approximately how long do you think it would take to complete this survey, including the time spent to consult
with others, run/pull reports, look at records, etc.?
• Who in your government would be the best person to complete this form?

GUS Round 1 Findings & Recommendations
Page 19 of 25

Appendix C: Questionnaire Used During Round 1 Testing
The following pages contain the questionnaire that was used during the cognitive interviews in Georgia and New Jersey.

2011 GOVERNMENT UNITS SURVEY
Census of Governments
Draft (11/10/2010)
Mailing and Contact Information:
Please respond by March 4, 2011.
RETURN VIA MAIL TO:
U. S. Census Bureau
th
1201 East 10 Street
Jeffersonville, IN 47132-0001

Introduction and Survey Purpose:
This survey collects descriptive information on the basic characteristics of local governments in preparation for the 2012
Census of Governments. Data from this survey will also be used to produce the official count of local government units in
the United States and to update and verify the mailing addresses of government units.
General Instructions
• Use a blue or black ink pen to complete the survey form.
• Mark the box with an X next to the appropriate response for each question.

BACKGROUND INFORMATION
1. Has your government ceased to exist?
 Yes – Effective Date: Month (MM) _______ Day (DD) _______ Year (YYYY)________
 No
2. Is your government a dependent unit, agency, office, or etc. of another government?
 Yes --  County Dependent
 City Dependent
 Township Dependent
 No

Other

3. Is the mailing address listed above correct?
 Yes
 No
4. Is your government’s physical location the same as the location shown in the mailing address? (P.O. Box and rural
route addresses are not physical locations).
 Yes
 No -- Enter physical location._____________________________________
5. Does your government have an Internet website that contains information about your government and its activities?
 Yes -- Enter the web address (e.g. www.mylocalgov.state.us)_______________________________
 No

GUS Round 1 Findings & Recommendations
Page 20 of 25

6. On what date does your government’s fiscal year end?
Month (MM)________ Day (DD)___________

7. What year was your government was first incorporated or formed?
Year____________
8. Does your government have a homerule charter?
 Yes – Enter the year your homerule charter was adopted
 No

____________

DEBT
9. Does your government have the authority to issue any of the following types of debt?
(Mark “Yes” or “No” for each item)
A. Short-term debt (1 year or less)………………………………………………………………..Yes

 No

B. Long-term conduit debt………………………………………………………………………….Yes

 No

C. Long-term general purpose debt……………………………………………………………….Yes

 No

D. General obligation

……………………………………………………………………… Yes

 No

E. Revenue bond……………………………………………………………………………………Yes

 No

F. Utilized debt………………………………………………………………………………………Yes

 No

G. Other debt (Specify___________________________________)…………………………Yes

 No

LICENSING AND TAX
10. Does your government have the authority to impose any of the following types of license or permit fees?
(Mark “Yes” or “No” for each item)
A. Alcoholic beverages license………………………………………………………………….. Yes  No
B. Amusement license……………………………………………………………………………. Yes  No
C. Hunting and fishing license…………………………………………………………………..  Yes

 No

D. Motor vehicles license…………………………………………………………………………. Yes  No
E. Motor vehicle operators license……………………………………………………………… Yes

 No

F. Public utilities license………………………………………………………………………….. Yes

 No

G. Occupation and businesses license…………………………………………………………. Yes

 No

H. Other licenses or permit fees (Specify__________________________________)…… Yes

 No

GUS Round 1 Findings & Recommendations
Page 21 of 25

11. Does your government have the authority to levy any of the following types of taxes?
(Mark “Yes” or “No” for each item)
A. Property taxes …………………………………………………………………………….  Yes
B. General sales and gross receipts taxes………………………………………………….  Yes

 No
 No

C. Alcoholic beverages sales tax …………………………………………………………….. Yes  No
D. Amusements sales tax……………………………………………………………………… Yes  No
E. Motor fuels sales tax………………………………………………………………………… Yes  No
F. Insurance premiums sales tax……………………………………………………………… Yes  No
G. Pari-mutuels sales tax……………………………………………………………………… Yes  No
H. Public utilities sales tax……………………………………………………………………… Yes  No
I. Tobacco products sales tax………………………………………………………………… Yes  No
J. Individual income taxes…………………………………………………………………….. Yes  No
K. Corporation net income taxes………………………………………………………………. Yes  No
L. Death and gift taxes…………………………………………………………………………. Yes  No
M. Documentary and stock transfer taxes……………………………………………………. Yes  No
N. Severance taxes……………………………………………………………………………... Yes  No
O. Other taxes (Specify____________________________________)………………….. Yes  No

RETIREMENT/PENSION PLAN
12. Does your government operate any defined benefit pension plans for your own
employees? (Defined benefit pension plans that specify the benefit to be paid to
employees upon retirement, generally either a specific amount or a percentage of
compensation. Employer contributions are based on actuarial computations that include
the employee’s compensation and years of service and are not allocated to specific
accounts maintained for employees)………………………………………………………………..Yes

 No

List these defined pension plans below:
________________________________________________________________________________
13. Does your government operate any defined contribution pension plans for your own
employees? (Defined contribution plans are pension plans that define the employer
contributions to a separate account provided for each employee. The employee “benefit”
at retirement depends on the amount contributed and the results of the account’s activity.
Examples include profit sharing plans, money purchase (e.g, 401k, 403b) and stock bonus
plans (e.g., ESOPs)……………………………………………………………………………………Yes
List these defined contribution pension plans below:

GUS Round 1 Findings & Recommendations
Page 22 of 25

 No

________________________________________________________________________________
14. Does your government operate any hybrid pension plans for your own employees?
(A hybrid pension plan is one that combines the features of a defined benefit and a
defined contribution system.)…………………………………………………………………………. Yes  No
List these hybrid pension plans below:
________________________________________________________________________________
15. Does your government operate any other post-employment benefit (OPEB) plans
for your own employees? (For example, post-employment healthcare, long-term care,
life insurance, disability benefits and tuition assistance)…………………………………………… Yes

 No

List these post-employment benefit plans below:
_________________________________________________________________________________
16. Do the employees of your government participate in the pension or OPEB plan of
another government? (For example, state or county retirement systems)……………………….. Yes  No
List these plans below:
_________________________________________________________________________________

GOVERNMENT ACTIVITY
Exclude regulation of privately owned utilities when answering these questions.
17. Does your government operate a water supply utility? ………………………………………Yes

 No

18. Does your government operate a gas utility? ………………………………………………...Yes

 No

19. Does your government operate an electric utility?

……………………………………… Yes

 No

20. Does your government operate a transit authority? …………………………………………. Yes  No
21. Does your government operate a liquor store? ………………………………………………..Yes

 No

22. Does your government operate an airport? ................................................................ …..Yes

 No

23. Does your government operate a sea or in-land port? ………………………………………..Yes

 No

24. Does your government operate a hospital? .......................................................................Yes

 No

25. Does your government offer refuse collection? ………………………………………………..Yes

 No

26. Does your government operate a landfill? ………………………………………………………Yes

 No

27. Does your government operate a sewage system? …………………………………………… Yes

 No

28. Does your government operate a water supply system? …………………………………….. Yes

 No

29. Is the sewer system separate from the water system? ……………………………………….. Yes

 No

GUS Round 1 Findings & Recommendations
Page 23 of 25

 No

30. Do you keep separate records for sewer and water operations? ……………………………. Yes
31. Does your government construct, operate, or support housing and redevelopment
projects and other activities to promote or aid public and private housing or community
development? (For example, use of community development block grants)………………..Yes

 No

32. Does your government regulate industries that develop, utilize, or affect natural
resources by irrigation, drainage, flood control, soil conservation, or related
management and protection practices? (For example, land reclamation,
fisheries, and forestry)……………………………………………………………………………..Yes

 No

33. Does your government provide or support recreational and cultural facilities? (For
example, galleries, zoos, botanical gardens, memorials, community music, drama,
celebration, etc)……………………………………………………………………………………..Yes

 No

34. Is your government responsible for highways, streets, roads, alleys, bridges, tunnels,
ferry boats, or related structures?....................................................................................... Yes

 No

PUBLIC SERVICES
35. Is your government responsible for providing police protection service?...........................Yes

 No

36. Is your government responsible for providing fire protection services?..............................Yes

 No

37. Is your government responsible for providing emergency medical services (EMS)?
Yes  No – Skip to 39.
38. Are emergency medical services separate from the fire services? …………………………..Yes

 No

39. Is your government responsible for providing regulation and inspection services?
(For example, buildings and construction regulation, licensing and regulation of financial
institutions and the examination of professional occupations)…………………………………..Yes

 No

40. Does your government provide welfare services? ................................................................Yes

 No

41. Does your government provide any of the following types of library services? (Mark “Yes” or “No” for each item)
A. Public regional libraries………………………………………………………………………… Yes

 No

B. Public community libraries…………………………………………………………………….. Yes

 No

C. Academic libraries……………………………………………………………………………… Yes

 No

D. Law libraries…………………………………………………………………………………….. Yes

 No

E. Other library services (Specify______________________________)…………………… Yes

 No

JUDICIAL AND LEGAL ACTIVITIES
42. Does your government perform judicial and legal activities?
 Yes
 No – Skip to 45.

GUS Round 1 Findings & Recommendations
Page 24 of 25

43. At what level does your government perform its judicial and legal activities?
County Municipality City
Town or township
Other level (Specify_____________)

44. Does your government operate any of the following types of correctional facilities? (Mark “Yes” or “No” for each item)
A. Prison…………………………………………………………………………………….. Yes  No
B. Jail………………………………………………………………………………………… Yes  No
C. Juvenile correctional facility……………………………………………………………. Yes  No
D. Other correctional facility (Specify_____________)
 Yes  No

FINANCE
Estimates are acceptable.
45. How much revenue did your government raise in the last completed fiscal year?
(Include all taxes, charges, state or Federal grants, etc. Exclude borrowings. If none, enter 0).
$_________________________
46. How much did your government expend in the last completed fiscal year?
(Include salaries, wages, capital outlay, interest on debt, current operations, payments to other governments, etc. If
none, enter 0.)
$_________________________
47. How much outstanding debt did your government have at the end of the last completed fiscal year?
(Include general purpose long term debt, short-term debt, conduit debt. If none, enter 0).
$_________________________
48. What was your government’s annual gross payroll (before deductions) in the last completed fiscal year?
(Include salaries, wages, fees, or commissions, overtime, premium, night differential pay, bonuses and incentive
payments. Exclude lump sum payments, and the value of living quarters and subsistence allowances furnished for
employees.)
$_________________________

I. REMARKS (Please use this space for any explanations that may be important to understanding any of your responses)
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________

_________________________________________________________________________________________
Name of person to contact regarding this report
Title
Telephone
Fax
__________________________________
Internet e-mail address

THANK YOU FOR COMPLETING GOVERNMENT UNITS SURVEY

GUS Round 1 Findings & Recommendations
Page 25 of 25