2010 ACS Content Test Evaluation Report - Computer and Internet

January 31, 2012

2012 AMERICAN COMMUNITY SURVEY RESEARCH AND EVALUATION REPORT
MEMORANDUM SERIES #ACS12-RER-08

MEMORANDUM FOR

ACS Research and Evaluation Steering Committee

From:

Robert A. Kominski /Signed/
Acting Division Chief
Social, Economic, and Housing Statistics Division

Prepared by:

Hyon B. Shin
Education and Social Stratification Branch
Social, Economic, and Housing Statistics Division

Subject:

2010 ACS Content Test Evaluation Report Covering Computer
and Internet

Attached is the final report for the 2010 ACS Content Test for the computer and Internet
questions. This report describes the results of the new computer and Internet content that will
go into the 2013 ACS questionnaire.
If you have any questions about this report, please contact Hyon B. Shin (301) 763-6169 or Kurt
J. Bauman (301) 763-6171.
Attachment: (2010 ACS Content Test Evaluation Report Covering Computer and Internet)
cc:
ACS Research and Evaluation Team
ACS Steering Committee
Kurt Bauman
(SEHSD)
Todd Hughes
(ACSO)
Debbie Klein
(ACSO)
David Raglin
(ACSO)
Donna Daily
(ACSO)

Anthony Tersine
Jennifer Tancreto
John Chesnut
Amy Lauger
Yuling Pan

(DSSD)
(DSSD)
(DSSD)
(DSSD)
(SRD)

American Community Survey Research and Evaluation Program
January 31, 2012

2010 ACS Content Test
Evaluation Report Covering
Computer and Internet

Hyon B. Shin
Social, Economic, and
Housing Statistics Division

Intentionally Blank

TABLE OF CONTENTS
ACKNOWLEGEMENTS ............................................................................................................. iv
LIST OF TABLES ..........................................................................................................................v
EXECUTIVE SUMMARY .......................................................................................................... vii
1. BACKGROUND ........................................................................................................................1
1.1 Motivation for the 2010 Content Test ..................................................................................1
1.2 Previous Testing or Analysis ...............................................................................................2
1.3 Recommendations from Cognitive Testing .........................................................................3
1.4 Recommendations from the Expert Review Panel ..............................................................3
2. SELECTION CRITERIA ...........................................................................................................4
3. METHODOLOGY .....................................................................................................................5
3.1 Data Collection Methods .....................................................................................................5
3.2 Sample Design .....................................................................................................................6
3.3 Methodology Specific to Computer and Internet Use .........................................................6
4. LIMITATIONS ...........................................................................................................................9
5. RESEARCH QUESTIONS AND RESULTS ..........................................................................10
5.1 Are the estimates for computer ownership and Internet subscription comparable to
or higher than existing data sources? .......................................................................................10
5.2 Which question version has lower item missing data rates?..............................................11
5.3 Do the question versions have similar or different response distributions? ......................12
5.4 Which question version results in more reliable estimates? ..............................................12
5.5 For the write-in responses to “Some Other Type of Computer” or “Some Other
Service,” is there a frequent response that indicates a need to edit the instructions
and/or the examples given for a response category? Does accounting for the
information gathered from the write-ins change any of the estimates? ...................................13
5.6 Which question version results in more respondents who inconsistently answer
Internet Subscription and Internet Subscription Type? ...........................................................14
5.7 For urban areas and for rural areas, do the two question versions have differential
item missing data rates, response distributions, or reliability of the data? ..............................14
5.8 For each mode of data collection, do the two question versions have differential
item missing data rates, response distributions, or reliability of the data? ..............................15
5.9 For each mail response stratum, do the two question versions have differential
item missing rates, response distributions, or reliability of the data? ......................................16
5.10 Does either question version elicit respondent or interviewer behaviors that may
contribute to interviewer or respondent error?.........................................................................16
6. SUMMARY ..............................................................................................................................17

ii

REFERENCES ..............................................................................................................................18
Appendix A: Tables .................................................................................................................... A-1
Appendix B: Images of the Mail Versions of the Control and Test Questions ...........................B-1
Appendix C: CATI and CAPI Versions of the Control and Test Questions ...............................C-1
Appendix D: Flow of the Content Follow-Up Interview ............................................................ D-1
Appendix E: Information Page .................................................................................................... E-1

iii

ACKNOWLEDGEMENTS
This report was written with the assistance of many statisticians and reviewers from
different divisions within the U.S. Census Bureau. Their review of this report was
invaluable. The statistical analysis for this report was provided by Amy Lauger and John
Chesnut, Decennial Statistical Studies Division. The reviewers of this report were
Jennifer Tancreto and Anthony Tersine in the Decennial Statistical Studies Division;
Todd Hughes in the American Community Survey Office; and Kurt Bauman and Robert
Kominski in the Social, Economic, and Housing Statistics Division.

iv

LIST OF TABLES
Table 1. Response Distributions of the Computer and Internet Categories....................................... A-1
Table 2. Item Missing Data Rates of the Computer and Internet Categories .................................... A-2
Table 3a. Response Distributions for the Computer and Internet Categories ................................... A-3
Table 3b. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories .......................................................................................................................................... A-3
Table 4a. Gross Difference Rates for the Computer and Internet Categories ................................... A-4
Table 4b. Index of Inconsistency for the Computer and Internet Categories .................................... A-5
Table 5a. Computer Types Write-Ins ................................................................................................ A-6
Table 5b. Subscription Type Write-Ins ............................................................................................. A-7
Table 5c. Estimate Differences Between Edited and Unedited Write-Ins in Appropriate
Categories ......................................................................................................................................... A-8
Table 6. Inconsistency, Omission, and Commission Rates for Internet Access and
Subscription Types ............................................................................................................................ A-8
Table 7a. Item Missing Data Rates of the Computer and Internet Categories for Urban Areas ....... A-9
Table 7b. Item Missing Data Rates of the Computer and Internet Categories for Rural Areas ...... A-10
Table 7c. Response Distributions of the Computer and Internet Categories for Urban Areas ........ A-11
Table 7d. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for Urban Areas ............................................................................................................. A-11
Table 7e. Response Distributions of the Computer and Internet Categories for Rural Areas ......... A-12
Table 7f. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for Rural Areas .............................................................................................................. A-12
Table 7g. Gross Difference Rates for the Computer and Internet Categories for Urban Areas ...... A-13
Table 7h. Gross Difference Rates for the Computer and Internet Categories for Rural Areas ....... A-13
Table 7i. Index of Inconsistency for the Computer and Internet Categories for Urban Areas ........ A-14
Table 7j. Index of Inconsistency for the Computer and Internet Categories for Rural Areas ......... A-15
Table 8a. Item Missing Data Rates of the Computer and Internet Categories for MAIL Mode ..... A-16
Table 8b. Item Missing Data Rates of the Computer and Internet Categories for CATI Mode ...... A-17
Table 8c. Item Missing Data Rates of the Computer and Internet Categories for CAPI Mode ...... A-17
Table 8d. Response Distributions of the Computer and Internet Categories for MAIL Mode ....... A-18
Table 8e. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for MAIL Mode ............................................................................................................. A-18
Table 8f. Response Distributions of the Computer and Internet Categories for CATI Mode ......... A-19
Table 8g. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for CATI Mode .............................................................................................................. A-19
Table 8h. Response Distributions of the Computer and Internet Categories for CAPI Mode......... A-20
Table 8i. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for CAPI Mode .............................................................................................................. A-20
Table 8j. Gross Difference Rates for the Computer and Internet Categories for MAIL Mode ....... A-21
Table 8k. Gross Difference Rates for the Computer and Internet Categories for CATI Mode ....... A-21
Table 8l. Gross Difference Rates for the Computer and Internet Categories for CAPI Mode ........ A-22
Table 8m. Index of Inconsistency for the Computer and Internet Categories for MAIL Mode ...... A-22
Table 8n. Index of Inconsistency for the Computer and Internet Categories for CATI Mode ........ A-23
Table 8o. Index of Inconsistency for the Computer and Internet Categories for CAPI Mode ........ A-24
Table 9a. Item Missing Data Rates of the Computer and Internet Categories for HIGH
Stratum............................................................................................................................................. A-25

v

Table 9b. Item Missing Data Rates of the Computer and Internet Categories for LOW
Stratum............................................................................................................................................. A-25
Table 9c. Response Distributions of the Computer and Internet Categories for HIGH Stratum..... A-26
Table 9d. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for HIGH Stratum .......................................................................................................... A-26
Table 9e. Response Distributions of the Computer and Internet Categories for LOW Stratum ..... A-27
Table 9f. Rao-Scott Chi-Square of the Multiple Answer Computer and Internet Use
Categories for LOW Stratum ........................................................................................................... A-27
Table 9g. Gross Difference Rates for the Computer and Internet Categories for HIGH
Stratum............................................................................................................................................. A-28
Table 9h. Gross Difference Rates for the Computer and Internet Categories for LOW Stratum .... A-28
Table 9i. Index of Inconsistency for the Computer and Internet Categories for HIGH Stratum ..... A-29
Table 9j. Index of Inconsistency for the Computer and Internet Categories for LOW Stratum...... A-30

vi

EXECUTIVE SUMMARY

Test Objective
In late August through mid-December 2010, the Census Bureau conducted a field test of
new and revised content in the 2010 American Community Survey (ACS) Content Test.
The results of that testing will determine the content to be incorporated into production
ACS starting in 2013.
Computer ownership and Internet subscription questions need to be added into the
American Community Survey due to legislative changes. The Broadband Data
Improvement Act requires that the Secretary of Commerce, in consultation with the
Federal Communications Commission, expand the ACS to elicit information from
residential households to determine whether they own or use computers at their address,
whether they subscribe to an Internet service and, if so, whether they subscribe to dial-up
or broadband Internet service at that address.
The goal of this project is to evaluate the two versions of computer ownership and
Internet subscription questions in the 2010 ACS Content Test to help determine which
one will ultimately go into ACS production starting in 2013.

Methodology
A set of three proposed new questions on computer ownership, Internet subscription, and
Internet subscription type were part of the 2010 ACS Content Test. One question asked
about computer ownership or usage. A second question asked whether the household
accessed or subscribed to the Internet. A third question asked to what type of Internet
service the household subscribed. The two versions of these three questions were tested
using a split-ballot design with the following differences between them:



Question order: Internet Access and Internet Subscription Types came first in the
control version, while Computer Types was first in the test version.
Internet question construct: The control version of Internet Access only measured
subscription, while the test version measured access and subscription. 1

The four original research questions were joined by two supplemental questions that were
originally not considered a selection criterion. These two questions were added after the
results from the content test were analyzed.

1

For an explanation of the terminology used to describe the different questions and answers, see Page 2.

vii

Research Questions and Results
1. Are the estimates for computer ownership and Internet subscription comparable
to or higher than existing data sources?
The estimates from the ACS Content Test were compared to the data collected in the
2010 October Current Population Survey (CPS).2 The data collected in the CPS are not
considered a “gold standard” and therefore the comparison should be interpreted with
care. The estimates for both the control and test versions were comparable to the CPS.
Estimates of mobile subscription and use of handheld devices were lower in the CPS but
the differences are most likely due to survey differences between the CPS and the ACS.
Further information is provided in the results section of this report.
2. Which question version has lower item missing data rates?
Overall, the test version performed better for Computer Types and Subscription Types.
The test version had lower item missing rates for two of the three Computer Types as
well as all the Subscription Types. There was no significant difference in the item
missing data rates between the test and the control versions for the overall estimates3 of
Computer Types or Subscription Types. For the question on Internet access, the control
version had a lower item missing data rates and therefore performed better.
3. Do the question versions have similar or different response distributions?
The response distributions showed the test version elicited higher estimates of the
Computer Types compared to the control version. The only statistically different
estimates for Subscription Types were for cable Internet, which had higher estimates in
the control version, and for mobile broadband plan, which had lower estimates in the
control version. Without a comparable data source, however, the differences should be
noted but do not yield conclusions as to which version performed better in the content
test.
4. Which question version results in more reliable estimates?
The test version had a significantly lower gross difference rate (GDR) than the control for
one Computer Type category, desktops, while there was no evidence of a difference for
the other categories. There were no significant differences for the GDRs for Subscription
Types. For Internet Access, the test version had a significantly higher GDR than the
control version.

2

The 2010 October Current Population Survey questions are shown in Appendix C.
Since the Computer Types and Subscription Types allowed for more than one answer to be checked, the
overall estimate for these two questions refer to the calculation of any of the answer categories being
checked.
3

viii

For Computer Types, the index of inconsistency (IOI) was significantly higher for
desktops in the control. For Internet Access, those who responded “Yes-have access”
had a significant difference where the test had almost double the IOI.
The GDR and IOI show that the test is more reliable for those reporting desktops but less
reliable for those who said they had Internet Access.
5. For the write-in responses to “Some Other Type of Computer” or “Some Other
Service,” is there a frequent response that indicates a need to edit the instruction
and/or the examples given for a response category? Does accounting for the
information gathered from the write-ins change any of the estimates?
The write-in frequency distributions for Computer and Internet Types show that the
majority of respondents supplied a type of computer that was already a choice in the
question. A notable exception, however, was tablet computers. Considering the
prevalence with which tablet computers were written in the write-in box, we will include
instructions to inform respondents to consider tablet computers an “other” type of
computer.
One category that is not considered a computer yet is the fourth most commonly writtenin device is gaming systems. Current gaming systems, while able to connect to the
Internet, are not considered a type of computer by the sponsoring agencies, FCC and
NTIA, and by the Census Bureau. We understand that this is a fast-changing sector and
we will be cognizant of perhaps changing this if future gaming systems advance enough
to be considered a computer. Currently, however, we will instruct respondents and
CATI/CAPI interviewers to not consider gaming systems a type of computer.
Where appropriate, the edited data were coded responses back into existing categories.
This process resulted in higher estimates for almost all of the categories. The differences
were not large but did increase the estimates.
6. Which question version results in more reliable estimates?
The test version had a higher CFU inconsistency rate, mail inconsistency rate, and a
higher commission rate with respect to the Internet Access and Subscription Types
questions. For the Content Follow-up (CFU), the test version had a higher inconsistency
rate compared to the control. For the mail, the omission rate was not statistically
different but the commission rate was more than double that of the control version.
While we don’t know whether the answers in the content test or the answers in the CFU
are the correct answers, we see from these results that the test version of Internet Access
elicits poorer results. Combining these results with the results from the other research
questions, we concluded that the control version performed better.

ix

Summary
Overall, the test version of the computer and Internet use questions performed better than
the control version. The test version results had lower item nonresponse rates, were more
reliable, and were answered consistently between the initial interview and the follow-up
for Computer Types and Subscription Types. The test version, however, did not perform
as well as the control version for the estimates on Internet Access. The results indicated
that the test version of this question was met with some confusion which led to higher
item missing data rates, higher rates of inconsistency, and lower reliability estimates.
Since the test version performed better on two out of the three possible questions and it
captures not only access but subscription to an Internet service, the recommendation is to
proceed with the test version of the computer and Internet use questions in the 2013
American Community Survey.

x

1. BACKGROUND
1.1 Motivation for the 2010 ACS Content Test
To evaluate proposed changes to the content of the American Community Survey (ACS),
the Census Bureau conducted the 2010 ACS Content Test. The objective of the ACS
Content Test, for both new and existing questions, was to determine the impact of
changing question wording, response categories, and redefinition of underlying
constructs on the quality of data collected.
Through the Office of Management and Budget (OMB) Interagency Committee on the
ACS, subject matter experts from the Census Bureau and key data users from other
federal agencies collaborated in identifying revised and new questions for inclusion in the
content test. The suggested new and revised questions affected both the housing and
detailed person sections of the ACS questionnaire.
In the housing section, the food stamps question was altered to reflect a name change for
the food stamps program. In addition, a series of new questions were added related to
household computer ownership and Internet subscription.
Several changes were made in the detailed person section. First, a change in data needs
for the veteran series led to a revised set of response categories for the veteran status and
period of military service questions. Second, the question wording of the cash public
assistance income question was modified to address under-reporting of assistance on
behalf of children and single payment recipients. Third, to simplify the income questions
related to wages (wages, salary, commissions, bonuses, or tips) and property income
(interest, dividends, rental income, royalty income or income from estates and trust),
these questions were broken up into smaller questions for the Computer-Assisted
Telephone Interviewing (CATI) and Computer-Assisted Personal Interviewing (CAPI)
instruments only. Fourth, a set of new questions on parental place of birth were added to
to allow data users to divide the population into “first generation” (the foreign born),
“second generation” (the children of immigrants), and “third or higher generation”
(native born with no foreign-born parents).
To meet the test objective of the 2010 ACS Content Test, analysts evaluated changes to
question wording, response categories, instructions, and examples relative to a control
version of the question or another version for new questions. Specifically, this report
discusses computer ownership and Internet subscription.
Computer ownership and Internet subscription questions need to be implemented into the
American Community Survey due to legislative changes. The Broadband Data
Improvement Act (SR.1492) requires that the Secretary of Commerce expand the ACS to
elicit information from residential households to determine whether households own or
use computers at their address, whether persons subscribe to an Internet service and, if so,
whether they subscribe to dial-up or broadband Internet service at that address.
1

Two versions of the computer ownership and Internet subscription question series were
tested in the 2010 ACS Content Test. The ordering of the question was altered between
versions to explore a potential context effect related to Internet subscription and computer
ownership. In combination with the ordering effect, the question on Internet
subscribership was broadened to ask about Internet access in conjunction with
subscribership to distinguish the concepts of Internet access versus subscription.
Since there are three separate questions asked about computer and Internet use4, for ease
of reading the results throughout the remainder of this report, the following is our
shorthand terminology for each of the three questions:
Computer Types = the questions on both the control and test that asks about what types
of computing devices are owned or used by any member of the household.
Internet Access = the question on the control that asks whether the household subscribes
to the Internet and the question on the test that asks if the household accesses the Internet
with or without a subscription.
Subscription Types = the questions on both the control and test that ask what type of
Internet subscription the household is using.
Additionally, the answer categories for the three questions were riddled with long
descriptors. For ease of reading this report, the following shorthand is used:
Desktop = desktop, laptop, netbook, or notebook computer
Handheld = handheld computer, smart mobile phone, or other handheld wireless
computer
Other = some other type of computer
Dial-up = dial-up service
DSL = DSL service
Cable = cable modem service
Fiber optic = fiber optic service
Mobile = mobile broadband plan for a computer or a cell phone
Satellite = satellite service
1.2 Previous Testing or Analysis
The computer ownership and Internet subscription questions were tested in the ACS for
the first time. Prior to the ACS, the Census Bureau asked a similar set of questions about
computers and Internet on the Current Population Survey (CPS). The questions asked on
the ACS Content Test were derived from various questions asked in the CPS but were not
taken directly from previously tested questions.

4

See Appendix B to read the three questions on computers and Internet usage.

2

1.3 Recommendations from Cognitive Testing
Prior to conducting the content test, the Research Triangle Institute (RTI), Westat, and
Research Support Services (RSS) conducted cognitive interviewing, under contract, to
assist in identifying a final set of questions for the field test. Three versions of each
question topic were tested with the goal of choosing the best one for the revised questions
and the best two for the new questions. 5 The questions were pretested in the three modes
used in the ACS data collection (paper, telephone interview, and personal interview) in
English and Spanish. Cognitive interviews consisted of one-on-one interviews using the
proposed questions in the context of the ACS survey. Survey methodologists also
conducted respondent debriefings.
All three cognitively tested versions asked about both computer ownership and Internet
subscription (Hinsdale, M., et. al., 2009). Overall, the terminology for computer types
and Internet subscription were universally understood. The cognitive testing report stated
that the intent of each of the questions was easily comprehended.
Versions 1 and 2 asked about the specific computer devices whereas Version 3 only
asked if a computer was owned or used in the household. Most respondents knew the
difference among the different computers listed.
All three versions asked about the specific Internet services. The types of services, while
easily understood, were not easily answered by some respondents. This was due to the
respondents’ lack of knowledge about the type of service the household used, mostly
because a spouse or a housemate was the responsible party, not because of confusion on
the types of service. In most cases, another household member took care of the Internet
service. This was true across the versions. The one item that was confusing to almost all
respondents was the exclusion of Wi-Fi (wireless fidelity) from the answer category
about wireless Internet service. Even those who knew what Wi-Fi was did not
understand why it was being excluded.
The cognitive testing report recommended going with either Version 2 or Version 3
without any significant wording changes, however, further recommendations were made
during the Expert Review (see Section 1.4)
1.4 Recommendations from the Expert Review Panel
Following the cognitive testing, an expert review panel, composed of government survey
methodology experts, reviewed and added changes to the final question versions
proposed to move forward from the cognitive testing into the field test. The proposed
changes for each question topic were approved by the corresponding OMB interagency
subcomittee responsible for initiating the research. The OMB provided final approval of
the proposed changes.

5

See Appendix E, page E-3 to read the three versions of the computer and Internet use questions that were
cognitively tested.

3

The expert review panel suggested several changes to the findings from the cognitive
testing report. Appendix E shows the three versions that were cognitively tested. Of
these choices, the expert review panel’s first suggestion was to go with the three-part
questions that asked about Computer Types, Internet Access, and Subscription Types.
The other change was to use the forced-choice answer categories for Computer Types
and Subscription Types. Finally, “At this home, apartment, or mobile home…” was
added to each question to keep it consistent with the other questions on the current ACS
form.
After consultation with the expert panel and the sponsoring agencies, it was determined
that a question order effect would also be tested in the field test as well as keeping the
instructions on what devices should be excluded for Computer Types.
Test Version 1 was chosen to be the basis for the three questions that went into the
content test. The same two questions were asked for Computer Types and Subscription
Types while two different questions were developed for Internet Access. In the control
version, Internet Access was asked first with a simple YES/NO answer category,
followed by the Subscription Types and Computer Types questions. In the test version,
Computer Types was asked first, followed by the modified Internet Access question
which asked about access with or without a subscription, and finally the Subscription
Types question. The final questions can be read in Appendix B.

2. SELECTION CRITERIA
Originally ten research questions were proposed with four of the questions impacting the
selection criteria questions and six supplemental research questions. The four main
research questions evaluated which version had comparable estimates compared to
existing data sources, which version had lower item missing data rates, whether the test
and control versions had similar or different response distributions, and which question
version resulted in more reliable estimates. An extensive explanation on how these
research questions were evaluated is discussed in section 3.3 in the Methodology Section.
After the field test, two supplemental questions were moved to be considered part of the
selection criteria. The research questions that evaluated the responses to the write-in
answers to Computer Types and Subscription Types and that assessed which question
version resulted in more respondents who inconsistently answered Internet Access and
Subscription Types were moved to the selection criteria.
These six questions were used to determine which version of the computer ownership and
Internet subscription questions would go forward in the 2013 American Community
Survey.

4

3. METHODOLOGY

3.1 Data Collection Methods
The initial stages of the Content Test consisted of content determination, cognitive
laboratory pretesting, and expert reviews for the purpose of developing alternate versions
of question content. The field test portion of the ACS Content Test used the data
collection methodology currently used in the production ACS (i.e., mail questionnaire,
follow-up CATI, and follow-up CAPI) with an added reinterview conducted via a CATI
instrument known as the Content Follow-Up (CFU). Additional data were collected on
respondent and interviewer behavior during the field test via Computer Audio Recorded
Interviewing (CARI) technologies for a subset of respondents during the CATI and CAPI
follow-up modes of data collection.
The Content Test followed the same schedule and procedures for the mail, CATI, and
CAPI operations as the September 2010 ACS production panel. Questionnaires were
mailed to sampled households at the end of August 2010. The Content Test used an
English-only mail form but the automated instruments (CATI, CAPI, and CFU) included
both English and Spanish versions. Households not responding by mail and for which we
had a phone number were contacted for a CATI interview during the month of October
2010. In November 2010, Census Bureau field representatives visited a sample of
households that did not respond by mail or CATI to attempt a CAPI interview. The CAPI
operations ended December 2, 2010.
The field test included a CATI CFU reinterview to collect additional measures for the
study of response error. This operation started approximately two weeks after the initial
mail out of questionnaires and ended two weeks after the end of the CAPI follow-up data
collection operation. The CFU included all occupied households for which we received a
response in the original interview and had a telephone number. A response was defined
as a case where the household provided data through at least the first person’s place of
birth question for mail cases or at least a sufficient partial interview for CATI/CAPI
interviews. The reinterview was conducted about 2 to 4 weeks after the original
interview and with the original respondent when possible. Note that the CFU CATI
interview was an abbreviated version of the original Content Test interview. The CFU
instrument included the basic demographic section and only those questions preceding
the questions being tested in the housing and the detailed person sections to provide
context (see Appendix D for the flow of the CFU instrument).
The ACS Content Test did not include all of the production data collection operations and
processes. First, while the Telephone Questionnaire Assistance program’s toll-free
number was available to Content Test respondents for assistance, the CATI instrument
did not include content changes from the Content Test. Therefore data collected from
Content Test respondents via TQA CATI interview were not included in our analysis.
Second, since our objective was to study response error using unedited data, the Content

5

Test excluded the Failed Edit Follow-up (FEFU) CATI operation and the edit and
imputation data processes.
3.2 Sample Design
The 2010 Content Test consisted of a national sample of 70,000 residential addresses in
the contiguous United States (the sample universe did not include Puerto Rico, Alaska,
and Hawaii). The sample design for the Content Test was largely based on the ACS
production sample design with some modifications to meet the test objectives. The
modifications included adding an additional level of stratification by stratifying addresses
into high and low mail response areas, over-sampling addresses from the low mail
response areas to ensure equal response from both strata, and sampling units as pairs.
The high and low mail response strata were defined based on ACS mail response rates at
the tract-level. The paired sample selection formed pairs by first systematically sampling
an address within the defined sampling strata and then pairing that address with the
address listed next in the geographically sorted list. However, the pair was not likely
comprised of neighboring addresses. One member of the pair was randomly assigned to
the control group and the other member was assigned to the test group. Those addresses
assigned to the test group received the revised ACS questions and the questions new to
the ACS. The control group received the current questions on the production ACS as
well as different versions of the new questions.
Another modification to the production ACS sample design included adding a third
sampling stage. At the first stage, the production 2010 ACS first stage sample was used
as the Content Test first stage sample. At the second stage, all housing units in the ACS
first stage sample not selected in the production 2010 ACS second-stage sample were
selected as the Content Test second-stage sample. In addition, any units that were
selected to be in other operations (e.g., training, other tests, etc.) were not selected in the
Content Test second stage sample. At the third stage, addresses were selected using a
sampling method similar to the production ACS second stage sample design with the
exception of adding the high and low mail response stratification.
3.3 Methodology Specific to Computer and Internet Use
A set of three proposed new questions on Computer Types, Internet Access, and
Subscription Types were part of the 2010 ACS Content Test. Given our split-ballot
experimental design, we tested two versions of the computer and Internet questions with
the following differences between them:



Question order: Internet Subscription and Internet Subscription Type came first in
control version, while Computer Ownership Type was first in the test version.
Internet question construct: The control version of Internet Subscription only
measured subscription, while the test version measured access and subscription.

The questions as they appear in the mail form are shown in Appendix B and the
CATI/CAPI question versions are shown in Appendix C.
6

Several methodologies were used to test which version had response distributions more
comparable to existing data sources, lower item nonresponse rates, more reliable
estimates, and which question version resulted in fewer respondents who inconsistently
answered the Internet Access and Subscription Types questions.
The first analysis compared the response distributions of the computer and Internet use
questions to the 2010 October Supplement to the Current Population Survey (CPS).
Formal statistical comparisons could not be made since the content test data were not
edited or imputed, adjusted for nonresponse, nor raked to known population totals,
however, we checked estimates for the computer and Internet use questions for
comparability.
To analyze which version had lower item nonresponse rates, three different types of rates
were calculated: item missing data rates, invalid data rates, and specification error rates.
The missing data rate is the proportion of households which fail to provide any type of
response, or which respond with “Don’t Know” or “Refused.” For Subscription Types,
the universe included all households which were offered the question, or (in the mail
mode) provided a response even if they were supposed to skip it.
The invalid data rate is the proportion of households which respond with both “Yes” and
“No.” As above, for Subscription Types, the universe included all households which were
offered the question, or in the mail mode, provided a response even though they were
supposed to skip it. For the test version’s Internet Access, the invalid data rate is the
proportion of households which respond to “No access to the Internet at this house,
apartment, or mobile home” AND to at least one of “Yes, with a subscription to an
Internet service” or “Yes, without a subscription to an Internet service.”
Specification errors occur when a respondent interprets and responds to a question
differently from the intent of the questionnaire designers. We considered one type of
specification error a type of nonresponse in this analysis. For the test version’s Internet
Access, we calculated the proportion of mail response households which responded to
both “Yes, with a subscription” and “Yes, without a subscription” but not “No access to
the Internet at this house, apartment, or mobile home.”6 It is possible for a household to
have Internet access both with and without a subscription. The question was not intended
to be mark-all-that-apply, however, so such responses would be indicative of
specification error.
We calculated rates two ways for the forced-choice questions. First, for the multiple
questions option, we calculated the rates separately for each response category as if it
were an individual question. A particular item was considered missing if “Yes” and
“No” were both unchecked, or in the case of “Some Other Type of Computer” and “Some
Other Service,” if “Yes” and “No” were both unchecked and there were no codable writein responses. A particular category was considered invalid if “Yes” and “No” were both
6

This scenario is not possible in CATI/CAPI since the instrument only allowed for one response.

7

checked, or in the case of “Some Other Type of Computer” and “Some Other Service,” if
“Yes” and “No” were both checked and there was no codable write-in.
Second, for the single question option, we calculated the rates as if Computer Types and
Subscription Types were each one question with several response options. In this
instance, a response was considered missing if “Yes” and “No” were both unchecked for
all of the categories and there was no codable write-in response. A response was
considered invalid if “Yes” and “No” were both checked for all of the categories and
there was no codable write-in response.
For the CATI/CAPI questions, an altered skip pattern was used in the Content Follow-up
(CFU). In the field test, proper skip patterns were implemented so if a household
indicated they did not have Internet access, the questions regarding which types of
Internet service the household might have wasn’t asked. In the CFU, however, the
altered skip pattern asked Subscription Types anyway. Using data derived from the
content test and the altered skip pattern in the CFU, the number of respondents who either
responded “No” for Internet Access but answered “Yes” to any one of the Subscription
Types, or “Yes” for Internet Access but “No” for all Subscription Types were compared.
For the mail questions, calculations were made for when a respondent answered a
question they were not meant to answer and when a respondent failed to answer a
question they were supposed to answer. Using data from the mail respondents, the
number of respondents who commit these two errors are calculated. Only households
with a valid response in both the content test and the CFU for an individual category were
included in this analysis.
Using data from the content test and CFU, we compared the gross difference rates,
indices of inconsistency, and the L-fold indices of inconsistency between the two
versions to check the reliability of the data.
The gross difference rate (GDR) is the percentage of inconsistent answers between the
original interview and the CFU. The GDR halved is an estimator for the simple response
variance. The GDR was calculated using a two-sided simultaneous pairwise comparison
between the control and the test versions.
The index of inconsistency (IOI) is the percentage of total variance due to simple
response variance and provides an estimate of the magnitude of response variability. An
IOI was computed for each response category and an overall index of inconsistency,
called the L-fold index of inconsistency, was reported for the entire question, where
applicable. The L-fold IOI is a weighted average of the individual indices computed for
each response category.
The final analysis to determine which version resulted in fewer respondents who
inconsistently answered the Internet Access and Subscription Types questions was to use
the data derived from the content test and the altered skip pattern in the CFU to calculate
the number of respondents who either report “No” for Internet Access and “Yes” for one
8

or more of the Subscription Types, or “Yes” for Internet Access and “No” for all
Subscription Types between the two versions. In mail cases, the error or omission and
error of commission were compared between the control and test versions.
4. LIMITATIONS
Control and test CATI-CAPI workload assignments were not assigned using an
interpenetrated experimental design. That is, interviewers were allowed to administer
interviews for both control and test cases, in addition to production ACS cases. The
potential risk of this approach was the introduction of a cross-contamination or carry-over
effect due to the interviewer administering multiple versions of the same question item.
Interviewers were trained to read the questions verbatim to minimize this risk, but there
still exists the possibility that an interviewer may have deviated from the scripted
wording of one question version to another. This could potentially mask a treatment
effect from the data collected.
The CFU reinterview was not conducted in the same mode of data collection for
households that responded by mail or CAPI in the original interview since CFU
interviews were only administered using a CATI mode of data collection. As a result, the
data quality measures derived from the reinterview may include some bias due to the
differences in mode of data collection.
Respondents needed to provide a telephone number in the original Content Test interview
or the Census Bureau had to be able to find a telephone number for that unit through
reverse address look-up to be included in the CFU interview. As a result, 18.4 percent of
the responding households from the original interview were not eligible for the CFU
reinterview.
We did not have the same respondent in the CFU that we had in the original interview for
9.1 percent of the CFU cases. This means that differences between the original interview
and the CFU for these cases could be due in part to having different people answering the
questions.
The Content Test did not include the production weighting adjustments for seasonal
variations in ACS response patterns, nonresponse bias, and under-coverage bias. The
CFU portion of the Content Test did include a unit nonresponse adjustment for those
Content Test cases that responded to the Content Test, but failed to respond to the CFU.
As a result, the statistics derived from the Content Test data do not provide the same level
of inference as the production ACS to the entire population of housing units and persons
in the contiguous United States.
Specific limitations to the computer and Internet use questions include not having exact
questions to compare the results, even though the 2010 CPS was a good data source to
use. Without exact question wording, implementation, and data collection, the data will
have comparability limitations. Another limitation was testing both an order effect
without using identical questions. Using two different questions for Internet Access most
9

likely affected the results beyond evaluating the order of the questions. Since the
computer and Internet use questions were asked at the household level, if the respondent
was not the one who was responsible for acquiring or maintaining their Internet
subscription, some were not knowledgeable enough to answer the Subscription Types
question accurately.
5. RESEARCH QUESTIONS AND RESULTS
Table A shows the unit response rates for each of the modes of data collection and all
modes combined (excluding CFU) by the control and test groups. The comparison
between control and test show that respondent participation was similar for both control
and test for each of the modes of data collection and all modes combined, with the
exception of the CATI mode. The test treatment produces a CATI rate of response that is
3 percentage points higher compared to that of the control. We are not able to explain the
increase in response due to the test treatment for the CATI mode of data collection other
than by random occurrence given that the conditions affecting unit response were
equivalent between the test and control groups.

Table A. Content Test Response Rate Comparisons Between the Control and Test Treatments
Standard
Standard
Test Standard
Test
Error
Control
Error
Control
Error
Mode
(%)
(%)
(%)
(%)
(%)
(%)
Significant
All Modes
(CFU
95.4
0.2
95.7
0.2
-0.3
0.3
No
excluded)
Mail
58.1
0.5
57.7
0.5
0.5
0.7
No
CATI
52.6
1.2
49.6
1.0
3.0
1.5
Yes
CAPI
90.4
0.5
91.5
0.5
-1.1
0.7
No
CFU
54.3
0.5
53.5
0.6
0.8
0.7
No
Source: U.S. Census Bureau, 2010 American Community Survey Content Test

5.1

Are the estimates for computer ownership and Internet subscription
comparable to or higher than existing data sources?

Table 1 (see Appendix A) shows the response distributions of the computer and Internet
categories for the test version, the control version, and the estimates from the
2010 October Current Population Survey (CPS). The 2010 October CPS, while not
considered a “gold standard”, was in the field at the same time as the content test which
makes it ideal to compare the results. Overall, the content test results were comparable to
the data from the CPS with the exception of those who reported having a mobile
subscription type and the use of handheld devices. The content test estimates were
considerably higher than the CPS, regardless of treatment. The test and the control
versions showed 40 percent and 34 percent, respectively, of respondents reporting they
owned a handheld device while the CPS showed 21 percent reporting they owned a
handheld. The difference in mobile broadband plans for a computer or a cell phone also
10

showed a big difference where the test estimated 23 percent and the control estimated 21
percent while the CPS estimated only 9 percent.
This disparity could be explained by the difference in how the questions are asked
between the ACS and the CPS. In the CPS instrument, after asking the questions on
computer ownership and Internet access, the answer categories are all read aloud and the
respondent is allowed to mark-all-that-apply. In the ACS CATI/CAPI instrument,
however, each individual answer category is read aloud and the respondent answers
either yes or no for each one before being asked the next answer category. This forcedchoice option forces allows the respondent to consider each category separately whereas
the CPS’s approach relies on the respondent to hear all the options and relay back to the
interviewer all that apply. Ericson and Nelson (2007) reported that questions that use a
forced-choice format elicit more responses than a mark-all-that-apply, as witnessed in
their test of health insurance questions in the 2006 ACS Content Test.
Analysis of the data show that for the ACS test and control treatments, multiple devices
were reported more in the ACS than in the CPS. By summing up the distributions of
Computer Types and Subscription Types for each survey/treatment, the ACS test version
had 121 percent reporting Computer Type and 127 percent Subscription Type. The ACS
control version had 112 percent and 132 percent. In the CPS, however, multiple devices
were reported 101 percent and multiple Subscription Types were reported 105 percent.
The lower reporting of multiple devices in the CPS is mostly due to the lower responses
in handheld devices and mobile Internet access.
Another explanation for the ACS/CPS differences in handheld devices and mobile
broadband is those devices/connection types could be secondary to their
laptop/computers and their home Internet connection type. Even though we asked about
all types they may own or use, reporting the secondary devices may not occur to some
respondents as quickly in the mark-all-that-apply approach.
5.2

Which question version has lower item missing data rates?

Table 2 highlights the different item missing data rates between the test and control
treatments. Overall, there was no significant difference between treatments for the
questions overall or for those who own or use a desktop computer. For all the other
answer categories under Computer Types and for each of the Subscription Types, the
control version had significantly higher missing data rates.
For Internet Access, the test version had higher item missing data rates. The test
version’s two “Yes” categories (“Yes with a subscription” and “Yes without a
subscription”) was not as easily understood or answered compared to the simpler
“Yes”/“No” answer category in the control version. This is illustrated by many of the
results throughout this section and by what we learned during cognitive testing.
So, while the test version had better results for Computer Types and Subscription Types,
it had poorer results for Internet Access.
11

5.3

Do the question versions have similar or different response distributions?

The response distributions shown in Table 3a show that there are statistical differences
between the test and the control versions for Computer Types and Subscription Types.
Without a comparable data source, however, the differences should be noted but do not
yield conclusions as to which version performed better in the Content Test.
Comparing the distribution who responded “Yes” to the control Internet Access question
and “Yes, access with a subscription” answer category in the test Internet Access
question, no significant difference is shown between these two categories (72 percent
versus 71 percent).
When reviewing the number of computers, the test version had more Computer Types
reported than the control version. The shift seems to occur between having just one
computer category to having more than one. Perhaps reading and hearing the Computer
Types questions first in the test veresion, respondents notice the handheld devices answer
category more readily compared to the control version which asks about Internet Access
and Subscription Types first. They may be more apt to report multiple devices.
For number of Subscription Types, the control version had a lower proportion of
households which had zero Subscription Types while the test had a lower proportion for
those with one Subscription Type. In the control version, the Subscription Types
question is asked before the Computer Types question, therefore, respondents may not be
thinking about their handheld devices which are more likely to be associated with the
“mobile broadband plan…” answer category for Subscription types. In the test version,
however, we ask about Computer Types first which could then help respondents recall
that their handheld devices (e.g. smart phones) use a mobile broadband plan. Hence the
test version could elicit reporting of more than one Subscription Type.
The additional analysis for broadband types show the control had lower proportions of
zero and two broadband types, while the test had a lower proportion with one broadband
type.7 Since the broadband types estimates follow a similar pattern as the Subscription
Types estimates, the same theory about having multiple types of Internet service would
apply here.
5.4

Which question version results in more reliable estimates?

Using data from the content test and the Content Follow-up (CFU), comparisons of the
gross difference rates (GDR), indices of inconsistency (IOI), and the L-fold indices of
inconsistency were compared between the two versions.
The GDR is the percentage of inconsistent answers between the original interview and
the CFU. Table 4a shows that the test version had a significantly lower GDR than the
control for those who provided a response for the desktop category in both the content
test and the CFU, while there was no evidence of a difference for the other categories.
7

Broadband Types include the following Subscription Types: DSL, cable, fiber-optic, mobile, and satellite.

12

There was no significant difference in the GDR for any of the Subscription Types.
Comparing the test and control versions for Internet Access with a subscription, the test
version had a significantly higher GDR than the control. It could be respondents
responded inconsistently when the concept of “access” was introduced during the CFU.
Table 4b shows that for desktop, the control version had higher IOI. For Internet Access,
those who responded “Yes - have access with a subscription” had a significant difference
where the test had almost double the IOI. For Subscription Types, the only significant
difference was in the aggregate L-fold indices where the test was lower than the control
version.
For most response categories for Internet Access, both treatments had low indices of
inconsistency. The index was high for the test’s “Yes, without a subscription” and the test
question’s Aggregate L-Fold was moderate.8 While the high and moderate
inconsistencies should be noted, the reader should not be alarmed by it. Singer and Ennis
(2003) conclude that answer categories with rarer populations, for instance those who
responded to the “other” categories for Computer Types and Subscription Types, the
index of inconsistency could be higher for these categories when only a few respondents
change their response from the initial interview to the re-interview.
Once again, the test version of Internet Access did not perform as well as the control
version. The concept of having Internet Access with or without a subscription resulted in
high inconsistency between the content test and the CFU. While the desktop category
performed better in the control version, the rest of the results were not conclusive as to
which one performed better.
5.5

For the write-in responses to “Some Other Type of Computer” or “Some
Other Service,” is there a frequent response that indicates a need to edit the
instructions and/or the examples given for a response category? Does
accounting for the information gathered from the write-ins change any of the
estimates?

The write-in frequency distributions for Computer and Internet Types show that the
majority of respondents supplied a type of computer that was already a choice in the
question (see Table 5a and Table 5b). A notable exception, however, was tablet
computers. Considering the prevalence with which tablet computers were written in the
write-in box, we will include instructions to inform respondents to consider tablet
computers an “other” type of computer.
One category that is not considered a computer yet is the fourth most commonly writtenin device is gaming systems. Current gaming systems, while able to connect to the
Internet, are not considered a type of computer by the sponsoring agencies, FCC and
NTIA, and by the Census Bureau. We understand that this is a fast-changing sector and
we will be cognizant of perhaps changing this if future gaming systems advance enough

8

The Census Bureau’s general benchmarks for reliability state that index values of less than 20 percent
indicate Low inconsistency, 20-50 percent indicate Moderate inconsistency, and over 50 percent indicate
High inconsistency.

13

to be considered a computer. Currently, however, we will instruct respondents and
CATI/CAPI interviewers to not consider gaming systems a type of computer.
Table 5c shows that editing the data based on the coded responses, while not large
differences, did result in higher estimates for almost all of the categories.
5.6

Which question version results in more respondents who inconsistently
answer Internet Subscription and Internet Subscription Type?

The test version had a higher CFU inconsistency rate, mail inconsistency rate, and a
higher commission rate with respect to the Internet Access and Subscription Types
questions (see Table 6).9 The test showed the mail inconsistency rate being 4 percent
versus 2 percent in the control. For the Content Follow-up (CFU), the test version had a
higher inconsistency rate (8 percent) compared to the control (5 percent). For the mail,
the omission was not statistically different but the commission rate was 12 percent in the
test and 5 percent in the control.
While we don’t know whether the answers in the content test or the answers in the CFU
are the true answers, we can see from these results that the test version of Internet Access
elicits poorer results. Combining these results with the results from the other research
questions, we concluded that the control version performed better.
5.7

For urban areas and for rural areas, do the two question versions have
differential item nonresponse rates, response distributions, or reliability of
the data?

The differential item missing data rates, response distributions, and reliability of data
information varied for urban and for rural areas. For item missing data rates, the same
items had similar levels of significance in both urban and rural areas (see Tables 7a and
7b). While the overall missing data rates for Computer Types and Subscription Types
were not significantly different between the control and the test, the individual items had
higher item missing data rates in the control version in both urban and rural areas. For
Internet Access, however, both the urban and rural areas had higher item missing data
rates in the test version.
In urban areas, the test version had more respondents reporting owning each computer
type as well as higher instances of mobile broadband connection. The control version,
however, had a higher estimate of those subscribing to a cable Internet connection (Table
7c). The Rao-Scott Chi-square tests of dependence indicate that the percent distribution
of multiple answer categories was dependent on question version in urban areas (Table
7d).

9

Inconsistency rate is the percentage of mail responders who answered Internet Access and Subscription
Types inconsistently. The commission rate is the percentage of mail respondents who responded "No"
(Control version) or "Yes, without a subscription" or "No" (Test version) to Internet Access but responded
to Subscription Types.

14

In rural areas, the only significant difference in the estimates of type of computer
ownership, Internet Access with or without a subscription, and type of Internet
subscription between the test and the control versions was for the percentage who
reported owning a handheld device (Table 7e). As with urban areas, the rural areas also
had significant Chi-square test statistic indicating that the percent distribution of multiple
answer categories was independent of question version (Table 7f).
The gross difference rates and index of inconsistency in urban and rural rates are shown
in Tables 7g, 7h, 7i, and 7j. The results are similar to those found at the national level.
The test version of Internet Access performed poorly compared to the control version but
performed better for the Computer Types that were statistically different. Most of the
results for Subscription Types were not significantly different and therefore cannot be
used to determine which version performed better.
5.8

For each mode of data collection, do the two question versions have
differential item nonresponse rates, response distributions, or reliability of
the data?

Tables 8a, 8b, and 8c show the item missing data rates by the three modes of collection,
Mail, CATI, and CAPI. For the Mail mode, the control version showed higher item
missing data rates than the test version for the overall Computer Types, handhelds, and
the other types of computers as well as for the Internet Access item. In the CATI and
CAPI modes, only those with Internet Access showed a significant difference with the
test version having higher item missing data rates than the control version.
Response distribution differences for the three modes are shown in tables 8d, 8f, and 8h.
Some sporadic differences were seen but without any discernible pattern. Tables 8e, 8g,
and 8i show the Chi-square statistics for the three modes.
In the Mail mode, the control had a higher gross difference rate (GDR) and a higher
index of inconsistency rate (IOI). The test version, however, had a higher GDR and
higher IOI for handhelds and a higher IOI for the other category than the control (Tables
8j and 8m). The test version had higher GDR and higher IOI for those with Internet
Access.
Similar patterns are discernible in the CATI and CAPI modes where the control had
higher GDR and IOI for desktop but higher GDR and IOI for the Internet Access with a
subscription item.
The only Subscription Type that showed significant differences for the reliability
measures was for satellite subscription. In the Mail mode, the control version had a
higher GDR (13 percent versus 10 percent) whereas in the CAPI mode, the test version
had a higher IOI (70 percent versus 42 percent).
The Mail mode, having the largest sample size compared to CATI and CAPI, had many
results that were significantly different between the test version and the control version,
15

however, all three modes followed the same patterns as the results from the national
level. The test version had favorable results with Computer Types and Subscription
Types but performed less well for the Internet Access items, compared to the control
version.
5.9

For each mail response stratum, do the two question versions have
differential item nonresponse rates, response distributions, or reliability of
the data?

For both the high and low strata, the item missing data rates are presented with the
control version having higher item missing data rates for handhelds and other devices and
for all Subscription Types (Tables 9a and 9b). The test version, however, had higher item
missing data rates for Internet Access for both strata.
As with the results for the item missing data rates and the Chi-square statistics, the
response distributions were similar to the national level for the high and low strata
(Tables 9c, 9d, 9e and 9f). Similarly, Tables 9g, 9h, 9i, and 9j showed similar gross
difference rates and index of inconsistency rates. As with the results from the other
research questions, both strata had estimates similar to that of the national level estimates.
The test version of Internet Access had poorer results than the control version whereas
the results with significant differences showed Computer Types and Subscription Types
having favorable estimates compared to the control version.
5.10

Does either question version elicit respondent or interviewer behaviors that
may contribute to interviewer or respondent error?

Data were collected on respondent and interviewer behavior during the field test for a
subset of the CATI and the CAPI modes of data collection. The behavior of the
respondents and interviewers were collected via Computer Audio Recorded Interviewing
(CARI) technologies. The Center for Survey Management (CSM) analyzed the behavior
coding for all the new items in the content test. The Behavior Coding Report (Pascale,
et.al, 2011) describes the methodology used by CSM to evaluate the content test
recordings and the results.
CSM evaluated the computer and Internet questions for all three questions: Computer
Types, Internet Access, and Subscription Types. The Behavior Coding Report’s findings
show that for Computer Types and Subscription Types, no significant difference between
the test and the control for interviewer and respondent behaviors to the question. This is
not surprising since the two questions were identical in both versions.
For Internet Access, however, the report shows that the test version was the most
problematic in how interviewers and respondents read, understood, and answered this
question. As we suspected with the higher item missing data rates, higher errors of
omission and commission, lower reliability, and higher inconsistency rates for the test
version of Internet Access, the confusion of Internet access with a subscription and

16

Internet access without a subscription appears to be substantiated by the behavior of both
interviewers and respondents.
6. SUMMARY
The 2010 ACS Content Test tested questions on computer ownership and Internet use,
which have never been asked on the ACS before. The inclusion of computer and Internet
questions were dictated by the introduction of Senate Resolution S.1492. Broadband Data
Improvement Act.
Four main research questions and six supplemental questions were used to gauge the
strength of one version of the computer and Internet questions over the other version.
Several analyses were used to evaluate each research question. We compared the content
test data to existing data sources, examined which version had lower item nonresponse
rates, had more reliable estimates, and which question version resulted in more
respondents who consistently answered the Internet Access and Subscription Types
questions.
We also evaluated the data written in the write-in fields on Computer Types and
Subscription Types. The fast-pacing change of technological devices and Internet-based
applications will hopefully be captured by what respondents supply in the write-in fields.
For instance, tablet computers existed when the questions were developed but in the short
two and a half years since the beginning of this content test cycle, the use of tablet
computers has far outpaced what we anticipated. These technological breakthroughs will
continue to add to our knowledge and understanding of what devices and services the
U.S. population is using.
The complexity of having three separate questions, with multiple answers possible for
Computer Types and Subscription Types, as well as the different wording for Internet
Access between the control and the test versions, created a multifaceted analysis of the
results. No one version clearly outperformed the other. The test version results had
lower item nonresponse rates and improved reliability in terms of respondents answering
consistently between the initial interview and the follow-up. The exception to the test
version performing better was with Internet Access. The results indicated that the test
version of this question was met with some confusion which led to higher item missing
data rates, higher rates of inconsistency, and lower reliability estimates.
The test version performed better on two out of the three possible questions. Despite the
results in the test version of Internet Access, the better outcomes of Computer Types and
Subscription Types, as well as the added benefit of having the word “access” which can
be used to compare ACS results to other data sources, the recommendation is to proceed
with the test version of the computer and Internet use questions in the 2013 American
Community Survey.

17

REFERENCES
Ericson, L. and Nelson, C. (2007), “A Comparison of Forced-Choice and Mark-All-ThatApply Formats for Gathering Information on Health Insurance in the 2006 American
Community Survey Content Test.” Presented at the 2007 Federal Committee on
Statistical Methodology Conference, U.S. Census Bureau, Washington D.C.
Hinsdale, M., McFarlane, E., Weger, S., Schloua-Glusberg, A., and Kerwin, J. (2009),
“Cognitive Testing of the American Community Survey Content Test Items”, RTI
International, Research Triangle Park, NC.
Pascale, J., Goerman, P., and Drom, K. (2011) “ACS 2010 Content Test Behavior
Coding Report.” U.S. Census Bureau, Washington D.C.
Singer, P. and Ennis, S. (2003), “Census 2000 Content Reinterview Survey: Accuracy of
Data for Selected Population and Housing Characteristics as Measured by Reinterview”,
Census 2000 Evaluation B.5, U.S. Census Bureau, Washington D.C.
Senate Resolution 1492 [110th]: Broadband Data Improvement Act
http://www.govtrack.us/congress/billtext.xpd?bill=s110-1492.

18

Appendix A: Tables

Table 1.  Response Distributions of the Computer and Internet Categories (Use for RQ 1)
ACS
CPS
Test
Standard  Control
Standard  Estimate  Standard 
(%)
Error (%)
(%)
Error (%)
(%)
Error (%)
1

Computer Types
Desktop
Handheld
Other
Internet Access
Yes

77.3
39.7
3.9

0.4
0.5
0.2

75.9
33.8
2.5

0.4
0.4
0.2

79.6
21.2
0.7

0.2
0.2
0.1

71.0

0.5

71.7

0.5

71.9

0.3

6.6
33.7
47.8
7.6
23.1
6.3
2.3

0.3
0.5
0.5
0.3
0.4
0.3
0.2

6.9
34.9
52.3
7.3
20.5
7.0
2.6

0.3
0.5
0.6
0.3
0.5
0.3
0.2

3.9
35.2
47.4
5.2
9.1
2.7
1.4

0.1
0.2
0.2
0.1
0.1
0.1
0.0

2,3

4

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t a nd the 2010 October Current 
Popul a ti on Survey.
1 For thi s  fa mi l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error ra te ha s  been control l ed us i ng the 
Bonferroni  mul ti pl e compa ri s on method a t the α = 0.10 l evel .
2 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to 
a n Internet s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who res ponded "Yes ."
3 For the compa ri s on of Internet Acces s , the CPS ques ti on us ed to crea te the percenta ge of hous ehol ds  wi th a  
s ubs cri pti on to a n Internet s ervi ce i s  HENET2b.  See the Computer a nd Internet Us e Bri efi ng Document to vi ew 
the CPS ques ti onna i re.
4 For thi s  fa mi l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error ra te ha s  been control l ed us i ng the 
Bonferroni ‐Hol m mul ti pl e compa ri s on method a t the α = 0.10 l evel .
Note: Si nce mul ti pl e a ns wers  coul d be chos en, the cumul a ti ve frequency does  not equa l  100%.
0.0 ‐ rounds  to zero

A-1

Table 2. Item Missing Data Rates of the Computer and Internet Categories (Use for RQ 2)
Test 
(%)
Computer Types
Overall 
Desktop
Handheld
Other

1

n=19,454
2.6
2.9
9.8
26.9

2

n=19,454 
4.2
0.1

Internet Access
Missing data rate
3

Invalid rate ‐ Access 

Specification ‐  Access
Subscription Types
Overall
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Standard 
Error (%)

5

4

0.2

Control
(%)

Standard 
Test ‐ 
Standard 
Significant
Error (%) Control (%) Error (%)

n=19,451
2.8
3.1
15.6
28.9

0.1
0.2
0.3
0.4

‐0.2
‐0.2
‐5.8
‐2.1

0.2
0.3
0.4
0.6

No
No
Yes
Yes

n=19,451
0.2
2.7
0
0

0.1
0

1.5
0.1

0.3
0

Yes
Yes

0.2
0.2
0.3
0.4

0

N/A

N/A

N/A

N/A

Yes

n=14,465
1.0
24.7
18.8
15.4
26.8
25.3
26.7

0.1
0.4
0.4
0.3
0.4
0.4
0.4

n=13,841
1.1
28.9
22.0
17.4
31.2
29.5
30.4

0.1
0.5
0.4
0.4
0.5
0.5
0.5

‐0.1
‐4.2
‐3.1
‐2.0
‐4.3
‐4.2
‐3.7

0.1
0.6
0.5
0.5
0.6
0.6
0.6

No
Yes
Yes
Yes
Yes
Yes
Yes

31.6

0.5

34.1

0.5

‐2.5

0.7

Yes

Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.
1 For thi s  fami l y of two‐s i ded hypothes i s  tes ts , the fami l y‐wi s e error rate has  been control l ed us i ng the Bonferroni  mul ti pl e 
compa ri s on method at the α = 0.10 l evel .
2 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who res ponded "Yes ."
3 Inva l i d ra te ‐ the proporti on of hous ehol ds  whi ch res ponded wi th both "Yes " and "No." 
4 Speci fi ca ti on error rate ‐ the proporti on of mai l  res pons e hous ehol ds  whi ch res ponded to both "Yes , wi th a  s ubs cri pti on" 
and "Yes , wi thout a  s ubs cri pti on" but not "Not acces s  to the Internet at thi s  hous e, a pa rtment, or mobi l e home" i n the tes t 
vers i on. Thi s  s cenari o i s  not pos s i bl e i n the CATI/CAPI s i nce the i ns trument onl y a l l ows  one res pons e.  The s i gni fi ca nt 
di fference of the s peci fi ca ti on error rates  refer to the tes t es ti ma tes  that di ffer from zero. Stati s ti ca l  s i gni fi ca nce of 
di fferences  i s  determi ned a t the α = 0.10 s i gni fi ca nce l evel  us i ng a  one‐s i ded tes t.
5 For thi s  fami l y of two‐s i ded hypothes i s  tes ts , the fami l y‐wi s e error rate has  been control l ed us i ng the Bonferroni ‐Hol m 
mul ti pl e compari s on method a t the α = 0.10 l evel .
0.0 ‐ rounds  to zero

A-2

Table 3a. Response Distributions for Computer and Internet Categories (Use for RQ 3)

Computer Types
Desktop
Handheld
Other

Test
Standard 
(%)
Error (%)
n= 18,907
77.3
0.4
39.7
0.5
3.9
0.2

1

Internet Access
Access with a subscription
Access without a subscription
No Internet access

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
n=18,802
75.9
0.4
1.4
0.5
Yes
0.4
5.9
0.6
Yes
33.8
2.5
0.2
1.4
0.3
Yes

n=18,507 
71.0
4.2
24.8

0.5
0.2
0.5

n=18,813
71.7
N/A
N/A

0.5
N/A
N/A

‐0.7
N/A
N/A

0.7
N/A
N/A

No
N/A
N/A

n=14,310 
6.6
33.7
47.8
7.6
23.1
6.3
2.3

0.3
0.5
0.5
0.3
0.4
0.3
0.2

n=13,678
6.9
34.9
52.3
7.3
20.5
7.0
2.6

0.3
0.5
0.6
0.3
0.5
0.3
0.2

‐0.2
‐1.2
‐4.5
0.3
2.5
‐0.7
‐0.2

0.3
0.7
0.7
0.4
0.7
0.4
0.2

No
No
Yes
No
Yes
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

n= 18,907
20.2
41.2
36.1
2.5

0.4
0.4
0.4
0.2

n=18,802
21.5
46.1
31.0
1.4

0.3
0.4
0.4
0.1

‐1.4
‐4.8
5.1
1.1

0.5
0.6
0.6
0.3

Yes
Yes
Yes
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

n=14,310 
7.7
64.7
21.7
5.9

0.3
0.5
0.5
0.3

n=13,678
4.0
69.0
20.5
6.6

0.2
0.5
0.5
0.3

3.7
‐4.3
1.2
‐0.7

0.3
0.8
0.7
0.4

Yes
Yes
No
No

n=13,870 
10.5
63.2
21.2
5.0

0.3
0.6
0.5
0.3

n=13,232
6.3
68.5
19.6
5.5

0.2
0.6
0.5
0.3

4.2
‐5.3
1.6
‐0.5

0.4
0.8
0.7
0.4

Yes
Yes
Yes
No

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

2

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

3

Source: U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.
1 For thi s  fa mi l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error ra te ha s  been control l ed us i ng the Bonferroni  mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .
2 For thi s  fa mi l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error ra te ha s  been control l ed us i ng the Bonferroni ‐Hol m mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .
3 The number of broa dba nd types  i ncl ude the fol l owi ng Subs cri pti on Types : DSL, ca bl e, fi ber‐opti c, mobi l e, a nd s a tel l i te.

Table 3b. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories (Use for RQ 3)
Chi‐Sq 
Statistic Significant
Number of computer types
99.5
Yes
Number of subscription types
119.3
Yes
Number of broadband types
101.1
Yes
Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.

A-3

1

Table 4a. Gross Difference Rates  for the Computer and Internet Categories (Use for RQ 4)
Test
(%)
Computer Types
Desktop
Handheld
Other

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

2

4.3
14.3
6.8

0.3
0.5
0.5

6.5
14.3
5.5

0.3
0.6
0.4

‐2.2
0.0
1.4

0.4
0.7
0.7

Yes
No
No

9.1
7.5

3

4.6 
N/A
N/A

0.3
N/A

4.5
N/A

0.5
N/A

Yes
N/A

5.3 

0.4
0.4
0.3

N/A

N/A

N/A

N/A

5.2
15.5
13.9
6.2
23.5
8.1
5.8

0.4
0.7
0.7
0.5
0.9
0.4
0.6

6.6
16.6
14.4
6.8
22.9
8.8
6.1

0.5
0.8
0.7
0.5
0.9
0.5
0.5

‐1.3
‐1.1
‐0.4
‐0.6
0.6
‐0.7
‐0.3

0.6
1.1
1.0
0.8
1.2
0.7
0.7

No
No
No
No
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

4

5

Source:  U.S. Cens us  Bureau, 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew and the Content Fol l ow‐
Up i ntervi ew.
2 For thi s  fami l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error rate ha s  been control l ed us i ng the Bonferroni  mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .
3 In the di chotomous  cas e, the GDRs  for the two res pons e ca tegori es  are equi va l ent.  Therefore, the control  vers i on's  GDR va l ue for 
the "No" res pons e category i s  equi va l ent to tha t of the "Yes " res pons e ca tegory (“Acces s  wi th a s ubs cri pti on” i n the tabl e.)
4 The GDR for the tes t vers i on's  combi ned ca tegori es  "Acces s  wi th a  s ubs cri pti on" and "Acces s  wi thout a s ubs cri pti on" i s  equi va l ent 
to the GDR for "No Internet acces s ."  
5 For thi s  fami l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error rate ha s  been control l ed us i ng the Bonferroni ‐Hol m mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .
0.0 ‐ rounds  to zero

A-4

1

Table 4b. Index of Inconsistency  for the Computer and Internet Categories (Use for RQ 4)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

2

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

12.6
29.8
81.4
33.8

0.8
1.1
3.7
0.9

18.6
30.5
82.8
34.6

0.9
1.4
4.0
0.8

‐6.0
‐0.7
‐1.5
‐0.8

1.2
1.5
5.4
1.2

Yes
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

21.8
76.5
14.3
24.8

0.9
3.2
0.8
0.8

11.6
N/A
N/A
N/A

0.7
N/A
N/A
N/A

10.2
N/A
N/A
N/A

1.1
N/A
N/A
N/A

Yes
N/A
N/A
N/A

36.6
31.9
28.2
34.0
55.3
68.5
86.0
58.1

2.3
1.4
1.4
2.5
2.0
2.8
3.5
0.8

43.3
33.6
30.2
34.8
54.9
59.1
83.2
61.0

2.5
1.6
1.4
2.7
2.1
3.2
4.0
0.8

‐6.6
‐1.7
‐2.0
‐0.8
0.4
9.4
2.8
‐2.9

3.6
2.2
2.1
4.1
2.8
4.6
5.3
1.3

No
No
No
No
No
No
No
Yes

3

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

Source:  U.S. Cens us  Bureau, 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce and provi des  an es ti ma te 
of the magni tude of res pons e vari a bi l i ty. The Cens us  Bureau's  general  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 
20 percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi cate hi gh 
i ncons i s tency.
2 For thi s  fami l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error rate ha s  been control l ed us i ng the Bonferroni  mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .
3 For thi s  fami l y of two‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error rate ha s  been control l ed us i ng the Bonferroni ‐Hol m mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .

A-5

Table 5a. Computer Types Write‐Ins  (Use for RQ 5)

Write‐in category
DESKTOP
LAPTOP
NOCOMP
GAMECONSOLE
IPAD
PC
IPOD
CELLPHONE
DELL
EREADER
ITOUCH
BLACKBERRY
IPHONE
SMARTPHONE
SERVER
HP
MAC
PDA
TABLET
HARDWARE
WEBTV
BRANDNAME
GPS
HANDHELD
NOTEBOOK
SOFTWARE
NETBOOK
WORK
HOMECOMPUTER
CALCULATOR
TYPEWRITER
IBM
LIBRARY
MP3
APPLIANCE
BLIND
GOOGLETV
INTERNET
PHONE
TABLETOP
TOSHIBA
WATCH

Frequency
208
155
145
129
86
42
30
27
27
19
19
16
16
16
14
13
13
12
11
10
10
9
9
9
8
7
6
6
5
4
4
3
3
3
2
2
1
1
1
1
1
1

Cumulative Cumulative
Percent
Frequency
Percent
18.8
208
18.8
14.0
363
32.9
13.1
508
46.0
11.7
637
57.7
7.8
723
65.5
3.8
765
69.3
2.7
795
72.0
2.5
822
74.5
2.5
849
76.9
1.7
868
78.6
1.7
887
80.3
1.5
903
81.8
1.5
919
83.2
1.5
935
84.7
1.3
949
86.0
1.2
962
87.1
1.2
975
88.3
1.1
987
89.4
1.0
998
90.4
0.9
1,008
91.3
0.9
1,018
92.2
0.8
1,027
93.0
0.8
1,036
93.8
0.8
1,045
94.7
0.7
1,053
95.4
0.6
1,060
96.0
0.5
1,066
96.6
0.5
1,072
97.1
0.5
1,077
97.6
0.4
1,081
97.9
0.4
1,085
98.3
0.3
1,088
98.6
0.3
1,091
98.8
0.3
1,094
99.1
0.2
1,096
99.3
0.2
1,098
99.5
0.1
1,099
99.6
0.1
1,100
99.6
0.1
1,101
99.7
0.1
1,102
99.8
0.1
1,103
99.9
0.1
1,104
100.0

Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.
Note: Frequenci es  a re unwei ghted.

A-6

Table 5b. Subscription Type Write‐Ins  (Use for RQ 5)

Coded Write‐In
CABLE
WIRELESS
NOINTERNET
MOBILE
UNCODABLE
ATT
FIBER
WIFI
SATELLITE
PHONE
VERIZON
DSL
DIALUP
OTHER
RADIO
BROADBAND
ANTENNA
COLLEGE
TOWER
HIGHSPEED
LINEOFSIGHT
T1
LIBRARY
TV
WEBTV
APT
LMDS
MOBILE

Frequency
163
98
87
84
50
45
44
44
41
32
22
18
13
8
6
5
4
4
4
3
3
3
2
2
2
1
1
1

Cumulative Cumulative
Percent
Frequency
Percent
20.6
163
20.6
12.4
261
33.0
11.0
348
44.1
10.6
432
54.7
6.3
482
61.0
5.7
527
66.7
5.6
571
72.3
5.6
615
77.9
5.2
656
83.0
4.1
688
87.1
2.8
710
89.9
2.3
728
92.2
1.7
741
93.8
1.0
749
94.8
0.8
755
95.6
0.6
760
96.2
0.5
764
96.7
0.5
768
97.2
0.5
772
97.7
0.4
775
98.1
0.4
778
98.5
0.4
781
98.9
0.3
783
99.1
0.3
785
99.4
0.3
787
99.6
0.1
788
99.8
0.1
789
99.9
0.1
790
100.0

Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.
Note: Frequenci es  a re unwei ghted.

A-7

Table 5c. Estimate Differences Between Edited and Unedited Write‐Ins in Appropriate Categories (Use for RQ 5)
Control Version
Computer Types
Desktop
Handheld
Other

Edited 
(%)

Unedited  Standard 
(%)
Error (%)

Edited ‐ 
Unedited  Standard 
(%)
Error (%) Significant

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

76.2
33.9
0.8

0.3
0.4
0.1

75.9

0.4

0.3

0.0

33.8

0.4

0.1

0.0

2.5

0.2

-1.8

0.1

6.9
35.0
52.6
7.4
21.0
7.0
0.9

0.1
0.5
0.6
0.3
0.5
0.3
0.1

6.9
34.9
52.3
7.3
20.5
7.0
2.6

0.3
0.5
0.6
0.3
0.5
0.3
0.2

0.1

0.0

0.1

0.1

0.2

0.0

0.1

0.0

0.5

0.1

Yes
Yes
Yes

2

Test Version
Computer Types
Desktop
Handheld
Other

Standard 
Error (%)

Edited 
(%)

Standard 
Error (%)

Unedited  Standard 
(%)
Error (%)

0.1

0.0

-1.7

0.1

Yes
Yes
Yes
Yes
Yes
Yes
Yes

Edited ‐ 
Unedited  Standard 
(%)
Error (%) Significant

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

77.6
39.8
1.3

0.4
0.5
0.1

77.3
39.7
3.9

0.4
0.5
0.2

0.3
0.1
‐2.7

0.0
0.0
0.2

Yes
Yes
Yes

6.6
33.7
48.1
7.7
23.6
6.3
0.8

0.2

6.6
33.7
47.8
7.6
23.1
6.3
2.3

0.3
0.5
0.5
0.3
0.4
0.3
0.2

0.0
0.0
0.3
0.1
0.5
0.1
‐1.5

0.0
0.0
0.1
0.0
0.1
0.0
0.1

No
No
Yes
Yes
Yes
Yes
Yes

2

0.5
0.5
0.3
0.4
0.3
0.1

Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.
1 For thi s  fa mi l y of one‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error ra te ha s  been control l ed us i ng the Bonferroni  mul ti pl e 
compa ri s on method a t the α = 0.10 l evel .
2 For thi s  fa mi l y of one‐s i ded hypothes i s  tes ts , the fa mi l y‐wi s e error ra te ha s  been control l ed us i ng the Bonferroni ‐Hol m 
mul ti pl e compa ri s on method a t the α = 0.10 l evel .
0.0 ‐ round to zero

Table 6. Inconsistency, Omission, and Commission Rates for Internet Access and Subscription Types (Use for RQ 6)
Test
Standard  Control
Standard 
Test ‐ 
Standard 
(%)
Error (%)
(%)
Error (%) Control (%) Error (%) Significant
Mail ‐ Inconsistency Rate
CFU ‐ Inconsistency Rate
Mail ‐ Omission Rate

2

3

Mail ‐ Commission Rate

1

4

4.1

0.2

2.2

0.2

1.9

0.3

Yes

8.1
1.2

0.4
0.1

4.8
1.2

0.3
0.1

3.3
0.0

0.5
0.2

Yes
No

11.6

0.4

5.2

0.2

6.4

0.4

Yes

Source:  U.S. Cens us  Burea u, 2010 Ameri ca n Communi ty Survey Content Tes t.
1 Ma i l  Incons i s tency Ra te ‐ The percenta ge of ma i l  res pondents  who a ns wered Internet Acces s  and Subs cri pti on Types  i ncons i s tentl y.
2 CFU Incons i s tency Ra te ‐ The percenta ge of CFU res pondents  who a ns wered Internet Acces s  a nd Subs cri pti on Types  i ncons i s tentl y. (Al l  CFU 
res pondents  recei ved the Subs cri pti on type ques ti on, rega rdl es s  of thei r a ns wer to Internet Acces s .)
3 Ma i l  Omi s s i on Ra te ‐ The percenta ge of ma i l  res pondents  who res ponded "Yes " (Control  vers i on) or "Yes , wi th a  s ubs cri pti on" (Tes t 
vers i on) to Internet Acces s  but di d not res pond to Subs cri pti on Types  
4 Ma i l  Commi s s i on Ra te ‐ The percenta ge of ma i l  res pondents  who res ponded "No" (Control  vers i on) or "Yes , wi thout a  s ubs cri pti on" or "No" 
(Tes t vers i on) to Internet Acces s  but res ponded to Subs cri pti on Types . 

A-8

Table 7a. Item Missing Data Rates of the Computer and Internet Categories for Urban Areas (Use for RQ 7)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Yes

2.5
2.8
9.6
27.2

Standard 
Error (%)
0.2
0.2
0.3
0.5

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
2.7
0.2
‐0.2
0.2
No
3.0
0.2
‐0.2
0.3
No
15.1
0.4
‐5.5
0.5
Yes
28.9
0.5
‐1.7
0.7
Yes

4.0

0.2

2.6

0.2

1.4

0.3

Yes

1.1
25.8
19.7
14.7
27.3
26.2
27.7
32.3

0.1
0.5
0.5
0.4
0.5
0.5
0.5
0.5

1.1
29.2
22.2
15.9
30.5
29.6
30.7
33.8

0.1
0.6
0.5
0.4
0.6
0.5
0.6
0.6

0.0
‐3.4
‐2.5
‐1.2
‐3.2
‐3.4
‐3.0
‐1.4

0.2
0.7
0.6
0.6
0.7
0.7
0.7
0.8

No
Yes
Yes
Yes
Yes
Yes
Yes
Yes

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."

A-9

Table 7b. Item Missing Data Rates of the Computer and Internet Categories for Rural Areas (Use for RQ 7)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Yes

3.0
3.4
10.5
25.6

Standard 
Error (%)
0.4
0.4
0.6
0.8

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
3.3
0.4
‐0.3
0.6
No
3.7
0.4
‐0.3
0.6
No
17.2
0.7
‐6.7
0.9
Yes
28.6
0.9
‐3.0
1.1
Yes

4.7

0.5

3.1

0.3

1.6

0.6

Yes

0.8
21.1
16.0
17.6
25.2
22.5
23.3
29.0

0.2
0.9
0.7
0.8
0.9
0.8
0.8
1.0

1.1
27.6
21.5
22.0
32.7
28.8
29.3
34.6

0.2
0.8
0.8
0.8
0.9
0.9
0.9
1.0

‐0.4
‐6.5
‐5.5
‐4.4
‐7.5
‐6.3
‐6.0
‐5.6

0.2
1.1
1.1
1.1
1.2
1.2
1.1
1.4

No
Yes
Yes
Yes
Yes
Yes
Yes
Yes

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."

A-10

Table 7c. Response Distributions for Computer and Internet Categories for Urban Areas (Use for RQ 7)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

78.0
41.4
4.1

0.4
0.6
0.3

75.9
35.3
2.4

0.5
0.5
0.2

2.2
6.1
1.7

0.6
0.6
0.4

Yes
Yes
Yes

Internet Access
Access with a subscription
Access without a subscription
No Internet access

71.9
4.3
23.8

0.5
0.3
0.5

72.2
N/A
N/A

0.5
N/A
N/A

‐0.3
N/A
N/A

0.7
N/A
N/A

No
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

5.4
32.5
52.0
8.6
23.5
4.7
2.3

0.2
0.6
0.6
0.3
0.5
0.3
0.2

5.7
33.4
56.8
8.1
20.5
4.9
2.3

0.3
0.6
0.6
0.3
0.6
0.3
0.2

‐0.4
‐0.9
‐4.8
0.5
2.9
‐0.2
0.0

0.3
0.8
0.8
0.4
0.8
0.4
0.3

No
No
Yes
No
Yes
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

19.5
40.1
37.7
2.6

0.4
0.5
0.5
0.2

21.5
44.8
32.4
1.3

0.4
0.5
0.5
0.1

‐2.0
‐4.6
5.3
1.3

0.5
0.7
0.6
0.3

Yes
Yes
Yes
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

7.5
64.1
22.2
6.2

0.3
0.6
0.5
0.3

4.1
68.4
21.0
6.6

0.2
0.6
0.6
0.3

3.4
‐4.2
1.1
‐0.3

0.3
0.9
0.7
0.5

Yes
Yes
No
No

9.4
63.5
21.8
5.3

0.3
0.6
0.5
0.3

5.8
68.2
20.5
5.5

0.3
0.7
0.5
0.3

3.6
‐4.7
1.3
‐0.2

0.4
0.9
0.7
0.5

Yes
Yes
No
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol lowi ng Subs cri pti on Types : DSL, ca bl e, fiber‐optic, mobil e, a nd s a tel l i te.

Table 7d. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for Urban Areas (Use for RQ 7)
Chi‐Sq 
Statistic Significant
Number of computer types
94.0
Yes
Number of subscription types
80.3
Yes
Number of broadband types
62.8
Yes
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-11

Table 7e. Response Distributions for Computer and Internet Categories for Rural Areas (Use for RQ 7)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

74.8
33.7
3.5

0.9
1.0
0.4

75.5
27.7
3.1

0.9
0.9
0.4

‐0.7
6.0
0.4

1.2
1.3
0.7

No
Yes
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access

67.6
3.9
28.4

0.9
0.5
0.9

69.4
N/A
N/A

1.0
N/A
N/A

‐1.8
N/A
N/A

1.3
N/A
N/A

No
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

10.8
38.1
33.9
4.2
21.2
11.4
2.6

0.7
1.2
1.2
0.5
1.0
0.8
0.4

10.8
39.4
36.8
4.7
20.0
13.3
3.4

0.6
1.1
0.9
0.5
0.9
0.8
0.5

0.1
‐1.3
‐3.0
‐0.4
1.2
‐1.9
‐0.8

1.0
1.6
1.5
0.7
1.5
1.2
0.5

No
No
No
No
No
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

22.5
44.9
30.7
1.9

0.8
1.0
1.0
0.3

22.3
50.6
25.6
1.5

0.9
1.0
0.9
0.3

0.3
‐5.7
5.0
0.5

1.1
1.4
1.2
0.4

No
Yes
Yes
No

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

8.4
66.7
20.0
4.8

0.5
1.0
1.0
0.5

3.8
71.8
18.1
6.3

0.4
1.1
0.9
0.5

4.6
‐5.1
2.0
‐1.5

0.6
1.5
1.4
0.7

Yes
Yes
No
No

14.6
62.1
19.2
4.0

0.7
1.2
1.1
0.5

8.4
70.1
16.3
5.2

0.5
1.1
0.9
0.6

6.2
‐7.9
3.0
‐1.2

0.9
1.7
1.6
0.7

Yes
Yes
No
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol lowi ng Subs cri pti on Types : DSL, ca bl e, fiber‐optic, mobil e, a nd s a tel l i te.

Table 7f. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for Rural Areas (Use for RQ 7)
Chi‐Sq 
Statistic Significant
Number of computer types
19.9
Yes
Number of subscription types
46.7
Yes
Number of broadband types
41.6
Yes
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-12

1

Table 7g. Gross Difference Rates  for the Computer and Internet Categories for Urban Areas (Use for RQ 7)
Test
(%)
Computer Types
Desktop
Handheld
Other

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

3.9
14.7
7.1

0.3
0.6
0.6

6.3
14.5
5.5

0.4
0.7
0.5

‐2.4
0.1
1.6

0.5
0.8
0.9

Yes
No
No

9.3
7.8
5.6

0.5
0.4
0.4

4.5
N/A
N/A

0.3
N/A
N/A

4.9
N/A
N/A

0.5
N/A
N/A

Yes
N/A
N/A

5.3
15.9
14.0
6.3
24.0
6.4
5.6

0.4
0.8
0.8
0.6
1.0
0.5
0.6

6.7
17.5
14.9
7.6
24.1
7.3
5.6

0.6
0.9
0.8
0.7
1.1
0.5
0.5

‐1.4
‐1.5
‐0.9
‐1.3
‐0.1
‐0.9
0.0

0.6
1.2
1.2
0.9
1.4
0.7
0.8

No
No
No
No
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Standard 
Error (%)

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

1

Table 7h. Gross Difference Rates  for the Computer and Internet Categories for Rural Areas (Use for RQ 7)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Access with a subscription
Access without a subscription
No Internet access
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

5.4
12.9
6.4

0.8
1.0
0.9

7.5
13.7
5.6

0.8
1.4
0.8

‐2.0
‐0.8
0.7

1.2
1.7
1.2

No
No
No

8.4
6.5
4.3

0.9
0.8
0.6

5.1
N/A
N/A

0.7
N/A
N/A

3.3
N/A
N/A

1.1
N/A
N/A

Yes
N/A
N/A

5.4
13.1
13.4
5.6
21.7
13.3
6.4

0.9
1.1
1.3
0.9
1.7
1.3
1.0

6.0
13.1
12.8
4.4
19.0
12.7
7.9

0.8
1.3
1.4
0.8
1.7
1.4
1.2

‐0.6
0.0
0.5
1.2
2.7
0.5
‐1.4

1.2
1.5
1.8
1.3
2.2
2.0
1.4

No
No
No
No
No
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

A-13

1

Table 7i. Index of Inconsistency  for the Computer and Internet Categories for Urban Areas (Use for RQ 7)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

11.9
30.1
78.3
33.9

0.9
1.2
4.3
0.9

18.1
30.5
84.5
34.9

1.0
1.5
4.6
0.9

‐6.2
‐0.4
‐6.2
‐1.0

1.3
1.6
6.1
1.2

Yes
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

22.6
76.6
15.5
26.0

1.0
3.6
1.0
1.1

11.4
N/A
N/A
N/A

0.8
N/A
N/A
N/A

11.2
N/A
N/A
N/A

1.2
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

48.1
33.0
29.0
30.4
55.3
72.2
92.2
58.5

3.4
1.6
1.7
2.6
2.2
4.9
3.3
1.1

51.8
35.7
33.4
34.7
57.8
77.8
82.7
63.1

3.5
1.8
1.8
3.1
2.3
4.0
5.5
1.0

‐3.7
‐2.7
‐4.4
‐4.3
‐2.4
‐5.7
9.5
‐4.6

5.1
2.5
2.7
4.3
3.3
6.0
6.4
1.6

No
No
No
No
No
No
No
Yes

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-14

1

Table 7j. Index of Inconsistency  for the Computer and Internet Categories for Rural Areas (Use for RQ 7)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

15.1
29.1
93.7
34.0

2.2
2.3
3.9
1.8

20.5
31.7
78.2
34.8

2.2
3.2
8.4
2.1

‐5.4
‐2.6
15.5
‐0.7

3.1
3.7
9.5
2.7

No
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

19.7
74.8
11.2
21.3

1.9
6.7
1.4
1.8

12.1
N/A
N/A
N/A

1.5
N/A
N/A
N/A

7.5
N/A
N/A
N/A

2.4
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

23.0
26.6
27.5
55.1
55.5
67.4
72.4
57.2

3.5
2.2
2.7
7.1
4.0
4.6
7.4
1.6

27.4
26.3
26.3
36.9
45.4
44.8
84.3
55.4

3.8
2.6
2.8
5.7
4.0
4.3
5.5
1.5

‐4.4
0.3
1.2
18.2
10.0
22.6
‐11.9
1.8

4.9
3.1
3.6
9.1
5.4
7.0
8.8
1.9

No
No
No
No
No
Yes
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-15

Table 8a. Item Missing Data Rates of the Computer and Internet Categories for MAIL Mode (Use for RQ 8)
Test
(%)
2.9
3.4
15.2
44.9

Standard 
Error (%)
0.2
0.2
0.4
0.6

4.9

0.2

3.4

0.2

1.5

0.3

Yes

2

0.1

0.0

0.0

0.0

0.1

0.0

Yes

3

0.2

0.0

N/A

N/A

N/A

N/A

Yes

25.7

13.3

N/A

N/A

N/A

N/A

Yes

1.4
37.9
28.0
22.9
39.7
37.7
40.8
48.6

0.1
0.6
0.6
0.5
0.5
0.6
0.6
0.7

1.4
45.9
33.8
26.6
47.4
46.1
48.3
54.3

0.1
0.6
0.6
0.5
0.6
0.6
0.6
0.6

‐0.1
‐8.0
‐5.7
‐3.7
‐7.7
‐8.3
‐7.5
‐5.7

0.2
0.8
0.7
0.7
0.7
0.8
0.8
0.9

No
Yes
Yes
Yes
Yes
Yes
Yes
Yes

Computer Types
Desktop
Handheld
Other

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
3.5
0.2
‐0.6
0.3
Yes
3.9
0.2
‐0.5
0.3
No
25.3
0.5
‐10.1
0.6
Yes
48.9
0.6
‐4.0
0.8
Yes

1

Internet Access
Yes
Invalid Rate ‐ Access 
(Mail Only)
Specification ‐ Access 
(Mail Only)
Error of Omission ‐ 
(spec error universe)
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

4

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."
2 Inva l i d ra te ‐ the proporti on of hous ehol ds  whi ch reponded wi th both "Yes " a nd "No" 
3 Speci fi ca ti on error rates  ‐ the proporti on of mai l  res pons e hous ehol ds  whi ch res ponded to both "Yes , wi th a  s ubs cri pti on" 
and "Yes , wi thout a  s ubs cri pti on" but not "Not acces s  to the Internet at thi s  hous e, a pa rtment, or mobi l e home." Thi s  
s cena ri o i s  not pos s i bl e i n the CATI/CAPI s i nce the i ns trument onl y a l l ows  one res pons e.  The s i gni fi ca nt di fference of the 
s peci fi ca ti on error ra tes  refer to the tes t es ti ma tes  that di ffer from zero.
4 Error of omi s s i on ra tes  ‐ of the hous ehol ds  whi ch res ponded to both "Yes " ques ti ons  but not the "No" ques ti on, the 
percenta ge of hous ehol ds  whi ch do not provi de a  va l i d "Yes " res pons e to at l ea s t one Internet Subs cri pti on type a nd do not 
provi de a  codabl e wri te‐i n.  The s i gni fi ca nt di fference of the error of omi s s i on rates  refer to the tes t es ti mates  tha t di ffer 
from zero.
0.0 ‐ rounds  to zero

A-16

Table 8b. Item Missing Data Rates of the Computer and Internet Categories for CATI Mode (Use for RQ 8)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Yes

0.2
0.6
0.8
0.6

Standard 
Error (%)
0.1
0.2
0.2
0.2

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
0.4
0.2
‐0.2
0.2
No
0.8
0.2
‐0.2
0.3
No
1.2
0.3
‐0.4
0.3
No
0.8
0.2
‐0.2
0.3
No

1.8

0.3

0.6

0.2

1.2

0.4

Yes

0.7
2.1
4.3
3.5
7.7
5.4
3.0
2.8

0.3
0.5
0.7
0.6
1.0
0.8
0.6
0.6

0.5
2.8
4.3
3.2
9.3
4.2
2.5
2.5

0.2
0.5
0.6
0.6
0.8
0.6
0.4
0.5

0.2
‐0.7
0.0
0.3
‐1.5
1.2
0.5
0.4

0.3
0.7
0.8
0.8
1.4
1.0
0.8
0.8

No
No
No
No
No
No
No
No

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."

Table 8c. Item Missing Data Rates of the Computer and Internet Categories for CAPI Mode (Use for RQ 8)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Yes

2.6
2.8
2.9
3.2

Standard 
Error (%)
0.4
0.4
0.4
0.4

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
2.3
0.3
0.4
0.5
No
2.4
0.3
0.4
0.5
No
3.1
0.3
‐0.1
0.5
No
2.9
0.3
0.3
0.5
No

3.6

0.5

2.1

0.3

1.5

0.6

Yes

0.3
0.6
1.9
1.4
2.9
2.3
1.0
0.8

0.1
0.2
0.3
0.3
0.4
0.5
0.2
0.2

0.6
1.3
2.8
2.5
4.2
2.5
1.7
1.7

0.2
0.3
0.5
0.4
0.5
0.4
0.3
0.4

‐0.3
‐0.6
‐0.9
‐1.1
‐1.3
‐0.2
‐0.7
‐0.9

0.2
0.3
0.5
0.5
0.7
0.6
0.4
0.4

No
No
No
No
No
No
No
No

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."

A-17

Table 8d. Response Distributions for Computer and Internet Categories for MAIL Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

79.8
38.7
2.4

0.4
0.5
0.2

79.4
32.3
1.3

0.4
0.5
0.1

0.4
6.4
1.1

0.6
0.7
0.2

No
Yes
Yes

Internet Access
Access with a subscription
Access without a subscription
No Internet access

75.9
2.6
21.5

0.5
0.2
0.5

76.0
N/A
N/A

0.4
N/A
N/A

0.0
N/A
N/A

0.7
N/A
N/A

No
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

6.9
30.5
44.8
7.0
18.3
6.8
1.7

0.3
0.5
0.6
0.3
0.4
0.3
0.1

6.4
32.2
49.5
7.2
16.2
7.8
2.4

0.3
0.6
0.6
0.3
0.4
0.3
0.2

0.4
‐1.7
‐4.8
‐0.1
2.1
‐1.0
‐0.6

0.4
0.8
0.8
0.4
0.6
0.4
0.2

No
Yes
Yes
No
Yes
Yes
Yes

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

18.5
43.6
36.5
1.4

0.4
0.5
0.5
0.1

19.2
49.2
31.0
0.6

0.4
0.5
0.5
0.1

‐0.7
‐5.7
5.6
0.8

0.5
0.7
0.7
0.1

No
Yes
Yes
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

11.2
67.3
17.1
4.3

0.4
0.6
0.5
0.2

5.3
74.5
15.2
5.1

0.2
0.5
0.4
0.2

5.9
‐7.2
2.0
‐0.7

0.4
0.7
0.6
0.3

Yes
Yes
Yes
Yes

13.6
65.8
16.7
3.9

0.4
0.6
0.4
0.2

6.8
74.0
14.8
4.4

0.3
0.5
0.4
0.3

6.8
‐8.2
1.9
‐0.5

0.4
0.6
0.6
0.3

Yes
Yes
Yes
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol l owi ng Subs cri pti on Types : DSL, ca bl e, fi ber‐opti c, mobi l e, a nd s a tel l i te.

Table 8e. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for MAIL Mode (Use for R
Chi‐Sq 
Statistic Significant
Number of computer types
111.5
Yes
Number of subscription types
237.6
Yes
Number of broadband types
256.9
Yes
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-18

Table 8f. Response Distributions for Computer and Internet Categories for CATI Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

75.5
34.2
4.1

1.2
1.2
0.5

76.4
30.4
2.9

0.9
1.1
0.5

‐0.8
3.8
1.1

1.7
1.7
0.6

No
Yes
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access

66.3
6.2
27.4

1.2
0.6
1.1

74.9
N/A
N/A

1.0
N/A
N/A

‐8.6
N/A
N/A

1.7
N/A
N/A

Yes
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

8.3
48.2
52.3
10.3
29.2
5.2
3.9

1.1
1.6
1.8
1.0
1.8
0.7
0.7

9.6
47.7
53.4
9.1
25.6
5.2
3.8

1.0
1.4
1.5
1.0
1.5
0.7
0.6

‐1.3
0.5
‐1.1
1.2
3.6
0.0
0.1

1.3
2.2
2.2
1.5
2.5
0.9
0.9

No
No
No
No
No
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

23.1
42.6
31.8
2.6

1.1
1.3
1.2
0.4

22.5
46.9
29.2
1.5

0.9
1.3
1.1
0.3

0.6
‐4.3
2.6
1.1

1.6
1.9
1.7
0.5

No
Yes
No
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

2.0
53.1
32.6
12.3

0.4
1.6
1.6
1.3

2.4
55.1
31.7
10.8

0.5
1.6
1.6
0.9

‐0.4
‐2.0
0.9
1.5

0.6
2.5
2.4
1.5

No
No
No
No

6.4
52.3
31.8
9.6

0.9
1.8
1.7
1.2

7.0
54.9
29.5
8.6

0.8
1.6
1.5
0.9

‐0.6
‐2.7
2.3
1.0

1.1
2.6
2.4
1.4

No
No
No
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol l owi ng Subs cri pti on Types : DSL, ca bl e, fi ber‐opti c, mobi l e, a nd s a tel l i te.

Table 8g. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for CATI Mode (Use for RQ
Chi‐Sq 
Statistic Significant
Number of computer types
9.0
Yes
Number of subscription types
1.7
No
Number of broadband types
2.0
No
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-19

Table 8h. Response Distributions for Computer and Internet Categories for CAPI Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

73.7
42.5
6.4

1.0
1.0
0.6

70.3
36.8
4.4

0.9
1.0
0.4

3.4
5.7
2.0

1.3
1.4
0.7

Yes
Yes
Yes

Internet Access
Access with a subscription
Access without a subscription
No Internet access

64.0
6.5
29.4

1.0
0.6
1.1

64.0
N/A
N/A

0.9
N/A
N/A

0.0
N/A
N/A

1.4
N/A
N/A

No
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

5.6
37.3
53.6
8.2
32.3
5.3
3.3

0.6
1.2
1.4
0.8
1.3
0.7
0.5

7.0
36.9
57.7
7.1
27.9
5.8
2.7

0.6
1.2
1.3
0.7
1.4
0.6
0.4

‐1.4
0.5
‐4.1
1.1
4.4
‐0.5
0.6

0.8
1.7
1.8
1.1
1.7
1.0
0.7

No
No
No
No
Yes
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

22.3
37.0
36.5
4.2

0.9
1.0
1.0
0.5

25.1
40.8
31.6
2.5

0.8
0.9
1.0
0.3

‐2.8
‐3.8
5.0
1.7

1.2
1.2
1.4
0.6

Yes
Yes
Yes
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

1.3
61.6
29.4
7.8

0.3
1.4
1.3
0.7

1.7
61.5
28.3
8.5

0.3
1.4
1.2
0.9

‐0.5
0.1
1.0
‐0.7

0.4
2.0
1.8
1.0

No
No
No
No

4.9
60.4
28.3
6.4

0.6
1.4
1.3
0.7

5.2
61.5
26.4
6.9

0.5
1.4
1.1
0.8

‐0.3
‐1.1
1.9
‐0.5

0.8
2.2
1.7
1.1

No
No
No
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol l owi ng Subs cri pti on Types : DSL, ca bl e, fi ber‐opti c, mobi l e, a nd s a tel l i te.

Table 8i. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for CAPI Mode (Use for RQ
Chi‐Sq 
Statistic Significant
Number of computer types
26.3
Yes
Number of subscription types
1.8
No
Number of broadband types
1.3
No
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-20

1

Table 8j. Gross Difference Rates  for the Computer and Internet Categories for MAIL Mode (Use for RQ 8)
Test
(%)
Computer Types
Desktop
Handheld
Other

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

3.3
14.8
6.2

0.2
0.5
0.5

5.1
12.9
4.3

0.3
0.5
0.4

‐1.8
1.9
2.0

0.4
0.7
0.7

Yes
Yes
Yes

7.7
6.2
4.0

0.4
0.3
0.3

3.5
N/A
N/A

0.3
N/A
N/A

4.2
N/A
N/A

0.5
N/A
N/A

Yes
N/A
N/A

5.5
13.0
11.7
6.3
20.8
10.0
4.9

0.4
0.6
0.6
0.5
0.8
0.6
0.4

6.4
13.2
12.1
7.0
22.3
12.6
6.5

0.6
0.7
0.6
0.6
1.0
0.8
0.5

‐0.9
‐0.2
‐0.5
‐0.8
‐1.5
‐2.6
‐1.5

0.7
0.8
0.9
0.8
1.2
1.0
0.7

No
No
No
No
No
Yes
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Standard 
Error (%)

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

1

Table 8k. Gross Difference Rates  for the Computer and Internet Categories for CATI Mode (Use for RQ 8)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Access with a subscription
Access without a subscription
No Internet access
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

4.3
13.8
4.7

0.7
1.3
0.8

7.3
13.0
4.3

0.9
1.2
0.7

‐3.0
0.8
0.4

1.1
1.9
1.1

Yes
No
No

8.7
8.0
3.9

1.0
1.0
0.7

4.1
N/A
N/A

0.7
N/A
N/A

4.6
N/A
N/A

1.2
N/A
N/A

Yes
N/A
N/A

5.8
19.8
18.4
7.9
24.3
4.7
6.4

1.0
1.8
1.9
1.5
2.2
1.1
1.3

7.8
22.3
20.3
9.2
23.4
6.3
7.6

1.2
1.8
1.5
1.5
2.0
1.1
1.2

‐2.0
‐2.5
‐1.9
‐1.4
0.9
‐1.6
‐1.1

1.5
2.6
2.5
1.9
3.1
1.6
1.7

No
No
No
No
No
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

A-21

1

Table 8l. Gross Difference Rates  for the Computer and Internet Categories for CAPI Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

5.9
13.7
7.9

0.6
1.1
1.0

8.5
16.4
6.8

0.8
1.4
0.8

‐2.7
‐2.7
1.2

0.9
1.7
1.4

Yes
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access

11.5
9.6
7.7

0.9
0.8
0.8

6.6
N/A
N/A

0.7
N/A
N/A

4.9
N/A
N/A

1.1
N/A
N/A

Yes
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

4.8
18.6
16.8
5.6
27.1
6.4
6.7

0.8
1.6
1.6
1.1
2.1
0.8
1.1

6.5
19.8
16.2
5.9
23.4
5.5
5.4

1.0
1.8
1.6
1.1
1.8
0.6
0.8

‐1.7
‐1.2
0.6
‐0.3
3.7
0.9
1.4

1.2
2.6
2.3
1.7
2.6
1.0
1.4

No
No
No
No
No
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

1

Table 8m. Index of Inconsistency  for the Computer and Internet Categories for MAIL Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

10.8
30.9
85.9
33.3

0.8
1.1
3.4
0.9

16.0
27.3
83.4
31.5

1.0
1.2
4.2
0.8

‐5.2
3.6
2.5
1.8

1.3
1.5
5.5
1.2

Yes
Yes
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

20.3
80.9
11.9
22.6

1.0
2.9
0.7
0.9

9.7
N/A
N/A
N/A

0.7
N/A
N/A
N/A

10.6
N/A
N/A
N/A

1.3
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

31.6
26.4
23.6
31.2
50.5
69.9
78.5
55.6

2.4
1.2
1.1
2.5
1.8
3.1
4.6
0.8

33.7
26.3
26.6
29.3
53.7
69.8
80.5
59.0

3.0
1.4
1.4
2.3
2.4
3.5
4.4
0.9

‐2.2
0.1
‐3.0
1.8
‐3.2
0.1
‐2.0
‐3.4

3.7
1.6
1.9
3.7
2.9
5.1
6.7
1.2

No
No
No
No
No
No
No
Yes

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-22

1

Table 8n. Index of Inconsistency  for the Computer and Internet Categories for CATI Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

11.3
31.3
75.2
31.7

1.9
2.8
9.2
2.2

19.3
31.3
70.5
34.6

2.3
3.0
8.6
2.0

‐8.0
0.0
4.7
‐2.9

2.7
4.4
11.4
3.0

Yes
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

19.5
87.1
9.6
21.7

2.2
6.1
1.7
2.2

10.3
N/A
N/A
N/A

1.7
N/A
N/A
N/A

9.2
N/A
N/A
N/A

2.8
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

34.6
39.6
36.9
37.5
56.1
51.5
82.4
63.9

6.8
3.7
3.8
6.5
5.1
10.0
11.1
2.4

42.6
44.3
40.6
44.4
60.3
62.0
87.3
67.1

5.4
3.6
3.1
6.7
4.7
9.2
8.4
2.3

‐8.0
‐4.6
‐3.7
‐6.9
‐4.2
‐10.5
‐4.9
‐3.2

8.7
5.2
5.0
8.8
7.2
12.9
14.8
3.4

No
No
No
No
No
No
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-23

1

Table 8o. Index of Inconsistency  for the Computer and Internet Categories for CAPI Mode (Use for RQ 8)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

15.6
28.2
79.2
35.4

1.6
2.3
6.1
1.8

22.1
34.5
84.7
39.6

2.0
2.9
6.6
2.0

‐6.5
‐6.4
‐5.5
‐4.2

2.3
3.4
8.7
2.6

Yes
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

25.1
71.4
19.0
28.8

1.9
5.4
1.9
1.7

14.7
N/A
N/A
N/A

1.7
N/A
N/A
N/A

10.4
N/A
N/A
N/A

2.4
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

51.9
39.5
33.9
38.3
61.7
69.6
94.7
62.7

6.8
3.4
3.3
6.5
4.5
7.3
4.8
2.5

63.7
42.1
33.2
42.6
54.9
42.3
84.9
63.6

5.3
3.8
3.2
7.5
4.1
5.7
9.5
2.3

‐11.8
‐2.6
0.8
‐4.3
6.8
27.3
9.8
‐0.8

8.8
5.5
4.7
10.2
5.8
9.2
10.7
3.2

No
No
No
No
No
Yes
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-24

Table 9a. Item Missing Data Rates of the Computer and Internet Categories for HIGH Stratum (Use for RQ 9)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Yes

2.5
2.9
10.4
29.7

Standard 
Error (%)
0.2
0.2
0.4
0.6

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
2.7
0.2
‐0.1
0.3
No
3.0
0.2
‐0.1
0.3
No
16.7
0.4
‐6.4
0.6
Yes
31.8
0.6
‐2.1
0.8
Yes

4.1

0.3

2.6

0.2

1.5

0.4

Yes

1.1
26.4
20.2
16.2
28.4
26.9
28.6
33.8

0.1
0.5
0.5
0.4
0.5
0.5
0.6
0.6

1.2
30.9
23.6
18.2
33.0
31.4
32.6
36.4

0.1
0.6
0.5
0.5
0.6
0.6
0.6
0.7

‐0.1
‐4.5
‐3.4
‐2.0
‐4.6
‐4.5
‐4.0
‐2.6

0.2
0.7
0.6
0.7
0.8
0.8
0.8
0.9

No
Yes
Yes
Yes
Yes
Yes
Yes
Yes

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."

Table 9b. Item Missing Data Rates of the Computer and Internet Categories for LOW Stratum (Use for RQ 9)
Test
(%)
Computer Types
Desktop
Handheld
Other
Internet Access
Yes

2.7
3.1
8.1
18.5

Standard 
Error (%)
0.2
0.2
0.2
0.4

Control
Standard 
Test ‐ 
Standard 
(%)
Error (%) Control (%) Error (%) Significant
3.2
0.2
‐0.5
0.2
Yes
3.5
0.2
‐0.4
0.2
No
12.3
0.3
‐4.2
0.3
Yes
20.5
0.4
‐2.0
0.5
Yes

4.6

0.2

3.1

0.1

1.5

0.2

Yes

0.8
18.7
14.0
12.3
21.0
19.2
19.6
23.2

0.1
0.4
0.4
0.4
0.5
0.4
0.4
0.5

1.0
21.5
16.0
14.2
24.4
22.2
22.3
25.3

0.1
0.5
0.5
0.4
0.5
0.5
0.5
0.5

‐0.2
‐2.9
‐2.1
‐1.9
‐3.4
‐3.0
‐2.7
‐2.1

0.1
0.7
0.6
0.5
0.7
0.7
0.7
0.7

No
Yes
Yes
Yes
Yes
Yes
Yes
Yes

1

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Source: 2010 Ameri can Communi ty Survey Content Tes t.
1 For the Tes t vers i on, the percenta ge repres ents  thos e who a ns wered the ca tegory "Yes , wi th a  s ubs cri pti on to an Internet 
s ervi ce" a nd for the Control  vers i on, the percenta ge repres ents  thos e who reponded "Yes ."

A-25

Table 9c. Response Distributions for Computer and Internet Categories for HIGH Stratum (Use for RQ 9)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

80.4
41.2
3.9

0.6
0.6
0.3

79.3
35.3
2.4

0.5
0.5
0.2

1.1
5.9
1.5

0.7
0.7
0.4

No
Yes
Yes

Internet Access
Access with a subscription
Access without a subscription
No Internet access

74.9
3.8
21.3

0.6
0.3
0.6

75.5
N/A
N/A

0.6
N/A
N/A

‐0.7
N/A
N/A

0.9
N/A
N/A

No
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

6.4
33.3
48.5
7.9
23.0
6.1
2.2

0.3
0.6
0.6
0.4
0.5
0.3
0.2

6.3
33.9
53.7
7.7
20.6
6.8
2.5

0.3
0.6
0.7
0.3
0.6
0.4
0.2

0.1
‐0.6
‐5.3
0.3
2.5
‐0.7
‐0.3

0.4
0.9
0.9
0.5
0.9
0.5
0.3

No
No
Yes
No
Yes
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

17.5
42.0
38.0
2.6

0.5
0.6
0.6
0.3

18.7
46.9
33.1
1.3

0.4
0.6
0.6
0.2

‐1.2
‐4.9
4.9
1.2

0.7
0.8
0.7
0.3

No
Yes
Yes
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

7.2
65.4
21.7
5.7

0.3
0.6
0.6
0.3

3.4
69.6
20.6
6.3

0.2
0.7
0.6
0.4

3.7
‐4.2
1.1
‐0.6

0.3
1.0
0.9
0.5

Yes
Yes
No
No

9.8
63.9
21.3
4.9

0.4
0.7
0.6
0.3

5.4
69.3
19.8
5.5

0.3
0.7
0.6
0.4

4.4
‐5.4
1.6
‐0.5

0.4
1.0
0.8
0.5

Yes
Yes
No
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol lowi ng Subs cri pti on Types : DSL, ca bl e, fiber‐optic, mobil e, a nd s a tel l i te.

Table 9d. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for HIGH Stratum (Use for RQ 9)
Chi‐Sq 
Statistic Significant
Number of computer types
58.5
Yes
Number of subscription types
81.3
Yes
Number of broadband types
75.9
Yes
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-26

Table 9e. Response Distributions for Computer and Internet Categories for LOW Stratum (Use for RQ 9)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

68.0
34.9
4.0

0.6
0.6
0.2

65.7
29.2
2.9

0.6
0.5
0.2

2.2
5.7
1.1

0.8
0.8
0.3

Yes
Yes
Yes

Internet Access
Access with a subscription
Access without a subscription
No Internet access

59.3
5.6
35.1

0.6
0.3
0.5

60.1
N/A
N/A

0.7
N/A
N/A

‐0.7
N/A
N/A

0.9
N/A
N/A

No
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

7.6
35.1
45.4
6.3
23.1
6.7
2.7

0.4
0.6
0.7
0.3
0.5
0.3
0.2

9.0
38.4
47.2
5.9
20.3
7.6
2.8

0.4
0.6
0.7
0.3
0.6
0.4
0.2

‐1.4
‐3.3
‐1.8
0.4
2.8
‐0.9
‐0.1

0.5
0.8
0.9
0.4
0.6
0.5
0.3

Yes
Yes
No
No
Yes
No
No

Number of computer types
0 computer types
1 computer type
2 computer types
3 computer types

28.2
38.9
30.6
2.2

0.6
0.6
0.5
0.2

30.1
43.5
25.0
1.5

0.6
0.6
0.4
0.1

‐1.9
‐4.6
5.7
0.8

0.8
0.8
0.7
0.2

Yes
Yes
Yes
Yes

Number of subscription types
0 subscription types
1 subscription type
2 subscription types
3 subscription types

9.7
62.3
21.6
6.4

0.3
0.6
0.5
0.3

5.9
66.8
19.9
7.4

0.2
0.5
0.5
0.3

3.8
‐4.5
1.7
‐1.0

0.4
0.8
0.7
0.5

Yes
Yes
Yes
No

13.2
60.8
20.6
5.3

0.4
0.6
0.5
0.3

9.5
65.7
19.1
5.7

0.3
0.6
0.5
0.3

3.7
‐4.9
1.6
‐0.4

0.5
0.8
0.7
0.4

Yes
Yes
Yes
No

Number of broadband types
0 broadband types
1 broadband type
2 broadband types
3 broadband types

1

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1  The number of broa dba nd types  i ncl ude the fol lowi ng Subs cri pti on Types : DSL, ca bl e, fiber‐optic, mobil e, a nd s a tel l i te.

Table 9f. Rao‐Scott Chi‐Square of the Multiple Answer Computer and Internet Use Categories for LOW Stratum (Use for RQ 9)
Chi‐Sq 
Statistic Significant
Number of computer types
77.2
Yes
Number of subscription types
89.8
Yes
Number of broadband types
61.8
Yes
Source: 2010 Ameri ca n Communi ty Survey Content Tes t.

A-27

1

Table 9g. Gross Difference Rates  for the Computer and Internet Categories for HIGH Stratum (Use for RQ 9)
Test
(%)
Computer Types
Desktop
Handheld
Other

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

3.4
13.6
7.3

0.4
0.7
0.7

5.8
14.0
5.8

0.4
0.8
0.6

‐2.4
‐0.4
1.5

0.5
0.9
1.0

Yes
No
No

8.1
7.1
4.2

0.5
0.5
0.4

3.9
N/A
N/A

0.4
N/A
N/A

4.1
N/A
N/A

0.6
N/A
N/A

Yes
N/A
N/A

4.5
14.3
12.8
5.9
23.6
7.8
5.7

0.5
0.8
0.8
0.6
1.1
0.6
0.7

5.7
15.9
12.8
6.7
22.5
8.7
6.2

0.6
1.0
0.8
0.7
1.1
0.6
0.6

‐1.2
‐1.6
0.0
‐0.8
1.2
‐0.9
‐0.5

0.8
1.3
1.1
1.0
1.5
0.8
0.9

No
No
No
No
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

Standard 
Error (%)

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

1

Table 9h. Gross Difference Rates  for the Computer and Internet Categories for LOW Stratum (Use for RQ 9)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other

7.1
16.6
5.6

0.4
0.7
0.5

8.9
15.3
4.5

0.5
0.7
0.4

‐1.8
1.2
1.1

0.7
0.8
0.6

Yes
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access

12.2
8.8
8.5

0.6
0.5
0.5

6.8
N/A
N/A

0.4
N/A
N/A

5.4
N/A
N/A

0.7
N/A
N/A

Yes
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other

7.8
19.9
18.2
7.0
23.0
9.0
6.1

0.6
1.0
1.0
0.7
1.0
0.6
0.5

9.3
19.1
20.0
7.0
24.4
9.0
5.8

0.6
1.0
1.1
0.6
1.0
0.7
0.6

‐1.5
0.9
‐1.8
0.0
‐1.4
0.1
0.4

0.9
1.6
1.6
0.9
1.3
0.9
0.8

No
No
No
No
No
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Gros s  Di fference Ra te (GDR) i s  the percenta ge of i ncons i s tent a ns wers  between the ori gi na l  i ntervi ew a nd the Content Fol l ow‐
Up i ntervi ew.

A-28

1

Table 9i. Index of Inconsistency  for the Computer and Internet Categories for HIGH Stratum (Use for RQ 9)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

11.1
27.9
83.9
32.8

1.2
1.4
4.7
1.1

18.2
29.2
85.1
34.3

1.2
1.7
5.1
1.1

‐7.0
‐1.3
‐1.2
‐1.5

1.7
1.9
7.1
1.5

Yes
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

21.1
75.2
12.8
24.0

1.2
4.3
1.1
1.1

10.8
N/A
N/A
N/A

1.0
N/A
N/A
N/A

10.3
N/A
N/A
N/A

1.5
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

32.3
29.3
25.9
31.4
55.1
67.3
85.5
56.5

3.0
1.5
1.6
2.8
2.5
3.6
4.6
1.0

40.9
32.2
27.5
32.5
53.1
60.2
81.3
59.6

3.6
1.9
1.6
3.1
2.5
4.3
4.8
1.0

‐8.6
‐2.8
‐1.6
‐1.0
2.1
7.1
4.2
‐3.1

5.2
2.6
2.4
4.5
3.4
5.9
6.7
1.6

No
No
No
No
No
No
No
Yes

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-29

1

Table 9j. Index of Inconsistency  for the Computer and Internet Categories for LOW Stratum (Use for RQ 9)
Test
(%)

Standard 
Error (%)

Control
(%)

Standard 
Test ‐ 
Standard 
Error (%) Control (%) Error (%) Significant

Computer Types
Desktop
Handheld
Other
Aggregate (L‐fold) Indices

16.5
36.2
73.8
37.4

1.0
1.5
4.6
1.0

20.5
35.5
75.8
36.3

1.1
1.6
4.1
0.9

‐3.9
0.7
‐2.0
1.1

1.6
1.8
6.2
1.4

Yes
No
No
No

Internet Access
Access with a subscription
Access without a subscription
No Internet access
Aggregate (L‐fold) Indices

25.1
80.0
18.7
28.1

1.2
2.9
1.1
1.1

14.3
N/A
N/A
N/A

0.9
N/A
N/A
N/A

10.8
N/A
N/A
N/A

1.4
N/A
N/A
N/A

Yes
N/A
N/A
N/A

Subscription Types
Dial‐up
DSL
Cable
Fiber‐Optic
Mobile
Satellite
Other
Aggregate (L‐fold) Indices

50.3
41.3
36.4
44.9
56.0
72.2
87.3
63.9

3.2
2.1
2.0
4.0
2.4
3.1
4.4
1.0

49.3
38.4
40.1
45.8
61.2
55.9
90.5
66.2

3.4
2.1
2.1
3.8
2.6
3.5
4.4
1.1

1.0
2.9
‐3.7
‐0.9
‐5.3
16.3
‐3.2
‐2.3

4.8
3.2
3.1
5.2
3.0
4.3
6.4
1.7

No
No
No
No
No
Yes
No
No

Source: 2010 Ameri ca n Communi ty Survey Content Tes t.
1 The Index of Incons i s tency (IOI) i s  the percenta ge of tota l  va ri a nce due to the s i mpl e res pons e va ri a nce a nd provi des  a n es ti ma te of 
the ma gni tude of res pons e va ri a bi l i ty. The Cens us  Burea u's  genera l  benchma rk for rel i a bi l i ty s ta tes  tha t i ndex va l ues  of l es s  tha n 20 
percent i ndi ca te l ow i ncons i s tency, 20 to 50 percent i ndi ca te modera te i ncons i s tency, a nd over 50 percent i ndi ca te hi gh i ncons i s tency.

A-30

Appendix B: Images of the Mail Versions of the Control and Test Questions
ACS Control Version

ACS Test Version

B-1

Appendix C: CATI and CAPI Versions of the Control and Test Questions
ACS Control Version
Q9. At this [house/apartment/mobile home] do you or any member of this household
subscribe to the Internet?
Yes
No [Skip to Q11a]
Q10a. At this [house/apartment/mobile home] do you or any member of this household
subscribe to the Internet using a dial-up service?
Yes
No
Q10b. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a DSL service?
Yes
No
Q10c. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a cable-modem service?
Yes
No
Q10d. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a fiber-optic service?
Yes
No
Q10e. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a mobile broadband plan for a computer or a cell phone?
Yes
No
Q10f. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a satellite service?
Yes
No

C-1

ACS Control Version
Q10g. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using some other service?
Yes [Go to Q10h]
No [Go to Q11a]

Q10h. What is this other type of Internet service?
_________________________________________

Q11a. For the next few questions about computers, EXCLUDE GPS devices, digital music
players, and devices with only limited computing capabilities, for example: household
appliances.
At this [house/apartment/mobile home] do you or any member of this household own
or use a desktop, laptop, netbook, or notebook computer?
Yes
No
Q11b. At this [house/apartment/mobile home] Do you or any member of this
household own or use a handheld computer, smart mobile phone, or other handheld
wireless computer?
FR Instruction: EXCLUDE GPS devices, digital music players, and devices with only limited
computing capabilities, for example: household appliances.
Yes
No
Q11c. At this [house/apartment/mobile home] Do you or any member of this
household own or use some other type of computer?
FR Instruction: EXCLUDE GPS devices, digital music players, and devices with only limited
computing capabilities, for example: household appliances.
Yes [Go to Q11d]
No [Go to Q12]
Q11d. What is this other type of computer?
FR Instruction: EXCLUDE GPS devices, digital music players, and devices with only limited
computing capabilities, for example: household appliances.
________________________________________________

C-2

ACS Test Version
Q9a. For the next few questions about computers, EXCLUDE GPS devices, digital music
players, and devices with only limited computing capabilities, for example: household
appliances.
At this [house/apartment/mobile home] do you or any member of this household own or
use a desktop, laptop, netbook, or notebook computer?
Yes
No
Q9b. At this [house/apartment/mobile home] Do you or any member of this household own
or use a handheld computer, smart mobile phone, or other handheld wireless computer?
FR Instruction: EXCLUDE GPS devices, digital music players, and devices with only limited
computing capabilities, for example: household appliances.
Yes
No
Q9c. At this [house/apartment/mobile home] Do you or any member of this household own
or use some other type of computer?
FR Instruction: EXCLUDE GPS devices, digital music players, and devices with only limited
computing capabilities, for example: household appliances.
Yes [Go to Q9d]
No [Go to Q10a]
Q9d. What is this other type of computer?
FR Instruction: EXCLUDE GPS devices, digital music players, and devices with only limited
computing capabilities, for example: household appliances.
___________________________________________________
Q10a. At this [house/apartment/mobile home] do you or any member of this household
access the Internet?
Yes
No -> Skip to Q12
Q10b. At this [house/apartment/mobile home] Do you or any member of this household
access the Internet with or without a subscription to an Internet service?
With a subscription to an Internet service
Without a subscription to an Internet service -> Skip to Q12

C-3

ACS Test Version
Q11a. At this [house/apartment/mobile home] do you or any member of this household
subscribe to the Internet using a dial-up service?
Yes
No
Q11b. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a DSL service?
Yes
No
Q11c. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a cable-modem service?
Yes
No
Q11d. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a fiber-optic service?
Yes
No
Q11e. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a mobile broadband plan for a computer or a cell phone?
Yes
No
Q11f. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using a satellite service?
Yes
No
Q11g. At this [house/apartment/mobile home] Do you or any member of this household
subscribe to the Internet using some other service?
Yes [Go to Q11h]
No [Go to Q12]
Q11h. What is this other type of Internet service?
_________________________________________

C-4

Current Population Survey 2010 Questionnaires
COMP1

At home, [do you / do you or any member of this household] own or use
any of the following computers?
<1> A desktop, laptop, netbook, or notebook computer
<2> A handheld computer, smart mobile phone, or other handheld wireless
computer
<3> Some other type of computer (Please Specify) (Go To COMP1s)
<4> Do not own a computer

COMP1s

What is this other type of computer?

NET2a

At home, [do you / do you or any member of this household] access the
Internet?
<1> Yes (Skip to N2WHO)
<2> No (Skip to NET4a1)

N2WHO
NET2b

Who is that? Enter person’s line number (1-16)
At home, [do you / does this household] access the Internet with or without
a subscription to an Internet service?
<1> With a subscription to an Internet service
<2> Without a subscription to an Internet service

SERV3

At home, [do you / does this household] access the Internet using<1> Dial-up service (Skip to NET4a)
<2> DSL service (skip to NET5a)
<3> Cable modem service (skip to NET5a)
<4> Fiber-optic service (skip to NET5a)
<5> Mobile broadband plan for a computer or a cell phone (skip to NET5a)
<6> Satellite service (skip to NET5a)
<7> Some other service (skip to SERVsp)

SERVsp

What is this other type of Internet service?

NET4a

What is the main reason that you do not have high-speed (that is, faster
than dial-up) Internet access at home?
<1> Don’t need it, not interested (Go To NET5a)
<2> Too expensive (Go To NET4b)
<3> Can use it somewhere else (Go To NET5a)
<4> Not available in area (Go To NET5a)
<5> Computer inadequate (Go To NET5a)
<6> Other reason(s) (Go To NET4sp)

NET4sp

What other reasons?

C-5

NET4a1

What is the main reason that you do not have Internet access at home?
<1> Don’t need it, not interested (Go To NET5a)
<2> Too expensive (Go To NET4b)
<3> Can use it somewhere else (Go To NET5a)
<4> Not available in area (Go To NET5a)
<5> No Computer or Computer inadequate (Go To NET5a)
<6> Other reason(s) (Go To N4a1sp)

N4a1sp

What other reasons?

NET4b

What costs are you most concerned about?
<1> Cost of the computer and/or other hardware (e.g., modem) (Go to NET5a)
<2> Cost of installing internet service (Go to NET5a)
<3> Cost of monthly internet service (Go to NET5a)
<4> Some other cost (Go to N4bsp)

N4bsp

What is that other cost?

NET5a

^NETFILL,[do you \ do you or any member of this household] access the
Internet at any location outside the home?
<1> Yes (Go To NET6) (If NUMHOU=1 Go To
Else If NUMHOU>1 Go To )
<2> No (Go To END)

NET6
NET7

Who is that? Enter persons line number (1-16)
[Do you / Do you or any member of this household] access the Internet at
any of the following locations outside the home?
<1> Workplace (Go To End)
<2> School (Go To End)
<3> Public library (Go To End)
<4> Community center (Go To End)
<5> Internet café / coffee shop in your area (Go To End)
<6> Someone else’s house (Go To End)
<7> Another place outside the home (Go To NET7sp)

NET7sp

What other place outside the home?

C-6

Appendix D: Flow of the Content Follow-Up

D-1

Appendix E: Information Page
Research Questions & Evaluation Measures:
No.

Research Questions

Evaluation Measures

1.

Does either version of the computer ownership and
Internet subscription questions have lower item
missing data rates?

• Compare the item missing data rates of the two
versions.

2.

Do the two versions of the computer ownership
and Internet subscription questions have similar or
different response distributions?

• Compare the response distributions of the two
versions.

3.

Which version results in more reliable estimates
(at equal levels of detail)?

• Compare the gross difference rates and indexes of
inconsistency of the two versions, using data
obtained from the CFU, for individual and
comparable response categories.

4.

Are the relative estimates of computer ownership
and Internet subscription roughly comparable to
existing data sources?

• Compare the computer ownership estimates to
similar statistics from the 2003 CPS Computer and
Internet Supplement data and 2004 SIPP Wave 5
data to see if the Content Test statistics are roughly
similar.

Note: The comparisons need to be interpreted
cautiously because of the different survey designs
and universes as well as the age of the comparable
data, given that this is a very time-sensitive topic.

• Compare the Internet subscription estimates to
similar statistics from the 2007 CPS Computer and
Internet Supplement to see if the Content Test
statistics are roughly similar.

5.

For the write-in responses to the “other”
categories, is there a frequent response that
indicates a need to edit the instructions and/or the
examples given for a response category?

• When applicable, evaluate the frequencies of the
coded responses individually and relative to the
overall distribution for the question.

6.

For each mode of data collection, does either
version have lower item missing data rates?

• For each mode of data collection, compare the item
missing data rates of the two versions.

7.

For each mode of data collection, do the two
versions have similar or different response
distributions?

• For each mode of data collection, compare the
response distributions of the two versions.

8.

For urban versus rural areas, does either version
have lower item missing data rates?

• For both urban and rural areas, compare the item
missing data rates of the two versions.

9.

For urban versus rural areas, do the two versions
have similar or different response distributions?

• For both urban and rural areas, compare the
response distributions of the two versions.

10.

For urban versus rural areas, which version results
in more reliable estimates (at equal levels of
detail)?

• For both urban and rural areas, compare the gross
difference rates and indexes of inconsistency of the
two versions, using data obtained from the CFU, for
individual and comparable response categories.

11.

For each mail response stratum (high and low
response areas), does either version of the
computer ownership and Internet subscription
questions have lower item missing data rates?

• For each mail response stratum, compare the item
missing data rates of the two versions.

E-1

Research Questions & Evaluation Measures:
No.

Research Questions

Evaluation Measures

12.

For each mail response stratum, do the two
versions have similar or different response
distributions?

• For each mail response stratum, compare the
response distributions of the two versions.

13.

For each mail response stratum, which version
results in more reliable estimates (at equal levels
of detail)?

• For each mail response stratum, compare the gross
difference rates and indexes of inconsistency of the
two versions, using data obtained from the CFU, for
individual and comparable response categories.

14.

Does either version elicit respondent and/or
interviewer behaviors that may contribute to
interviewer and/or respondent error?

• Compare the behavior coding results of the two
versions derived from the CARI recordings.

Selection Criteria:
Research
Question
1,2,3

Criteria
The item missing data rates, response distributions, and reliability (as measured by gross
difference rates and the Index of Inconsistency) will be considered together when determining
which version performs better.

Additional Criteria for Descriptive Statistics: (to be analyzed after all selection criteria)
Research
Question
4-14

Criteria
Not part of selection criteria, they are presented to give information to help plan for the use of
the computer ownership and Internet subscription data.

E-2

The two test versions to be field tested will be decided upon after cognitive testing is complete.
Test Version 1

Test Version 2

E-3

Test Version 3