Attachment A-3
November
2017 Current Population Survey Computer and Internet Use
Supplement
Comments on Proposed Information Collection
On January 9, 2017, the National Telecommunications and Information Administration (NTIA) published a notice in the Federal Register requesting comments on the next edition of its Computer and Internet Use Supplement to the Census Bureau’s Current Population Survey (CPS), pursuant to the provisions of the Paperwork Reduction Act (PRA). NTIA received four comments in response to this notice. All four comments contained valuable and thoughtful feedback that NTIA has considered in the course of finalizing its proposed information collection. NTIA summarizes and responds to each comment below.
In its letter, Etsy expresses its support for “NTIA’s interest in further investigating the innovative ways that people use the internet today,” and suggests additional questions to gather further detail on the extent to which individuals rely on selling goods online to make a living. In particular, Etsy recommends asking respondents whether they earn their primary and/or supplemental income from selling goods on the Internet, and if so, whether they do so “by choice or out of necessity.” The company further recommends two follow-up questions directed at Internet users who answer affirmatively to EGOODS in the draft survey instrument (asking whether the individual uses the Internet to sell goods, such as by selling items on Etsy or eBay): 1) what percentage of the individual’s income is derived from online sales; and 2) whether the individual uses the Internet to sell goods to markets outside the United States.
NTIA is very interested in fostering a better understanding of the importance of the digital economy, and appreciates Etsy’s interest in expanding this line of questioning. In fact, NTIA is proposing to add the new EGOODS question to its CPS Supplement for precisely this purpose. The additional questions proposed by Etsy could potentially add further depth to research on this topic; however, NTIA notes that the proposed survey instrument seeks to cover a wide range of topics, and that there is little space for adding additional questions without removing existing ones. NTIA has been advised by the Census Bureau that response rates for CPS supplements may suffer when average response time exceeds ten minutes, so it is important to ensure that the length of the survey instrument remains below that threshold. For this edition of the CPS Supplement, NTIA believes it is most prudent to proceed with the new EGOODS question alone, and then to consider expanding the line of questioning in a future CPS Supplement if initial results and current policy priorities warrant it.
EveryoneOn expresses a number of ideas primarily aimed at expanding on the CPS Supplement’s existing questions aimed at gathering data on the digital divide. For example, the organization recommends adding a detailed assessment of Internet users’ digital literacy levels by “asking a question evaluating comfort with” each of the sixteen online activities asked about in the USEINT section of the survey, by using “a scale from 1 to 5.” EveryoneOn also advocates for adding similar questions for households that don’t use the Internet that “delve deeper into the respondent’s comfort with digital tasks.” NTIA agrees that understanding levels of comfort with technology is an important area for research, and that a set of binary questions about online activities is an imperfect vehicle for understanding varying levels of digital literacy. Unfortunately, the EveryoneOn proposal to improve in this area would either necessitate a drastic expansion of the survey that may reduce response rates, or require NTIA to remove a large number of other important survey questions. Additionally, the proposed comfort level scale would require significant cognitive testing prior to use in production. That said, while this particular proposal may be difficult to implement, NTIA will keep the need for more extensive digital literacy data in mind when contemplating future modifications to the CPS Supplement.
EveryoneOn also advocates for additional expansions to the section of the questionnaire that gathers data from households not using the Internet. Specifically, the organization suggests gauging “what Internet-based activities are the strongest motivators behind unconnected folks’ desire to connect to the Internet,” e.g., the prospect of engaging in online banking or another activity. EveryoneOn further recommends modifying PSENSI (asking whether households not using the Internet at home would buy home Internet service if offered at a lower price) “ to ascertain what the ‘lower price’ at which people would purchase Internet is.” NTIA is always seeking a better understanding of households and individuals not using the Internet, and these are interesting ideas that merit further consideration. Such questions would, however, require extensive development and testing using resources not currently available for the November 2017 edition of the CPS Supplement.
Next, EveryoneOn recommends asking additional questions about the affordability and “data sufficiency” of Internet service plans. In its comments, EveryoneOn notes that “it would be useful to learn about price points for current Internet subscribers.” NTIA agrees that these data would be useful; in fact, questions about Internet access service pricing were included in the 2011 and 2013 editions of the CPS Supplement. Unfortunately, that experience suggests it is difficult to obtain accurate pricing data through a household survey. Bundles with other services, modem rentals, taxes and fees, and special offers may contribute to confusion and inconsistency in reported prices, and survey respondents are unlikely to look at a bill from their service provider in the middle of an interview. NTIA believes that, at least in terms of data quality, it would be more prudent to obtain pricing information directly from Internet service providers—though such an initiative would surely come with its own set of challenges. Separately, EveryoneOn recommends gathering “data on the amount of data that consumers purchase.” This is an interesting idea, and possibly less complicated than pricing data, but would require testing to ensure respondents can consistently report on the varying nuances of data plans, e.g., whether mobile data caps are per-device or a shared pool, and whether plans that are advertised as either “unlimited” or without any reference to data limits have quotas that could result in throttling or surcharges.
Finally, EveryoneOn suggests adding two further questions to the CPS Supplement: 1) whether “respondents are using the Internet for educational purposes (accessing resources, studying for standardized tests, reviewing material, taking self-paced enrichment, etc.) beyond enrolled online courses or job training;” and 2) whether “people are accessing federal or state-level social benefits online.” On the first proposal, as EveryoneOn acknowledges, NTIA is planning to (and did in 2015) ask Internet users whether they use the Internet for educational classes or job training, through the EDTRAI question. While more information about online educational activities would undoubtedly be useful for researchers, NTIA must balance the desire for more detail on a particular topic against the need to gather data on other topics related to computer and Internet use. The second proposal is interesting, though NTIA would want to investigate whether usage statistics (ideally, with some demographic breakouts) might be available directly from sites like benefits.gov prior to adding a question about using those sites.
In a comment submission that includes some points raised in ITIF’s comments submitted for the 2015 edition of the CPS Supplement, ITIF breaks out its recommendations into two parts: additional areas of inquiry beyond NTIA’s proposed questions, and suggested changes to NTIA-proposed questions. The first set of additional questions recommended by ITIF is aimed at better understanding “the true economic impact of broadband deployment and access, including the increased productivity that comes from citizens using e-government services.” Specifically, ITIF reiterates its prior request for respondents to specify how much money they spent on goods or services ordered online, what proportion of that amount was spent on physical goods delivered to the household, how often respondents used websites for rating consumer goods or services, and how often they used government services online. While NTIA has not had space in the CPS Supplement to add these questions, we recognize the potential value of gathering such data. We continue to note that parties interested in e-commerce may benefit from the Census Bureau’s existing E-Stats reports and related studies, which for some economic impact studies may be a better source than a household survey.
ITIF continues to recommend adding two questions regarding what it characterizes as “important inhibitors of the Internet economy.” First, it proposes asking how often respondents “accessed movies, music, or other digital products or services” from a website that the respondent suspects did not have distribution rights for that content. NTIA appreciates the need to gather data that could help inform policies aimed at protecting intellectual property online. However, we remain skeptical that a government agency could elicit accurate responses from respondents as to whether they have engaged in potentially illegal activity. The CPS is a voluntary survey, and while the Census Bureau emphasizes the confidential nature of responses, this kind of question may increase the risk of respondents either refusing to proceed or not responding accurately. Additionally, ITIF suggests a question about how often respondents “used software to block online advertisements.” We agree that data on the use of such software could be interesting, though we suspect respondents may have difficulty answering accurately. Most modern web browsers include the ability to block so-called “pop-up” advertisements, and it is not clear whether respondents would think of web browsers as software used to block advertisements. Moreover, applications that block online advertisements are sometimes also aimed at increasing security, protecting user privacy, or avoiding excessive use of computing power.
ITIF’s final suggestion for additional questions is aimed at exploring “how households use mobile broadband and how this differs from fixed broadband.” ITIF remains concerned that “the survey fails to draw a distinction between the services that respondents access on mobile broadband compared to the services they access on fixed broadband.” To remedy this perceived shortcoming, ITIF again recommends asking respondents separately about the online activities in which they participate while using mobile Internet services and while using fixed Internet services. NTIA appreciates this suggestion, as the question of to what extent Internet users behave differently while using mobile networks and fixed networks is an important one. Unfortunately, our experience suggests there may be substantial barriers to obtaining accurate data in this area. Cognitive testing and reports from the field from both the 2015 and 2017 editions of the CPS Supplement and the new computer and Internet use questions on the American Community Survey (ACS) suggest that many people have difficulty identifying the exact technologies they use to get on the Internet. This issue is what led us in the 2015 survey to combine different types of fixed technologies (e.g., cable, DSL, and fiber optic service), and to substantially revise language describing mobile broadband, in our questions about the types of technologies households use to go online (which NTIA is proposing to further refine in the 2017 survey). Furthermore, given the prevalence of Wi-Fi networks, users are not always certain as to when they are connected to the Internet via a fixed or a mobile network.
In addition to recommending additional questions, ITIF again suggests revising two proposed questions related to online privacy and security. The commenter is concerned that “the proposed privacy questions will not help policymakers make better decisions or aid critical analysis.” The commenter first discusses PSPRE, which asks respondents whether privacy or security concerns have stopped them from undertaking specific activities online during the past year (with a list of categories read to the respondent). ITIF asserts that “there is no attempt to differentiate between significant and insignificant privacy and security concerns” in this question, and proposes modifying the question to ask whether privacy or security concerned “substantially impeded” the respondent’s “ability” to undertake the activities at issue. NTIA respectfully disagrees with both the characterization of the proposed question and with the suggested modifications. It is common for individuals to have concerns about privacy and security, but concerns that are great enough to stop someone from banking, buying goods or services, utilizing social media, or expressing an opinion online are, in our view, significant by definition. ITIF’s proposed language may, depending on interpretation, either elicit less-significant concerns (because the behavior was “impeded” rather than “stopped”), or else in almost no affirmative answers at all (if “ability” is interpreted literally). Furthermore, NTIA hesitates to dramatically revise PSPRE, which was part of the 2015 survey, as there is significant value in having data that provides insight into whether potential impacts from privacy and security concerns are becoming more or less common over time.
ITIF also expresses concerns about PSCON, asserting that the question “uses vague language and fails to put the respondent’s concerns about privacy and security in context and assess their relative importance.” The question, as drafted by NTIA, would ask respondents what concerns them the most when it comes to online privacy and security risks, while ITIF would prefer to “ask respondents about Internet concerns overall… so that respondents can discuss affordability, ease of use, or other issues that may be important to them” in addition to privacy and security, “and then rank those concerns.” While this remains an interesting suggestion, it still seems more like a new question than a modification of PSCON, which is aimed specifically at assessing the most pressing concerns, if any, Americans have about privacy and security online. Alternately, ITIF recommends at a minimum that “the response should allow respondents to differentiate between items of high and low concern” within privacy and security. In asking households what concerns them “the most” about privacy and security risks, NTIA is aiming to continue gathering data on those risks which are of high concern to Internet users. This is part of the reason why interviewers do not read response options for this question, and can select “no concerns” when respondents do not indicate any particular areas of concern. NTIA has also chosen not to read response options in order to avoid leading respondents in one direction or another, and any future modification to this question should be structured to avoid that risk.
Finally, ITIF suggests altering the proposed questions about online activities to rank how often respondents use these applications and services, as opposed to just learning whether they use them or not. We agree that such additional detail could be of use to researchers and policymakers, and will consider it for future editions of the CPS Supplement. Unfortunately, a side effect of this methodology would be to significantly increase the amount of time it takes for respondents to complete the survey. It may also increase difficulty and reduce the accuracy of proxy response, which takes place when the randomly-selected Internet user in the household for the series on online activities is not the household respondent.
The commenter suggests improving the 2017 supplement by including more questions and examples to clarify the survey. At the outset, USTelecom also recommends that NTIA conduct the CPS Supplement annually to enable policymakers to respond to rapidly changing technologies and applications influencing broadband adoption. NTIA appreciates the commenter’s concern, but limited fiscal and staff resources do not allow for an annual data collection. Further, NTIA has previously conducted surveys each calendar year from 2009 to 2013 (Oct. 2009, Oct. 2010, July 2011, Oct. 2012, and July 2013), although at intervals of 9 and 15 months depending on the availability of a CPS supplement on the Census Bureau’s schedule. Based on that experience, we determined that biennial surveys in November, resources permitting, provide sufficient time for new technologies, applications and service offerings to permeate the market and facilitate survey responses that reflect the changing landscape.
As USTelecom requests, NTIA will consider adding questions in future CPS supplements that ask about more refined device categories as new technologies develop. Specifically, the commenter suggests that NTIA include “standalone e-readers” to distinguish “detachable hybrid tablets” from the LAPTOP and TABLET responses, the latter of which lists e-readers as an example. USTelecom also urges NTIA to add digital glasses and virtual reality devices to exclude them from smart watches and fitness monitors described as examples in WEARAB. The recommendations reflect NTIA’s challenge surveying respondents about emerging devices and their uses without confusing survey takers with options they do not understand. While we understand the commenter’s desire for data on early adoption of new technologies, NTIA believes the proposed questions will not yield useful data that justify devoting additional questions to the topic at this time.
USTelecom also suggests that NTIA delete INTRAV, which seeks information about Internet use while traveling because the question is too broad. We acknowledge that the question encompasses a variety of transportation modes for online users, but want to gather information about overall Internet use while traveling irrespective of the mode. Therefore, we will retain the question as drafted. Similarly, we decline to ask respondents to identify the “other” locations where they go online because the survey asks about Internet use at the most common locations, including at home (INHOME), work (INWORK), school (INSCHL), coffee shop or business (INCAFE), library or community center (INLICO), someone else’s home (INOTHR), and while traveling (INTRAV).
The question of technology type for home Internet use in HOMTE should include fixed terrestrial wireless as an option, according to the USTelecom comments. Although the commenter notes FCC data on the growth in fixed wireless subscriptions to 939,000 in 2015, NTIA will monitor adoption of this technology to consider listing it as an example in future editions of the survey.
Further, the commenter recommends deleting the HOMSU question about types of Internet service providers because USTelecom disagrees with NTIA’s groupings of illustrative companies we consider responsive. Further, USTelecom expresses concern about the vagueness of the term “public agency” as an entity providing Internet service. Although respondents could not provide examples of such a public agency during cognitive testing, NTIA is aware that some public housing authorities and municipalities offer broadband service. Despite these concerns, NTIA finds value in understanding households’ awareness of their service providers.
Similarly, we believe it useful to gather data in PECOMP on Americans’ perceptions about competition for home Internet service. USTelecom objects to this question because such perceptions may be inaccurate. Instead, the commenter proposes that NTIA broaden the examples to include mobile wireless or to inquire whether consumers consider mobile broadband as an alternative to fixed broadband Internet service at home. NTIA has revised the survey to minimize respondents’ confusion about whether mobile broadband is considered a home Internet service by asking separately in MOBDAT if household members access the Internet using a data plan for a cell phone, smartphone, tablet, or other mobile device. In HOMETE, NTIA specifically excludes mobile data plans from the question about home Internet use since by its nature, mobile broadband is accessible at numerous locations, including home. Moreover, consistent with the commenter’s recognition of the importance of respondents clearly understanding technical distinctions between types of mobile wireless service, such as Wi-Fi, NTIA’s experience reveals that most respondents are unlikely to have such knowledge. Consequently, the reliability of the data USTelecom recommends that NTIA collect would be questionable.
The commenter suggests that NTIA include more examples to questions about buying or selling goods or services in the “sharing economy” in USESVC, ESVCS and ECOMME, as well as to MEDDOC regarding communications with health professionals. Given that numerous examples are possible to illustrate intended responses, NTIA appreciates USTelecom’s desire to provide as many options as possible to stimulate responses. We work with Census staff to find widely recognized examples to avoid providing an exhaustive list of potential choices that would unduly lengthen the survey beyond the 10 minutes allotted for the supplement. Therefore, as an example, we included “requesting a home repair through Angie’s List” to USESVC at Census’ urging to offer an example that may be more readily available to rural respondents than Uber, Lyft or Airbnb.
NTIA also declines to adopt USTelecom’s suggestion to clarify in PSPRE whether privacy or security concerns caused household members to cease listed activities entirely or only with a certain vendor. We do not believe the question requires further clarification since cognitive testing did not indicate PSPRE confused respondents.
NTIA agrees with USTelecom about the utility of adding “I don’t know how to use” as an answer choice to questions about reasons household members do not use the Internet at home or elsewhere (NOHM, PRINOH, NOOU, and PRINOO) and will consider adopting the recommendation in future CPS supplements. We disagree, however, with the commenter’s suggestion that interviewers probe further about whether the decision to not use the Internet in or out of home in NOHM, PRINOH, NOOU, and PRINOO because of cost concerns relates to the price of service or devices. A past CPS supplement that asked respondents to differentiate between these costs did not yield useful results, therefore we have returned to our previous formulation asking about overall costs. We also decline to delete PSENSI asking whether respondents would buy lower priced home Internet service, as USTelecom suggests. NTIA has determined the question helps to confirm whether the affordability of home Internet service is the true deterrent even if respondents are unwilling to say so.
Finally, USTelecom asks NTIA to make clear that TVINT questions asking about cable TV and satellite TV service include TV services offered by telecommunications companies. Although cognitive testing did not reveal that respondents experienced difficulty understanding the questions, we agree that adding “other pay TV services” to the listed companies could clarify the question in upcoming CPS supplements, and we will explore this possibility in future surveys.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rafi Goldberg |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |