Computer and Internet Use Attachment B-2
Cognitive Testing Round 1 Findings
1/27/2017
General Testing Comments and Respondent Characteristics
Respondent Geography: Respondents from Round 1 resided in the following states: Maryland (5 respondents), Washington, DC (6 respondents), and Pennsylvania (3 respondents).
Participants: CSM intended to recruit 4 paired interviews from unrelated households, and 4 paired interviews from related households (N=16 total). During data collection for Round 1, one participant canceled, and another participant failed to attend their scheduled interview during the final days of data collection (there will be 2 additional interviews for Round 2 to make up for these cancellations). Therefore, the sample consists of 2 single interviews, 3 related household pairs, and 3 unrelated household pairs. CSM intends to recruit more participants of Hispanic origin for Round 2 since we could not recruit any during Round 1.
Roster Questionnaire: Two participants from an unrelated household were interviewed immediately after the Christmas and New Year holidays. Both participants are employed in the education sector, working in a school in Washington, DC. The Christmas break period had just ended. During the interview, they were asked if they worked last week for pay or for profit from the roster questionnaire. They were inclined to say ‘’no’’ because they were off on leave for Christmas break during the reference period. These participants did not interpret this question to mean if they were employed last week for pay or for profit.
Sensitivity of Questions (Q43-Q46): While interviewing related and unrelated households, participants indicated that some respondents may be uncomfortable answering questions that discuss one’s health, such as medical monitoring devices, researching health information online, and if they can access their health insurance or health insurance records online. Participants also expressed the thought that some people may feel uncomfortable providing responses to questions on cyberbullying if they have experienced bullying or harassment. Because this supplement discusses technology and Internet use, respondents also felt some people may think their Internet privacy and security concerns are a personal matter, and may not be forthcoming while discussing questions about it.
Use of Proxies: Members of unrelated households often mentioned that the medical questions are difficult to answer on behalf of someone else because they are not family members, but close friends living together or roommates. They felt that family members are more likely to share this type of information with one another. The data from Round 1, discussed in the Question Testing Results section of this report, also revealed that proxies do not provide the most accurate information about other people living in their household because their answer is either an assumption, or what the proxy believes is characteristic of the intended respondent’s personality and not factual.
Changes in Response Reporting Format: Respondents were not always aware when the question transitioned from inquiring about everyone in the household who has used Internet-connected devices at varying locations, to learning about the general availability of Internet in their household (Q18) and Internet privacy and security concerns (Q48).
Table 1: Select Demographic Characteristics of All Respondents*
|
Round 1 |
|||
|
Related (n=7) |
Unrelated (n=7) |
Total (n=14) |
|
Age (mean) |
36 |
41 |
39 |
|
Sex |
Male |
2 |
1 |
3 |
Female |
5 |
6 |
11 |
|
Race** |
White |
4 |
1 |
5 |
Black or African American |
4 |
6 |
10 |
|
American Indian or Alaska Native |
1 |
0 |
1 |
|
Asian |
0 |
1 |
1 |
|
Native Hawaiian or other Pacific Islander |
0 |
0 |
0 |
|
Ethnicity |
Hispanic |
0 |
0 |
0 |
Non-Hispanic |
7 |
7 |
14 |
|
Laptop |
Laptop |
4 |
4 |
8 |
Desktop |
0 |
0 |
0 |
|
Neither |
3 |
3 |
6 |
|
Smart Phone |
Yes |
2 |
7 |
9 |
No |
2 |
3 |
5 |
|
Internet Frequency |
Several times a day |
4 |
3 |
7 |
About once a day |
0 |
2 |
2 |
|
A few times a week |
2 |
2 |
4 |
|
A few times a month |
0 |
1 |
1 |
|
* Data in this table comes from both the survey screener and household roster.
**The race total exceeds the sample size because respondents were allowed to report multiple races. One respondent identified as Black & White, one respondent identified as Black & Native American, and one respondent identified as Black and Asian.
Questionnaire Testing Results
Question 2 - DESKTP
Original Wording
2. [Do you/Does anyone in this household, including you,] use a desktop computer?
(If needed) A desktop is a personal computer that is too large or heavy to be frequently moved from place to place.
Yes1
No
(If yes & is multi-person household) Who is that?
Findings
During Round 1 of testing, eight of the 14 respondents stated that a desktop computer is used by someone in their household. Out of the six paired interviews, only 1 pair disagreed about their answers to this question. The individual that said “yes” to this question mentioned during the interview that she is aware her roommate goes to the library and uses their computers. Her assumption is that the computers in the library are desktop computers. Overall, respondents did not display any issues with comprehension while answering this question.
Suggested Round 2 Wording
Same as Round 1.
Question 3 - LAPTOP
Original Wording
3. What about a laptop or notebook? [Do you/Does anyone in this household] use a laptop or notebook computer?
(If yes & is multi-person household) Who is that?
Findings
When asked if anyone in their household uses a laptop or notebook, 10 respondents said ‘’yes’’, and four respondents stated ‘’no’’. All six sets of interview pairs were in agreement about a laptop being used by someone in their household. The majority of respondents did not display any comprehension issues with this question. However, one respondent owned a netbook, but asked the interviewer what the difference is between a laptop and a notebook. Despite not knowing the formal definition of, or difference between a laptop or notebook, the respondent was able to accurately include their device under this question.
Suggested Round 2 Wording
Same as Round 1.
Suggested New Probes for Round 2
I asked does anyone in this household use a laptop or notebook computer, what does “notebook” mean in this question?
Question 4 - TABLET
Original Wording
4. What about a tablet or e-book reader, such as an iPad or Kindle?
(If needed) [Do you/Does anyone in this household] use a tablet or e-book reader?
(If yes & is multi-person household) Who is that?
Findings
Out of the 14 participants interviewed during Round 1, seven respondents said that someone in their household uses a tablet or e-book reader, and seven respondents stated that no one in their household uses a tablet or e-book reader. When asked about the types of tablets they have in their household, respondents listed a Google Nexus, iPad, and Kindle. Respondents did not exhibit any comprehension issues with this question.
Out of the three interview pairs from a related household, two sets of interview pairs presented conflicting information about a tablet being used. The first related pair that gave two different responses for the same question, one respondent suggested that her brother uses a tablet, the other did not list anyone in the household as having a tablet. They were identified as a related pair because they are sisters-in-law. The second related pair that provided conflicting responses for the same question were unmarried partners of the opposite sex. The girlfriend stated that no one in the household uses a tablet, however, her boyfriend indicated during the interview that he has an iPad, and both he and his girlfriend use it.
Suggested Round 2 Wording
Same as Round 1.
Question 5 - MPHONE
Original Wording
5. What about a smartphone, or a cell phone that connects to the Internet?
(If needed) [Do you/Does anyone in this household] use a smartphone?
(If yes & is multi-person household) Who is that?
Findings
A total of 14 respondents answered this question during Round 1. While the usage of smartphones were very popular amongst the households we chose, there was one respondent who selected ‘’no’’. This respondent was identified as not being technology savvy or familiar with technological terms, and may not have been aware that there are others living in his household who own, and/or use cell phones that can connect to the Internet. Although he was not familiar with many of the questions being asked during testing, he did not display any visible expressions of confusion or comprehension with this specific question.
Suggested Round 2 Wording
Same as Round 1.
Question 6 - WEARAB
Original Wording
[Do you/Does anyone in this household] use a wearable device that is connected to the Internet, such as a smart watch or fitness band? Examples include an Apple Watch, Fitbit, or Microsoft Band.
(If yes & is multi-person household) Who is that?
Findings
During Round 1, 11 respondents reported no one in their household uses a wearable device that can connect to the Internet. The remaining three respondents did report that someone in their household uses a wearable device. Only one related interview pair offered conflicting responses for this question about the presence of wearable Internet-connected devices in their household. The first person interviewed from the pair said that no one in their home has a wearable device. However, when interviewing the second person, she indicated that she owns a Fitbit.
Respondents were probed to provide examples of the wearable devices they thought about when they answered this question. Out of the 14 respondents, 12 mentioned an Apple Watch or a similar smart watch. Seven respondents thought of a Fitbit or a similar fitness device when they answered this question. Three respondents mentioned heart rate monitors. Other uncommon examples were Google Glasses, a blood pressure monitor, and Amazon Alexa. Two of the 14 respondents were unsure if their cell phones could be included in this question because they “wear” them often. Although they provided this as an example of a wearable device, they did not include them when they answered this question.
Suggested Round 2 Wording
Same as Round 1.
Suggested New Probes for Round 2
IF YES: Did you include your smartphone or a cell phone that connects to the Internet when you answered this question?
Question 7 - TVBOX
Original Wording
[Do you/Does anyone in this household] use a smart TV, a game or video system, or another device that connects to the Internet and plays through a TV? Examples include an Xbox, Apple TV, PlayStation, Roku, or a Blu-Ray player that can access the Internet.
(If yes & is multi-person household) Who is that?
Findings
During Round 1, 10 respondents said they have a device that connects to the Internet and plays through a TV in their home. Four respondents indicated that a device of this type does not exist in their home. When probed for the various devices that apply to this question, the various examples provided consisted of gaming systems, such as an Xbox or Playstation, Smart TVs, Amazon Firestick, and Roku. One respondent mentioned that their computer also connects to the Internet and plays through a TV. Only one respondent was slightly confused by this question. While this respondent did mention an Xbox, he also mentioned Netflix and Crackle which are companies that provide streaming media.
Suggested Round 2 Wording
Same as Round 1.
Question 9 - INHOME
Original Wording
[Do you/Does anyone in this household, including you,] use the Internet at home?
(If yes & is multi-person household) Who is that?
Findings
Two respondents interviewed during the first round of testing stated that no one uses the Internet at their home. Of these two, one explicitly stated that this is because they do not have Internet at home. However, during retrospective probing, this same respondent changed her response. She recalled that the other person residing in their household uses the Internet on their own smart phone, but mostly goes to the library to access the Internet.
Suggested Round 2 Wording
[Do you/Does anyone in this household, including you,] use the Internet at home? This includes accessing the Internet with a cell phone, computer, tablet or other device.
(If yes & is multi-person household) Who is that?
Suggested New Probes for Round 2
How do you connect to the internet at home?
Question 10 - INWORK
Original Wording
10. What about at work? [Do you/Does anyone in this household] use the Internet at work?
(If yes & is multi-person household) Who is that?
Findings
Respondents did not report or demonstrate any comprehension issues with this question. Several respondents were currently unemployed so this question was not applicable to them.
Suggested Round 2 Wording
Same as Round 1.
Question 11 - INSCHL
Original Wording
11. What about at school? [Do you/Does anyone in this household] use the Internet at school? Please count students only, and do not include online classes.
(If yes & is multi-person household) Who is that?
Findings
During Round 1 of testing, only one related household interview pair had adolescent children in their home currently enrolled in school. There were also a few respondents currently continuing their post secondary education. Of these respondents, no one exhibited any comprehension issues with this question.
Suggested Round 2 Wording
Same as Round 1.
Question 12 - INCAFE
Original Wording
12. What about at a coffee shop or other business that offers Internet access?
(If needed) [Do you/Does anyone in this household] use the Internet while at a coffee shop or other business that offers Internet access?
(If yes & is multi-person household) Who is that?
Findings
During Round 1, 10 respondents stated that someone in their household uses the Internet while at a coffee shop or another business that offers Internet access. Four respondents stated that no one in their household uses the Internet while at a coffee shop. Of the four respondents who answered ‘’no’’, one respondent changed her mind as she thought more about the question during probing. She felt that to a degree, everyone uses the Internet from their phone while at a café. When asked about examples of other businesses, they stated Starbucks, McDonalds, Target. One respondent gave the library as an example even though it is not considered a business. A different respondent stated she was slightly confused by the term ‘’business,’’ because the Internet can also be used at the library.
Suggested Round 2 Wording
Same as Round 1.
Question 13 - INTRAV
Original Wording
13. What about while commuting or traveling from one place to another?
(If needed) [Do you/Does anyone in this household] use the Internet while commuting or traveling?
(If yes & is multi-person household) Who is that?
Findings
This question was commonly misinterpreted by respondents during Round 1. When asked about the use of Internet while commuting or traveling from one place to another, six respondents said someone in their household does this, whereas eight respondents said that no one does this. Two sets of interview pairs (related and unrelated) reported opposing information for this question. For the related pair, the wife mentioned being able to connect to the Internet from a mobile device so she answered ‘’yes’’, but her husband understood this to be traveling while on business so he said ‘’no’’. For the unrelated pair, the male roommate said no one in their home does this, but the female roommate exclaimed that they both use the Internet to look up information while going somewhere.
Respondents were asked during probing to tell the interviewer what this question is asking. Two respondents thought this question was limited to using the Internet while on work or business travel. Most respondents believed that this question was inclusive of using GPS to get to a destination, or looking up information on the Internet about a specific business. Respondents also included surfing the Internet while on public transit until they arrived to their destination.
Suggested Round 2 Wording
13. What about while commuting or traveling from one place to another? Examples include using the Internet while riding public transit, or using the GPS on your cell phone while driving.
(If needed) [Do you/Does anyone in this household] use the Internet while commuting or traveling?
(If yes & is multi-person household) Who is that?
Suggested New Probes for Round 2
Does anyone in your household use a GPS while commuting or traveling?
Question 14 - INLICO
Original Wording
14. What about at a library, community center, park, or other public place?
(If needed) [Do you/Does anyone in this household] use the Internet at a library, community center, park, or other public place?
(If yes & is multi-person household) Who is that?
Findings
Respondents did not report any confusion or comprehension issues with this question. Ten respondents mentioned that someone in their household uses the Internet at a library, community center, or other public space. Of the respondents that answered ‘’yes’’ to this question, libraries and community centers were the public places they have used the Internet at.
Two sets of related interview pairs provided conflicting information about whether or not people in their household use the Internet at a public place. For the first related pair, a respondent mentioned she uses the Internet at a library or other public space, the other person interviewed from her home did not include this. For the second related pair, one respondent thought about her son who uses the Internet at a library, whereas her boyfriend did not.
Suggested Round 2 Wording
Same as Round 1.
Question 15 - INELHO
Original Wording
15. What about at someone else’s home?
(If needed) [Do you/Does anyone in this household] use the Internet at someone else’s home?
(If yes & is multi-person household) Who is that?
Findings
Many of the respondents selected for testing commonly use the Internet while they are at someone else’s home. One respondent argued that while at someone else’s home, if another individual pulls something up on the computer from the Internet, she may look at it. She considered this to still be ‘’using’’ the Internet even though she was not the person operating the computer or medium used to connect to the Internet. No respondents reported any confusion or comprehension issues with this question.
Suggested Round 2 Wording
Same as Round 1.
Question 16 - INOTHR
Original Wording
16. [Do you/Does anyone in this household] use the Internet at some other location we haven’t covered?
(If yes & is multi-person household) Who is that?
Findings
Respondents did not report or demonstrate any comprehension issues with this question. The majority of respondents felt like most locations were covered under the other locations previously asked. However, there were a couple of respondents who suggested that the Internet could be used at other relatives’ homes, Target, and their church.
Suggested Round 2 Wording
Same as Round 1.
Question 17 - NETCHK
Original Wording
17. Based on your responses, it sounds like (list names where all IN* != 1) [does/do] not use the Internet. [Does/Do] [he/she/these people in your household] use the Internet from any location, and for any purpose?
(If additional users) Who uses the Internet from any location, and for any purpose?
Findings
Item 17 was skipped for all 14 respondents because they conveyed throughout the questionnaire that at least one other individual in their household uses the Internet at home, at a public space, or both.
Suggested Round 2 Wording
Same as Round 1.
Question 18 - MOBDAT
Original Wording
18. [Do you/Does anyone in this household] access the Internet using a mobile Internet service or a data plan for a cell phone, smartphone, tablet, mobile hotspot, or other device?
(If needed) This type of Internet service is provided by a wireless carrier, and may be part of a package that also includes voice calls from a cell phone or smartphone.
Findings
Item 18 is where the reporting format changed in the questionnaire from asking who specifically uses Internet-connected devices, to asking about general household use. We recommend that a transition sentence or paragraph be added so that respondents know they don’t have to specify who in their home has a mobile data plan. All participants interviewed with the exception of one revealed that there is at least one person in their household who has access to the Internet through a mobile data plan. The individual who said ‘’no’’ during the administration of the survey did reveal during probing for a different question that she believes the other person living in her home may have a mobile data plan, but she is not completely confident.
Respondents were then asked what ‘’mobile Internet service’’ means to them. The varying responses to this question revealed that three respondents were unclear regarding the intent of this question. One respondent understood the question to be asking if anyone has the capability to use their cell phone service to make calls. The two other respondents interpreted the question to mean if you can use your device while being ‘’mobile’’ such as, while in transit or commute.
Respondents were also probed about the type of mobile Internet service they have. Nearly all provided examples of the data packages they have with their cell phone providers. A few respondents also revealed having mobile hotspot capability through their data plans. Respondents were typically confident in their knowledge of other household members’ access to mobile Internet service. If they were not on the same data plans together, respondents have seen the others use the mobile Internet service on various devices.
Suggested Round 2 Wording
18. [Do you/Does anyone in this household] access the Internet using a data plan for a cell phone, smartphone, tablet, mobile hotspot, or other device?
This type of Internet service is provided by a wireless carrier, and may be part of a package that also includes voice calls from a cell phone or smartphone.
Question 21 - HOMTE
Original Wording
21. I am going to read a list of ways that people access the Internet from their homes, other than a mobile data plan. At home, [do you/does anyone in this household] access the Internet using:
1. High-speed Internet service installed at home, such as cable, DSL, or fiber-optic service? (If needed) This type of Internet service is often provided by a cable company or phone company.
2. Satellite Internet service?
3. Dial-up service?
4. Some other service?
Findings
Only 11 respondents answered this question because the other three did not have Internet installed in their homes. For the respondents that did have a home Internet connection, all of them received their Internet through cable, DSL or fiber optic service. None of these respondents currently had satellite, dial-up, or some other service.
Respondents were able to articulate in their own words the type of Internet service they have in their homes. However, when they were probed on the difference between high speed and satellite Internet service, several respondents struggled to do this. One respondent interpreted satellite service to be through the radio. Two respondents from unrelated households explicitly stated that they did not know the difference between satellite and high-speed Internet service. One of the respondents who expressed not understanding the difference between the two options said that as a person leasing a room in her friend’s home, they just have Wi-Fi. She is not responsible for paying the Internet bill.
Respondents were not asked Q22 since no one selected ‘’some other service’’ in Q21.
Suggested Round 2 Wording
21. I am going to read a list of ways that people access the Internet from their homes, other than a mobile data plan. At home, [do you/does anyone in this household] access the Internet using:
1. High-speed Internet service installed at home, such as cable, DSL, or fiber-optic service? (If needed) This type of Internet service is often provided by a cable company or phone company.
2. Satellite Internet service? This type of Internet connection is received through a satellite dish.
3. Dial-up service?
4. Some other service?
Question 23 - HOMSU
Original Wording
Thinking about the company or other organization that provides your home Internet service, who provides your Internet service?
1. A company that sells Internet service, such as a cable or phone company.
2. A public agency, nonprofit, or cooperative that sells Internet service.
3. Internet service is provided for the entire apartment building, condominium, campus, or other community and included in housing costs.
4. Internet service is publicly available and provided at no charge.
5. Internet service is provided through some other method.
Findings
Eleven respondents identified their home Internet as provided through a company that sells Internet service such as a cable or phone company. One respondent who rents a room in her friend’s home also said ‘’yes’’ to response option 3: ‘’Internet service is provided for the entire apartment building, condominium, campus, or other community and included in housing costs”. The leasing agreement includes Internet because all utilities are covered in the lease so she is not responsible for paying for it.
During probing, respondents were asked to provide examples of a public agency, nonprofit, or cooperative that sells Internet service. Eight respondents said they could not give examples of an Internet service provider such as this. Three respondents stated they think the Internet and cable provider, Residential Communications Network (RCN) is an example, although, one of three were not confident that this example is correct. NTIA, is RCN cable service an example of option 2? What would be an example of this?
Suggested Round 2 Wording
Thinking about the company or organization that provides your home Internet service, who provides your Internet service?
1. Internet service is provided by A company that sells Internet service, such as a cable or phone company.
2. Internet service is provided by A public agency, nonprofit, or cooperative that sells Internet service. Include example/definition.
3. Internet service is provided for the entire apartment building, condominium, campus, or other community and included in housing costs.
4. Internet service is publicly available and provided at no charge.
5. Internet service is provided through some other method.
Suggested New Probes for Round 2
Can you provide an example of Internet service that is publicly available and provided at no charge?
Question 24 - PECOMP
Original Wording
24. To the best of your knowledge, are there other companies or organizations where you live that would offer high-speed Internet service installed in your home, such as cable, DSL, or fiber-optic service?
(If needed) This type of Internet service is often provided by a cable company or phone company.
Findings
Respondents did not report or exhibit any comprehension issues with this question. Some were aware of the presence of alternative Internet providers and others were not. Eight respondents stated that there are other companies where they live that could provide Internet services. Three respondents said that there are not other options. The three respondents who stated no additional options exist, reported different answers for this question than the other person interviewed from their household.
Suggested Round 2 Wording
Same as Round 1.
Question 26 - EMAIL
Original Wording
26. [Do you/Does NAME] use email?
Findings
The use of email is common amongst the participants selects for Round 1. Only one person indicated that she does not use email. Overall, respondents did not exhibit any confusion when asked this question. Respondents were also able to make accurate assessments about the use of email by the other member interviewed from their household, regardless if the pair were unrelated or related.
Suggested Round 2 Wording
Same as Round 1.
Question 27 - TEXTIM
Original Wording
27. What about texting or instant messaging? [Do you/Does NAME] use a texting or instant messaging service?
Findings
All 14 respondents during Round 1 use a texting or instant messaging service. Participants were asked to tell their interviewer in their own words what this question is asking. One respondent was particularly troubled by the phrase ‘’texting or instant messaging service’’. She stated this language suggests that a separate service offers the ability to text message; however, this is a part of her phone’s capability. The other respondents understood this question to ask if they use text messaging or imessaging through their phone as a way to communicate with other people.
Respondents were also probed if they use the Internet to send text messages, instant messages or both. Many respondents said that they used both. One respondent was unclear on what instant messaging is and how it works. Despite some issues with comprehension, all respondents correctly identified the use of texting and instant messaging services by the other participant interviewed from their household.
Suggested Round 2 Wording
27. What about texting or instant messaging? [Do you/Does NAME] use texting or instant messaging?
Question 28 - SOCIAL
Original Wording
28. What about using social networks, such as Facebook, Twitter, Snapchat, or Instagram?
(If needed) [Do you/Does NAME] use social networks?
Findings
All respondents were confident in their ability to understand and answer this question. No participants from Round 1 exhibited any confusion or difficulty. One respondent did not know if her roommate (goddaughter) uses social media or not. Another respondent initially stated her husband uses social networks, but after thinking through the question she changed her answer to ‘’no’’ because he only logs on if she asks him to check something. Responses were consistent for both the intended respondent and their proxy for this question.
Suggested Round 2 Wording
Same as Round 1.
Question 29 - CONFER
Original Wording
29. In the past six months, [have you/has NAME] participated in video or voice calls or conferencing over the Internet, such as with Skype or FaceTime?
(If needed) [Do you/Does NAME] participate in video or voice calls or conferencing?
Findings
Respondents did not exhibit any comprehension issues with this question. When asked during probing about the types of things they thought about when answering this question, respondents gave examples of using Facetime, Skype, Oovoo, Google Hangouts, and video calls. One respondent is currently in school and mentioned that her program uses a videoconferencing service for classes. Only 3 of the 6 pairs agreed on the answer to this question. For two of the pairs that disagreed, this was because they were unaware that at least one person in their household uses videoconferencing. For the remaining pair that disagreed, this was because the older respondent was not aware of videoconferencing programs, whereas the younger respondent was because she has used both Oovoo and Facetime.
Suggested Round 2 Wording
Same as Round 1.
Question 30 - VIDEO
Original Wording
30. What about streaming or watching videos? [Do you/Does NAME] watch videos over the Internet? Examples include YouTube and Netflix.
Findings
Respondents did not display any comprehension issues with this question. For those that reported watching or streaming videos, this was often watched through Netflix, YouTube, and Facebook. One pair disagreed during the administration of the questionnaire if someone else in their home streams or watches videos. However, during probing one person changed her answer, acknowledging that she does watch Netflix with her boyfriend who reported this same information.
Suggested Round 2 Wording
Same as Round 1.
Question 31 - AUDIO
Original Wording
31. What about streaming or downloading music, radio programs, or podcasts?
(If needed) [Do you/Does NAME] stream or download music, radio programs, or podcasts?
Findings
All respondents were clear on what this question is asking. However, one respondent was not sure how to classify music that she streams online through YouTube. YouTube is classified as a video sharing website. Alternatively, there are users who upload musical content in the form of a video file because that is the only file format the website accepts. NTIA: Do we want respondents to disregard music, podcast recordings, and other audio content they access through YouTube?
Respondents said they were knowledgeable about the others in their home because often times they have seen someone else listen to music, or they have done these things together such as streaming Pandora. Three of 12 respondents disagreed about who streams or downloads music, radio programs, or podcasts in their household. Two respondents said they do not know if the other person interviewed from their household streams music, and one respondent said they do not stream music but their pair reported they did.
Suggested Round 2 Wording
Same as Round 1.
Question 32 - PUBLSH
Original Wording
32. What about posting or uploading blog posts, original videos, or other content that [you/NAME] created?
(If needed) [Do you/Does NAME] publish or upload original content that [you/NAME] created to the Internet?
Findings
This item was very problematic for respondents during testing. Respondents were probed to elaborate more on what they were thinking about when they answered this question. The variation in responses reflect that the original wording was unclear to many respondents. Some respondents did not realize this question goes beyond taking a personal photo or recording a personal video and uploading it to their social media pages.
Two respondents provided their own examples of item 32 which were accurate. One respondent is a painter and thought of her original content she uploads to social media for advertisement. Another respondent said she also thought of posters and advertisements, even though she does not do this herself. Three respondents thought about creating and uploading blog posts, which is listed as an example in the original question.
Suggested Round 2 Wording
32. What about posting or uploading blog posts, original videos, or other content that [you/NAME] created? Exclude pictures and videos that you take and record for personal use and post to social media.
(If needed) [Do you/Does NAME] publish or upload original content that [you/NAME] created to the Internet?
Question 33- TELEWK
Original Wording
33. What about telecommuting, or working while away from [your/his/her] usual workplace?
(If needed) [Do you/Does NAME] use the Internet to telecommute or work while away from [your/his/her] usual workplace?
Findings
The majority of respondents did not struggle to answer the question about telecommuting. Two respondents did require the if needed statements be read so they could better understand the meaning of the question. After reviewing the proxy data, many respondents were not aware that others living in their household worked from home at times. This is because working from home did not happen formally on a pre-scheduled date, but simply to complete a few unfinished tasks from work.
Suggested Round 2 Wording
33. [Do you/Does NAME] use the Internet to telecommute or work while away from [your/his/her] usual workplace?
Question 34 - JOBSCH
Original Wording
34. In the past six months, [have you/has NAME] used the Internet to search or apply for a job?
(If needed) [Do you/Does NAME] use the Internet to search or apply for a job?
Findings
This question was very easy for respondents to interpret. No one exhibited any issues with comprehension. However, one respondent was unsure if he could include applying for graduate school acceptance here. Only one interview pair was able to accurately report about each other’s job search and application via the Internet. They were aware that both had recently applied for jobs within the reference period. For the other 5 interview pairs, one person would report that they had been searching for jobs online but the other person interviewed from their household did not report this.
Suggested Round 2 Wording
Same as Round 1.
Question 35 - EDTRAI
Original Wording
35. What about online classes or job training?
(If needed) [Do you/Does NAME] use the Internet for educational classes or job training?
Findings
Overall, respondents felt this question was straightforward and easy to understand. One respondent initially said she does not take any online classes or job training. The interviewer recalled that the participant had previously mentioned she takes a lot of online training for work, and questioned her more about this. The respondent stated that when she first heard this question, she thought it was asking if she was enrolled in an online education program.
Suggested Round 2 Wording
Same as Round 1.
Question 36 - USESVC
Original Wording
36. What about requesting services provided by other people, for example hailing a ride using Uber or Lyft, or reserving a room through Airbnb?
(If needed) [Do you/Does NAME] use the Internet to request services provided by other people?
Findings
During Round 1 of testing, four respondents revealed they do not request services by other people using the Internet, and ten stated they do. Out of the 14 total respondents, two displayed issues comprehending this question. The first respondent is not technologically or computer savvy. He owns a cell phone but it is not a smart phone. When he was probed on what this question was asking, he said this question asks if he gets Internet through his neighbor, such as using their hot spot or Internet connection. For the second respondent who was confused, after probing she asserted that this question asks if she fills out any applications online or downloads documents. Interviewers tried to probe both respondents further to get a greater understanding of what misdirected their thinking, but it was unclear.
During questionnaire administration, most respondents said they have used Uber or Lyft within the past six months. Responses from Round 1 did not differ by those living in urban or rural areas, but the examples provided in the question may not be applicable to those of lower education and income. More testing is needed for this question. Two sets of paired interviews did not accurately report information about the use of services requested by someone else living in their household.
Suggested Round 2 Wording
Same as Round 1.
Question 37 - ESRVCS
Original Wording
37. What about offering [your/their] own services for sale via the Internet? Examples include offering rentals on Airbnb and driving for Uber or Lyft.
(If needed) [Do you/Does NAME] offer your own services for sale via the Internet?
Findings
Some respondents confused the difference between selling a good and service. While the examples provided were very helpful in aiding respondents, some still considered goods and services to be very similar, if not the same. CSM staff identified three respondents who struggled with this question because they provided examples of selling a Christmas card, or selling clothing or crafts on websites like eBay and etsy. The most common examples provided across all participants was working as a driver for Uber or Lyft.
When asked this same question about the other participant being interviewed from their house, three respondents disagreed in their responses.
Suggested Round 2 Wording
37. What about offering [your/their] own services for sale via the Internet? Examples include offering rentals on Airbnb and driving for Uber or Lyft. Do not include any goods or possessions you sell online, such as, clothing, shoes, or crafts on Etsy, Ebay or Craigslist.
(If needed) [Do you/Does NAME] offer [your/their] own services for sale via the Internet?
Question 38 - ECOMME
Original Wording
38. In the past six months, [have you/has NAME] used the Internet for online shopping, travel reservations, or other consumer services on the Internet?
(If needed) [Do you/Does NAME] use the Internet for online shopping, travel reservations, or other consumer services?
Findings
Nearly all respondents used the Internet for at least one type of consumer service and could comprehend this question. Only one participant stated that he does not do any of this. During probing, respondents were asked to convey in their own words what this question is asking. They stated this question is asking about activities such as online shopping, booking hotel reservations, cruises, flight tickets, and bus tickets. However, one respondent said this question is asking if she uses the Internet while traveling and not at home. Respondents were also probed on their interpretations of the words ‘’consumer services’’. Some of the definitions provided included things that you are willing to buy, getting services cheaper online, eating something, and requesting a service.
Suggested Round 2 Wording
Same as Round 1.
Question 39 - EGOODS
Original Wording
39. [Do you/Does NAME] use the Internet to sell goods? Examples include selling items on Etsy or eBay.
Findings
During the administration of the questionnaire, the majority of our participants indicated that they do not sell goods. Only three respondents answered ‘’yes’’ to this question. During probing, it was evident that some respondents were confusing selling goods with selling services as previously mentioned. One respondent even said that she thought she had already answered ‘’no’’ to this question, when it was item 37 that she was thinking of. As mentioned previously, CSM recommends that Q39 be asked following Q37.
Interviewers probed respondents to elaborate more on how they came up with their answers to this question. Most respondents knew they are not selling anything, which made it easy for them to answer this question. Other respondents discussed clothing items they are currently trying to sell on the Internet, whereas others mentioned that this question is asking about items sold for profit. When asked about the other individual interviewed from their household for the study, four respondents provided incorrect answers for the person they served as a proxy for.
Suggested Round 2 Wording
Same as Round 1.
Question 40 - FINANC
Original Wording
40. [Do you/Does NAME] use the Internet for financial services such as banking, investing, or paying bills online?
Findings
Respondents did not report any confusion or comprehension issues with this question. Using the Internet for financial services is commonly done by 11 of the 14 participants we interviewed during Round 1 testing. Respondents were not probed on this question because CSM did not think this question would be difficult for them to comprehend or answer. Two sets of paired interviews disagreed about the use of financial services by the other person interviewed from their household.
Suggested Round 2 Wording
Same as Round 1.
Question 41 - HOMIOT
Original Wording
41. What about interacting with household equipment or appliances that are connected to the Internet, such as a connected thermostat, light bulb, or security system?
(If needed) [Do you/Does NAME] use the Internet to interact with household equipment or appliances?
Findings
Although Internet-connected household equipment and appliances are growing in popularity, only two respondents use them. During probing, respondents were asked about the types of appliances or equipment that came to mind when they answered this question. Respondents provided a variety of examples, including alarm systems, thermostats, stoves, refrigerators, and lighting. One respondent could not give an example of this because she was not familiar with this capability. A different respondent gave examples of an Amazon Firestick and a DVD player. Other than these two exceptions, respondents were able to comprehend what was being asked in this question. CSM does not recommend any changes for Round 2.
Suggested Round 2 Wording
Same as Round 1.
Question 43 - MEDINF
Original Wording
43. [Do you/Does NAME] research health information online, such as with WebMD or similar services?
Findings
Respondents did not demonstrate or report any comprehension issues with this question. Ten of the 14 respondents report using the Internet to research health information. Although respondents were not probed about this question, participants did reveal that they were not confident in their answers for other household members because they did not witness other household members doing this. Several respondents did however assume that most people would research health information online.
Suggested Round 2 Wording
Same as Round 1.
Suggested New Probes for Round 2
How do people in your household research health information online?
Question 44 - MEDDOC
Original Wording
44. What about communicating with a doctor or other health professional using the Internet?
(If needed) [Do you/Does NAME] communicate with a doctor or other health professional using the Internet?
Findings
Communicating with a health professional using the Internet was a very uncommon practice for the participants we interviewed. Only two of the fourteen respondents use this type of communication. One respondent works as a nurse and communicates with doctors as a part of her job responsibilities. Respondents were probed to describe for the interviewer what this question is asking. Some answers respondents gave included logging into an online portal to communicate with your health professional, contacting your doctor online with any questions you may have, as well as viewing test results and health records online. When respondents were asked to answer this question about the other person interviewed from their household, they revealed this was difficult to do if they are not related to one another. They stated that medical information is personal and is more likely to be discussed amongst family members.
Suggested Round 2 Wording
Same as Round 1.
Question 45 - MEDREC
Original Wording
45. What about accessing health records or health insurance records online?
(If needed) [Do you/Does NAME] access health records or health insurance records online?
Findings
The responses for this question were almost evenly split in half. Six respondents do not use the Internet to access their health records or health insurance, whereas eight respondents do. CSM probed respondents to describe what ‘’access’’ means to them in this question. Some of the answers they provided included being able to go online, to gain entry, and to view your records. One respondent did not thoroughly define the term access in the context of the question asked. She stated access means to ‘’open up and to come through’’, but she did not relate this back to health records.
It was also very difficult for participants to serve as proxies for this question. Respondents living in unrelated households explicitly stated that it was difficult to answer this question on behalf of someone else, implying that it could be easier to do if they were related. The data revealed that this was difficult to answer for related households as well.
Suggested Round 2 Wording
We don’t recommend revising the wording, but asking Q45 prior to Q44 so respondents don’t think accessing health records is a form of researching health information on line.
Question 46 - MEDMON
Original Wording
46. [Do you/Does NAME] use an electronic health monitoring service that collects and sends data to [you/his/her] doctor or health care provider through the Internet? Examples include connected devices that monitor vital statistics, blood glucose levels, or blood pressure.
Findings
Only one respondent revealed during testing that they use an electronic monitoring service. When asked to describe this service, she mentioned Verisign. Verisign is a program that allows a doctor to provide a signature to automatically fulfill your prescriptions within a 1 hour timeframe. NTIA: I believe you all were interested in devices only. Is this correct? Or, does the respondent’s scenario apply?
Respondents did not demonstrate any comprehension issues with this question. All respondents were probed to provide examples of an electronic health monitoring service. Most respondents thought of devices such as heart monitors. Others also mentioned pacemakers, and devices that can monitor diabetes and dialysis. Two respondents also discussed the use of Fitbits and smart watches, which monitor your heart rate. There were three respondents who could not think of an example. NTIA, do you want respondents to include Fitbits and smart watches here?
Suggested Round 2 Wording
Same as Round 1.
Question 48 - PSPRE
Original Wording
48. During the past year, have concerns about privacy or security stopped [you/anyone in this household] from doing any of these activities online:
Conducting financial transactions such as banking, investing, or paying bills online?
Buying goods or services online?
Posting photos, status updates, or other information on social networks?
Expressing an opinion on a controversial or political issue on a blog or social network, or in a forum, email or any other venue?
Searching for information using a platform such as Google Search, Yahoo Search, Microsoft Bing, or another web search engine?
Findings
No respondents exhibited any confusion or comprehension issues with this question. However, two respondents did lose sight of the question when asked about buying goods and services, and searching for information online. The interviewer repeated the question stem so that their respondents could provide answers.
Respondents expressed that some members in their household have concerns when it comes to privacy and security online. Five respondents stated their concerns have stopped them from conducting financial transactions online, five respondents mentioned their concerns are with buying goods and services online, two respondents stated their concerns are with using social media, two respondents suggested their concerns are with expressing opinions publicly online, and two respondents stated their privacy concerns are with searching for information. Only one pair agreed on what the privacy and security concerns are for their household because they both indicated there were not any concerns. The response options selected varied for the other respondents.
Suggested Round 2 Wording
Same as Round 1.
Question 49 - PSCON
Original Wording
49. Overall, what concerns [you/members of this household] the most when it comes to online privacy and security risks?
Identity theft
Credit card or banking fraud
Data collection or tracking by online services
Data collection or tracking by government
Loss of control over personal data such as email or social network profiles
Threats to personal safety, such as online harassment, stalking, or cyber-bullying
No concerns
Other: ___________________________
Findings
When asked what concerns members of their household the most, respondents mentioned identity theft, banking fraud, email and social media hacking, and computer viruses. One participant provided a response that we believed applied to the ‘’other’’ category. It was clear that they were not answering on behalf of their household but themselves. What concerned this respondent the most is not wanting their job to view their social media activity for professional reasons. Six respondents said that these things do not concern the members of their household. Responses were only consistent for the three pairs that stated their household has no concerns.
Suggested Round 2 Wording
Same as Round 1.
Question 50 - PSCYBA
Original Wording
50. During the past year, [have you/has any member of your household] been affected by an online security breach, identity theft, or a similar crime?
Findings
No respondents had trouble comprehending this question. Eight respondents revealed that someone in their household has been affected during the past year by an online security breach, identity theft, or a similar crime. Respondents were probed during testing to define what “online security breach” means in the context of this question and to provide relevant examples. A variety of answers were given, including someone accessing your banking information, email hacking, identity theft, and having your personal information and identification compromised.
Respondents mostly answered this question about themselves, saying they have been affected by a breach. When interviewers asked if anyone else in their household was affected by security breaches, they indicated they do not believe so. Respondents were confident in the answers they provided for their other household members despite saying no one else in their home has experienced this. Three respondents from paired interviews disagreed with the other person from their household about security breaches and similar crimes. Two respondents acknowledged being affected personally, but their pair was unaware of this. For the third pair, a respondent mentioned her husband had an experience recently when she said ‘’yes’’, but her husband made no mention of this during his interview.
Suggested Round 2 Wording
Same as Round 1.
Question 51 - CBULLY
Original Wording
51. During the past year, [have you/has any member of your household] experienced online harassment, stalking, or cyber-bullying?
Findings
Respondents did not display any comprehension issues with this question. Most respondents did not have any experiences with online harassment or cyberbullying within the past year, and their interview pairs provided consistent answers. However, one respondent recalled an incident with her biological sister who was harassed on Facebook. This participant’s interview pair, her sister-in-law, did not mention this during their interview.
When respondents were probed on what they thought about when they answered this question, they mentioned people posting pictures and saying mean things, people stalking and calling you attempting to meet in person, bullying, and making threats. The examples respondents gave were mostly in the context of social media and online chat forums.
Suggested Round 2 Wording
Same as Round 1.
Question 54 EVRHOM – Question 61 PRINOO
Findings
Very few respondents were asked why no one in their household has used the Internet at a location at least occasionally within the past six months. The respondents who were asked these questions did not report or display any comprehension issues. No probes were written for these questions during Round 1 of cognitive testing. The wording for Round 2 will remain the same as Round 1. One respondent mentioned she does not use the Internet at home because they cannot afford Internet service, and the service for her smart phone is currently turned off. A different respondent stated they do not use the Internet at her household because they do not have a computer anymore. The computer they had previously was infected with a virus.
Question 63 - TRADTV
Original Wording
63. [Do you/Does anyone in this household] subscribe to a cable TV or satellite TV service? Do not include services accessed through the Internet.
Findings
The wording of this question was very problematic for two respondents because they could not comprehend what this question was asking. One respondent chose not to answer the question because she did not know what was being asked about her household. Another respondent said ‘’no’’ when she should have answered ‘’yes’’. The exclusionary clause confused them both because they were directed to not include services accessed through the Internet. Both respondents revealed during probing that they subscribe to a cable and Internet bundle, paying a set price to receive both services. Because their Internet is linked to their cable TV, respondents thought they were instructed to exclude them.
The remaining respondents correctly responded ‘’no’’ because they receive their TV services either through the Internet, Roku, or an Amazon Firestick.
Suggested Round 2 Wording
63. [Do you/Does anyone in this household] subscribe to cable TV or satellite TV? Do not include TV services streamed through Netflix, Sling TV, Hulu, Roku, Firestick, Apple TV or other Internet streaming devices or programs.
Suggested New Probes for Round 2
Please describe how you watch TV channels or TV services.
Question 64 - PREVTV
Original Wording
64. [Have you/has anyone in this household] ever subscribed to a cable TV or satellite TV service?
Findings
A total of 11 respondents answered item 64, the other three skipped this question because they currently subscribe to cable TV. Of the 11 respondents, nine indicated that someone in their household has subscribed to cable or satellite TV service before. Two respondents replied ‘’no’’ after being read this question. However, it was difficult to distinguish if they were still answering about the actions of everyone in their entire household, or themselves individually.
Suggested Round 2 Wording
64. Has this household ever subscribed to a cable TV or satellite TV service? Do not include TV services streamed through Netflix, Sling TV, Hulu, Roku, Firestick, Apple TV or other Internet streaming devices or programs.
Question 65 - NOTV
Original Wording
65. What are the reasons why [you/members of your household] do not subscribe to a service providing access to TV channels?
1. Using Internet-based video services instead
2. Don’t need it
3. Not interested
4. Can’t afford it
5. Not worth the cost
6. Can watch at another location
7. Can watch using an antenna
8. Not available in area
9. No television, or television inadequate or broken
10. Household moved or is in the process of moving
11. Previously experienced poor customer service
12. Previously experienced poor quality
13. Other: __________________
Findings
Eleven respondents did not exhibit any issues with this question. Many revealed that their reasons for discontinuing their TV service subscriptions through a cable or satellite company is because they cannot afford it, cable is not worth the cost, or because they use Internet-based video services instead. One respondent added that she thinks the quality of TV services streamed through the Internet is so great, this reduces the need or desire to pay for cable or satellite services.
Suggested Round 2 Wording
Same as Round 1.
1 CAPI response options are yes/no for Q2-Q18, Q24-Q48, and Q50-Q64.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Amber Henderson (CENSUS/CSM FED) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |