Attachment H - ATUS Web Development Summary

April 12, 2023

ATUS Web Collection Development Background:

The American Time Use Survey (ATUS) has been conducted solely via Computer Assisted Telephone Interviewing (CATI) since the survey began in 2003. ATUS would like to add a self-administered, web-based collection tool such that the ATUS would become a mixed-mode, CATI and web-based survey. Many surveys have implemented web collection tools and this option has proven to appeal to certain cohorts of respondents, especially young individuals and those who like the flexibility of responding on their own schedule.

In an effort to modernize ATUS data collection and offer an alternative mode of data collection to CATI, BLS has been developing a self-administered web collection tool. One concern with adding a self-administered response option to the ATUS is the respondent’s ability to report activities at the level of detail ATUS requires without the interventions of a trained interviewer. The presentation of the activity categories, including the activity names and structure of the lexicon, the tools used to identify activity categories, and the probes appearing within a self-administered ATUS can all influence what is recalled and how people report and self-record activities of the previous day.

BLS has completed the following work toward developing a self-administered web collection tool:

• Spring 2015 – Westat ATUS Web Feasibility Study

ATUS contracted with Westat to explore the feasibility of an online mode for ATUS collection. As part of the project, Westat provided the following: Literature review, Web design task, Comparability report, and a Web mock-up.

• Spring 2020 – Revision and Testing of ATUS Web Lexicon

BLS staff revised the current ATUS activity lexicon, developed for CATI, to be more usable for online collection. This involved streamlining many of the activity categories and cognitively testing the ATUS web lexicon. Using MTurk participants, BLS used navigation, or tree testing, to see how easily participants could categorize common time-use activities using the revised ATUS lexicon. The main questions tested:

• Do the categories make sense to people?

• Is the content grouped logically?

• Can participants find the category they want easily and quickly? If not, why not? What is distracting or misleading them?

• Spring 2021 – Westat Development of ATUS Web Diary Prototype

ATUS contracted with Westat to develop a prototype for an ATUS web diary. This included instructions, flow, and probes needed to collect activity data in a self-administered instrument. This also included a database that can be used by respondents using a search function to identify and select their activities. Westat developed a prototype with limited functionality.

• Fall 2021/Spring 2022 – NORC Web Diary Test

Using the prototype developed by Westat, ATUS contracted with NORC to collect self-administered and CATI diaries from research participants about their time use on an assigned day. Data were collected using CATI and web for each participant and then compared. The main questions tested:

• Are participants able to successfully report their activities using the web instrument (search tool, browse, and manual entry options)?

• How do their web diaries compare to those collected via CATI?

Findings: Overall, participants reported more activities in CATI than in web. Web activity count was greater when CATI was administered first than when web was administered first. This finding suggests that participants were “trained” in how to complete the diary with sufficient detail. The observed mode effect is likely due to two respondent behaviors – disaggregation of activities and inconsistent reporting of infrequent activities. The findings suggest that – without being told to disaggregate activities per CATI instructions – participants were likely combining multiple, smaller activities into fewer, larger activities. While common activities (i.e., sleeping and working) are reported relatively consistently between modes, other activities that are likely infrequently discussed or not especially memorable (e.g., grooming oneself or grooming or care of children) are underreported. Finally, most participants reported differences in activity durations between modes. The most pronounced difference occurred amongst participants who reported engaging in childcare activities. Only 21% of participants with children did not show any differences in reported activity durations. In contrast, 63% of employed participants did not report a difference in their work durations between the two modes. Additional research is needed to further investigate these findings.

• Fall 2021/Spring 2022 – NORC Web Eldercare Questions

ATUS contracted with NORC to develop/modify eldercare questions for online collection. The modified questions will inform how eldercare is collected in the final self-administered web collection tool.

• Fall 2022/Spring 2023 – Westat

  • BLS is funding additional work to further develop a self-administered diary tool and make recommendations to improve its design.

  • Westat will evaluate the web diary and functionality of the prototype and improve or provide alternative approaches for self-reported time and activity data in a web-based tool. Any improvements or alternative approaches should allow respondents to more easily and accurately report information.

Using best practices and expert methodological review, the contractor will improve the design and functionality of the web-based diary prototype