2009 Methods Panel OMB Supporting Statement Pta 9-24-09_revised 12-28-09

SUPPORTING STATEMENT

U.S. Department of Commerce

U.S. Census Bureau

American Community Survey Method Panel Tests

OMB Control Number 0607-0936

Part A. Justification

1. Necessity of the Information Collection

The U.S. Census Bureau requests authorization from the Office of Management and Budget (OMB) to conduct the American Community Survey (ACS) Methods Panel tests.

Given the rapid demographic changes experienced in recent years and the strong expectation that such changes will continue and accelerate, the once-a-decade data collection approach to a census is no longer acceptable as a source for the housing and socio-economic data collected on the census long-form. To meet the needs and expectations of the country, the Census Bureau developed the ACS. This survey collects detailed socioeconomic data every month and provides tabulations of these data on a yearly basis. The ACS allows the Census Bureau to provide more timely and relevant housing and socio-economic data while also reducing operational risks in the census by eliminating the long-form historically given to one in every six addresses.

Full implementation of the ACS includes an annual sample of approximately three million residential addresses a year in the 50 states and the District of Columbia, and another 36,000 addresses in Puerto Rico. A sample this large allows for annual production and release of single-year estimates for areas with a population of 65,000 or more. Lower levels of geography require aggregates of three and five years’ worth of data in order to produce estimates of comparable reliability to the census long-form. However, an ongoing data collection effort with an annual sample of this magnitude requires that the ACS continue research, testing and evaluations aimed at improving ACS questionnaires content and related data collection materials. In addition, the ACS Methods Panel during the 2010-2012 period may include testing methods for increasing survey and operational efficiencies; alternative methods or procedures may be developed and evaluated that could potentially reduce the overall survey cost, lessen respondent burden, and improve response rates. At this time, specific plans are in place to propose three methods panel tests: a content reinterview study, the 2010 ACS Content Test, and an Internet Mode Test. Since the ACS Methods Panel is designed to address emerging issues, we may conduct additional testing as needed. Testing would focus on methods for reducing data collection costs or testing new questions that have an urgent need to be included on the ACS.

During Census 2000, a content reinterview study (CRS) was conducted in conjunction with the long form, which the ACS now replaces. The decennial CRS was an evaluation of the quality of the data collected in the census, focusing on response bias and simple response variance (reliability). The Census Bureau proposes to design and implement a CRS to look at the current ACS production questions on an ongoing basis. This will allow for the identification of problems with reliability. Results from the CRS will provide data users with concrete data quality measures (such as reliability or bias measures) for each ACS item.

The ACS CRS will allow the Census Bureau to continuously monitor the data quality of the ACS and identify questions that are currently unreliable or that may become unreliable due to changes in the survey climate (e.g., changes in policy that change the definition of what the ACS is trying to measure). The results from the CRS, generated on a yearly basis, would identify which questions require modifications and future testing via a content test, thus providing a more scientific approach to determining the need for content testing of current ACS items. The CRS will be conducted by telephone only with a small sample of cases that responded during production.

Second, in response to federal agencies’ requests for new and revised ACS questions, the Census Bureau plans to conduct the 2010 ACS Content Test. Changes to the current ACS content and the addition of new content were identified through the Interagency Committee for the ACS and through recent legislative action. The primary objective of the 2010 ACS Content Test is to test whether changes to question wording, response categories, and redefinition of underlying constructs improve the quality of data collected. The Census Bureau proposes to evaluate changes to the questions, or for new questions, to compare the performance of question versions to each other as well as to other well-known sources of such information. The proposed topics for content testing are new questions to measure computer and Internet access and usage, as well as parental place of birth and revisions to veteran’s identification and period of service, cash public assistance income, wages income and property income, and the Food Stamp program name.

A third test, the ACS Internet Mode Test, is planned to determine the best methods for informing sample households about an ACS Internet response option and encouraging them to respond. By offering an Internet response option in the ACS, the Census Bureau is taking further steps to comply with the e-gov initiative and potentially reduce collection costs. The objectives of the Internet Mode Test include: potential improvement in self-response rates; potential cost savings if we can change the distribution of responses by mode (i.e. obtain more responses by Internet); and potential improvement in data quality including a potential reduction in item nonresponse.

Two additional tests are currently planned. The specific details of these tests are not known at this time but would include an additional content test for new questions, revised questions, and/or forms design changes that have an urgent need as well as an additional test of new methods to address an emergent need.

The Census Bureau collects data for this survey under Title 13, United States Code, Sections 141, 193, and 221. All data are afforded confidential treatment under Section 9 of that Title.

For minor changes to the tests described in this justification, the Census Bureau will submit a non-substantive change request documenting the change or revision. For substantive changes to the tests described in this justification or for any new tests that are needed to address a new or emerging data collection issue, the Census Bureau will submit a revised Information Clearance Request to OMB and simultaneously publish a 30 day notice for public comment in the Federal Register describing the changes or new tests proposed.

2. Needs and Uses

The ACS must collect data on a continual basis and aggregate one, three, or five years worth of data to release data for all states, Congressional districts, counties, cities, and small towns down to the census tract and block group level. Essentially the ACS collects data every day of the year, either by mail, telephone interviews or personal-visit interviews. Federal agencies use the ACS data to determine appropriate funding for state and local governments through block grants. State and local governments use ACS data for program planning, administration and evaluation. Thus, the reliability and the quality of the data must remain high in order for the users to rely on the data for funding decisions.

So that the Census Bureau can provide critical information to governments and the private sector, the ACS collects comprehensive and timely demographic, social, economic, and housing statistics covering every community in the nation. The ACS provides, for the first time on an annual basis, a continuous stream of updated information for states and local areas, and has revolutionized the ways the country uses data to understand our communities and plan for our future. The ACS allows federal agencies, state governments, tribal officials, and local customers to make decisions based on current information, not the situation years ago, and will allow the Census Bureau to refine the process and ensure that it is in place to produce tract-level data (in general, areas with populations between 1,500 and 8,000 people) comparable in quality to the Census 2000 long-form data.

There are many federal programs that distribute funds based in whole or in part on population and income data attributed to data from the Census Bureau, including data from the ACS. Eliminating the long form and implementing the ACS allows small geographic areas and small population groups to obtain a more current portrait of their communities and to identify change more frequently than once every ten years. This is particularly true for small towns, rural America, and tribal lands including American Indian reservations and Hawaiian Homelands, each of which will rely solely on five years of combined ACS data to help them identify the occurrence of real change.

ACS Methods Panels, such as the 2010 Content Test, Internet Mode Test, and the ACS Content Reinterview Study, provide a mechanism to investigate ways to reduce or at least maintain data collection costs and improve the quality of the data.

Information quality is an integral part of the pre-dissemination review of the information disseminated by the Census Bureau (fully described in the Census Bureau’s Information Quality Guidelines). Information quality is also integral to the data collection conducted by the Census Bureau and is incorporated into the clearance process required by the Paperwork Reduction Act.

3. Use of Information Technology

The ACS CRS will use ACS production cases and follow-up a sample of those with a telephone number using a Computer Assisted Telephone Interview (CATI).

The 2010 ACS Content Test field portion will largely be based on the data collection methods currently used in the production ACS. All sampled addresses will be mailed a pre-notice letter, a self-administered paper questionnaire, and a reminder postcard. Households that do not return their initial questionnaire in a timely manner will also be mailed a replacement questionnaire. For households that do not return their mailed questionnaire, we will attempt to collect their data through a follow-up CATI or Computer Assisted Personal Interview (CAPI). During CATI and CAPI interviews, we will use Computer Audio Recorded Interviewing (CARI) technology to record portions of the interview related to the questions being tested for use in behavior coding. A script of the planned questions requesting consent for the recording will be provided to OMB when available.

Several of the mail pieces (the questionnaires, the cover letters, the reminder postcard, and the brochure) for these tests will include a toll-free number to reach staff at the Telephone Questionnaire Assistance (TQA) center. In the TQA operation, census interviewers can complete the ACS interview using an automated data collection instrument. Additionally, several mailing pieces will also include a URL for the ACS where respondents can go to obtain more information about the ACS. This test will also include a CATI Content Follow-up (CFU).

The Internet Mode Test, a test of new collection methodology, will include an online instrument in English and Spanish in addition to the mail data collection. This test will also include some variation on the standard ACS mailing procedure in order to measure whether we can encourage response to the ACS on the Internet by offering this option without the presence of a mail questionnaire at the same time.

4. Efforts to Identify Duplication

The ACS Methods Panel is the only testing vehicle for the ACS. There is no other program whose purpose is to improve the ACS.

5. Minimizing Burden

The proposed data collections consist of questions asked of a small sample of households, not businesses or other small entities.

6. Consequences of Less Frequent Collection

The 2010 Content Test and Internet Mode Test represent one-time, special tests with a defined period for data collection. The ACS CRS will be an ongoing effort to study the data quality of the ACS production data on an annual basis. Thus, the ACS CRS will be implemented each month and the results tabulated for each year.

7. Special Circumstances

The Census Bureau will collect these data in a manner consistent with the OMB guidelines.

8. Consultations Outside of the Agency

The Internet Mode Test and the ACS Content Reinterview Study reflect internal Census Bureau needs, thus we have not involved other federal agencies directly in the design of the test. However, we regularly consult with mathematical statisticians and researchers from Westat, NORC, RTI and other survey and research firms as well as the University of Michigan’s Institute for Survey Research and other academic institutions. In addition, methods panel staff regularly review survey methodology and form design literature and attend workshops and conferences that present state-of-the-art methods and procedures. For the Internet Mode Test we have consulted with experts in the fields of web survey design and usability/graphical user interface.

For the 2010 Content Test we consulted the following federal agencies: Office of Management and Budget, Federal Communications Commission, Department of Commerce, National Telecommunications and Information Administration, Department of Transportation, U.S. Department of Agriculture, Department of Veterans Affairs, Department of Health and Human Services, National Institute of Child Health and Human Development, National Center for Health Statistics, Department of Homeland Security, Department of Labor, and Bureau of Labor Statistics.

We published a notice in the Federal Register on June 5, 2009 (vol. 74, pg. 27013), inviting the public and other federal agencies to comment on our plans to submit this request. We received three comments in response to that notice. One response the Census Bureau felt was not relevant to these proposed data collections. The second response “strongly supports the proposed information collection,” and the third from the Department of Transportation (DOT) had two requests.

The first DOT request was the addition of a “light rail” response to the answer categories for Question 31 Means of Transportation to Work. The Census Bureau cannot accommodate this request in the current ACS Content Test. Questions currently proposed for the next ACS Content Test were developed in January 2009 and have been cognitively tested. However, the Census Bureau will consider this request for the next ACS Content Test and in the interim, will, as requested, include information in the ACS mail-out instruction guide clarifying how to respond to the Means of Transportation to Work question, including the category to use if the respondent takes light rail. In addition, telephone and field interviewers have been trained to code ‘light rail’ responses appropriately.

The second DOT request was for the Census Bureau to add a calculated distance to work variable to the ACS data. However, the Census Bureau has determined that releasing such information would pose a very high disclosure risk, particularly for small geographies including census tracts and block groups. The necessary disclosure protections that would be placed on the data, such as using geographic centroids for calculation and releasing data in large category sizes, would render the resulting information to be of very low added value.

9. Paying Respondents

We do not pay respondents or provide respondents with gifts.

10. Assurance of Confidentiality

The Census Bureau collects data for this survey under Title 13, United States Code, Sections 141, 193, and 221. All data are afforded confidential treatment under Section 9 of that Title.

In accordance with Title 13, each household will be assured of the confidentiality of their answers. A brochure is included in the questionnaire packet that is sent to sample households and contains the following assurance:

The law, Title 13, Sections 141, 193, and 221 of the U.S. Code, authorizing the American Community Survey, also provides that your answers are confidential. No one except Census Bureau employees may see your completed form and they can be fined and/or imprisoned for any disclosure of your answers.

This brochure also explains that participation in the ACS is mandatory. These data collection materials are approved under OMB clearance 0607-0810.

11. Justification for Sensitive Questions

Some of the data we collect, such as race and sources of income and assets may be considered to be of a sensitive nature. The Census Bureau takes the position that the collection of these types of data is necessary for the analysis of important policy and program issues and has structured the questions to lessen their sensitivity.

12. Estimate of Hour Burden

2010 Content Test: During the period of September 1 – December 10, 2010, we plan to contact a maximum of 70,000 residential addresses for the field test portion of the content test. We estimate the time to complete the ACS Content Test will be the same as ACS production (38 minutes). Based on the 2007 ACS unweighted interview rate of 67 percent (the percent of sampled addresses that respond by mail, CATI, or CAPI) and an expected phone number rate of 85 percent (the percent of households with phone numbers either acquired from a vendor or provided by the respondent in the initial interview), we plan to contact approximately 40,000 households that responded during the field test during the Content Follow-up (CFU) portion of the content test. The CFU reinterview should average about 15 minutes per household.

Internet Mode Test: For an average household, the estimated time to complete the ACS form is 38 minutes, including the time for reviewing the instructions and answers. Total sample size is 120,000 residential addresses in the United States and 2,000 residential addresses in Puerto Rico.

ACS Content Reinterview Study: For an average household, the estimated completion time is 20 minutes per reinterview and the yearly sample is 72,000. The test of this instrument will include a sample of 1000 sample cases at 20 minutes per reinterview.

Other Potential Content Test: For another potential content test we would expect to contact a maximum of 70,000 residential addresses for the field test portion of the content test. We estimate the time to complete a second ACS Content Test will be the same as ACS production (38 minutes). Based on the 2007 ACS interview rate of 0.67 and an expected phone number rate of 0.85, we would plan to contact approximately 40,000 households that responded during the field test during the CFU portion of the content test. The CFU reinterview should average about 15 minutes per household.

Other Potential Test of New Methods: For an average household, the estimated time to complete the ACS form is 38 minutes, including the time for reviewing the instructions and answers. Total sample size is 30,000 residential addresses in the United States

Table 1. Response Burden

Test and Test Components	Respondents	Responses	Estimated Average Burden Hours
2010 Content Test	70,000		54,333
Field Test	---------	70,000	(44,333)
CFU	---------	40,000	(10,000)
Internet Mode Test	122,000	122,000	77,267
ACS Content Reinterview Test	1,000	1,000	334
ACS Content Reinterview Study	216,000	216,000	72,000
Other Potential Content Test	70,000		54,333
Field Test	---------	70,000	(44,333)
CFU	---------	40,000	(10,000)
Other Potential Test of New Methods	30,000	30,000	19,000
Total	509,000*	589,000*	277,267*

*Annual estimates of the number of respondents, responses and reporting burden are calculated by dividing totals above by three years.

13. Estimate of Cost Burden

There are no costs to the respondent other than his/her time to respond to the survey.

14. Cost to Federal Government

2010 Content Test: $4 million

Internet Mode Test: $2 million

ACS Content Reinterview Study: $1.2 million

The Census Bureau will pay all costs of the Methods Panel tests.

15. Reason for Change in Burden

This collection is being submitted as revised as it is an ongoing activity. The burden hours are increasing because the ACS has received full funding for the Methods Panel and is implementing new and different tests to improve the ACS. The experimental designs and objectives for these tests require additional sample to measure the impact of the changes.

16. Project Schedules

Table 2a. 2010 Content Test

Activities	Time Frame
Field test data collection	Early-September - late-November 2010
CFU data collection	Mid-September - mid-December 2010
Statistically analyze results	Early-February - late-August 2011
Final results and recommendations released	Early-September - early-October 2011

Table 2b. Internet Mode Test

Activities	Time Frame
Design Internet instruments	October 2008 - October 2009
Program and System Test Internet instruments; Usability Test, Iterative Versions of User Interface	November 2009 - February 2011
Field test	March 2011 - June 2011
Analyze results	July 2011 - December 2011
Implementation	January 2013

Table 2c. ACS Content Reinterview Study

Activities	Time Frame
Determine reinterview modules	October 2009 – January 2010
Develop specifications for the reinterview instrument	February 2010 – April 2010
Develop reinterview instrument	May 2010 – May 2011
Field test of system	June 2011
Review results of field test and make modifications as necessary	September 2011 – December 2011
Start reinterview	January 2012 – December 2012
Analyze first year reinterview results	January – July 2013

17. Request Not to Display Expiration Date

We will display the expiration date on the form(s).

18. Exceptions to the Certification

There are no exceptions to the Certification for Paperwork Reduction Act submission.

9