IA Supporting Statement B_jp(3)

Department of Commerce

United States Census Bureau

OMB Information Collection Request

2017 Economic Census of Puerto Rico, the U.S. Virgin Islands, Guam,

Commonwealth of the Northern Mariana Islands, and American Samoa—

Collectively Referred to as Island Areas¹

OMB: 0607-0937

Part B. Collections of Information Employing Statistical Methods

1. Universe and Respondent Selection

The 2017 Economic Census of Island Areas is a complete enumeration of establishments with payroll in 2017 located in Puerto Rico, the U.S. Virgin Islands, Guam, the Commonwealth of the Northern Mariana Islands, and American Samoa.

The universe of establishments is obtained from the U.S. Census Bureau's Business Register. To be included in the Economic Census of Island Areas, an establishment must be: (i) classified in an industry in-scope to the Economic Census of Island Areas; (ii) active with payroll in 2017; and (iii) located in Puerto Rico, the U.S. Virgin Islands, Guam, the Commonwealth of the Northern Mariana Islands or American Samoa.

The estimated size of the universe for the 2017 Economic Census of Island Areas is approximately 51,100 establishments. This estimate is based on the most recent data available from the Census Bureau’s Business Register. Table 1 below shows the sectors covered and the estimated universe size for 2017.

2. Statistical Methods

A. Required Accuracy and Estimation Procedures

The Economic Census of Island Areas is a complete enumeration of establishments located in the Island Areas. Therefore, the accuracy of tabulations is not affected by sampling error.

The accuracy of all census data is influenced by non-sampling errors, such as those affecting coverage, administrative records, questionnaire design, reporting, processing, and tabulation. Although we make no direct measurement of non-sampling errors, we take precautionary steps in all phases of planning, report form development, data collection, processing, and tabulation to minimize their influence.

The census uses periodic (5-year) data collection, as required by Title 13 U.S.C., Section 131. The Census Bureau uses administrative data from the IRS for payroll data of non-response cases. Subsequently, other data items are imputed using ratios from response data of other establishments in the same industry. Tabulations are simple summations of data from all in-scope establishments using reported data collected from the Economic Census plus imputed data for item and unit nonresponse.

B. Disclosure

In accordance with federal law governing census reports (Title 13 of the United States Code, Section 9), no data are published that would disclose the operations of an individual establishment or business. However, the number of establishments in a kind-of-business classification is not considered a disclosure; therefore, this information may be released. Cell values that have a potential for revealing confidential information must be suppressed or have their values perturbed by using the "Noise Infusion" technique.²

C. Response Metrics

Definitions and Formulae

For the 2017 Economic Census of Island Areas, the Census Bureau will calculate official measures of response rates according to OMB’s standards (Office and Management and Budget Standards and Guidelines for Statistical Surveys, September, 2006). For recent past economic censuses, the check-in rate was used to measure, monitor, and manage data collection and response. The check-in rate is calculated as the ratio of the number of reporting units³ returning⁴ a questionnaire to the number of reporting units mailed a request to complete a questionnaire:

The check-in rate is readily available in “real-time” during data collection, but does not incorporate information on the quality of the response data and the ability to consider them as respondent-reported data in summary statistics. The check-in rate for the 2012 Economic Census of Island Areas was 77%.

The OMB standards indicate that response rates must be computed using standard formulas to measure the proportion of the eligible sample that is represented by the responding units in each study, as an indicator of potential nonresponse bias. Both unweighted and weighted response rates should be calculated. Additionally, weighted response rates should incorporate the probability of selection or, in the case of establishment surveys, the proportion of key characteristics that is represented by the responding units.

In previous Economic Censuses, the method of calculating response rates did not conform with the OMB standard. In preparation for the 2012 Economic Census, the Census Bureau agreed to calculate future response rates in a manner consistent with the OMB standard, starting with the 2017 Economic Census. Accordingly, the Census Bureau’s Methodology and Standards Council approved a one-time waiver for the calculation of response rates according to Census Bureau Standards for the 2012 Economic Census (and 2012 Economic Census of Island Areas) with the condition that the Economic Census provide standard response metrics with the release of the 2017 Economic Census (and 2017 Economic Census of Island Areas) results. A copy of the approved waiver is provided in Attachment I. The OMB desk officer for the 2012 Economic Census ICR verbally agreed with the proposed waiver.

The OMB Unit Response Rate (URR) calculation uses only valid responses in the numerator of the ratio, and eliminates known ineligible reporting units from the denominator:

The URR is the proportion of reporting units, based on unweighted counts, that were eligible and provided valid responses or were of unknown eligibility (expressed as a percentage). For the 2017 Economic Census of Island Areas, a valid response is defined to be a unit that reported total receipts (i.e., value of shipments/receipts/revenue/sales ≥ $0) matching the definition of a valid response used by the rest of the 2017 Economic Census.

It is not possible to accurately measure the URR during data collection, because eligibility and response validity are frequently not determined until after collection is completed, during the editing phases. To monitor and manage response during data collection, a proxy URR will be calculated, as follows:

Research using 2012 Economic Census of Island Areas data has shown that the proxy URR appears to better reflect the URR than the check-in rate.

The OMB standards also define a weighted item response rate, where the item of interest is a quantity of primary interest for the survey. The Census Bureau defines the Quantity and Total Quantity Response Rates (QRR and TQRR) as item-level indicators of the "quality" of each estimate. Both are weighted response measures that take the size of the tabulating unit⁵ into account as well as the associated sampling parameters. The QRR measures the weighted proportion of an estimate obtained directly from the respondent for the survey. The TQRR expands the rate to include data substituted from secondary sources considered to be equivalent-in-quality to reported data, such as businesses’ annual reports or data provided by respondents for other Census Bureau surveys.

For a data item t, the QRR and TQRR are defined below:

,

where

w_i is the design weight of tabulating unit i,

r_ti is the indicator variable for reported data for tabulating unit i and data item t,

q_ti is the indicator variable for equivalent quality data sources for tabulating unit i and item t,

t_i is the data value for tabulating unit i, and

N_T is the total number of eligible tabulating units.

The TQRR(t) and QRR(t) are weighted item response rates for data item t. QRRs and TQRRs will be calculated post-collection for the following data items: receipts, payroll, and number of employees.

As with the URR, quantity response measures cannot be calculated during data collection, because the final source of the tabulated data is not determined until after collection is completed, during the post-collection processing. Additionally, quantity response measures are based on tabulating units instead of reporting units. Therefore, proxy QRR and TQRR metrics that use an appropriate measure of size characteristic, such as administrative annual payroll, will be calculated periodically during data collection to monitor the proportion of total administrative payroll represented by respondents, defined according to the same criteria used for the Proxy URR.

Projections for the 2017 Economic Census

Due to major changes in data collection procedures – e.g., increased reliance on electronic Web-based collection – and the implementation of these new response metrics, it is difficult to confidently project URR and TQRRs for the 2017 Economic Census of Island Areas.

Since data are published on an island-by-island basis, a nonresponse bias study will be conducted at an island level for any key variable with a TQRR below 70%. For this purpose, key variables are identified as receipt and payroll totals. The TQRR was chosen because it is better than the URR at reflecting the skewness of the population and the use of administrative data and other data sources when calculating response metrics. If a nonresponse bias study is warranted, a description of the findings will be included in the 2022 Economic Census OMB information collection request.

3. Methods to Maximize Response

This information collection will maximize response through the following means:

1. public awareness campaign to assist businesses and the public in understanding the importance of the 2017 Economic Census of Island Areas by providing news stories through trade and professional associations, chambers of commerce, and business and general media;

2. assignment of account managers to large businesses to provide personalized assistance;

3. redesigned census questionnaires and information sheets with standard formats and terminology that try to simplify reporting and minimize response burden;

4. the use of electronic reporting capabilities that will allow all businesses to report using the Internet;

5. mailing materials that emphasize the mandatory and confidential nature of economic census reports, as provided by Title 13 U.S.C.;

6. toll-free assistance and/or a web-based help desk for any business that has questions about completing the Economic Census of Island Areas; and

7. systematic mail follow-up for nonresponse, supplemented by telephone follow-up for selected firms. Through these and other response improvement strategies, we expect to maintain or improve the 77 percent check-in rate obtained by the 2012 Economic Census of Island Areas. This level of response when combined with data review and corrections will yield accuracy and reliability that are adequate for intended uses of economic census data; and

8. respondent portal where respondents can obtain answers to frequently asked questions, send secure emails to Census Bureau staff, report electronically and perform other self-service options without having to pick up the phone.

4. Testing of Procedures

This information collection will use procedures that are based on a considerable body of experience with the Economic Census and surveys along with research conducted for the 2017 Economic Census of Island Areas. Re-engineering economic census data collection instruments for implementation via the Web required a substantial multi-year, multi-method program of research, development, testing and evaluation.

Activities included task analyses, record-keeping studies, exploratory early stage scoping interviews, cognitive testing, usability testing, analyses of electronic paradata, respondent debriefings, and field tests, along with early implementation of designs within other Census Bureau establishment surveys, such as the annual Company Organization Survey (aka Report of Organization) and the Annual Survey of Manufactures. Different research components, in turn, focused on different subpopulations, different instrument components, or different question(naire) content, and results and recommendations were used to aid design decisions.

Of particular note is the use of the annual ASM/COS as a testing and proving ground for many new techniques planned for the Economic Census, as these two surveys collect data at the establishment level like the Economic Census. This began by conducting debriefing interviews with 2014 ASM/COS multi-unit establishment (MU) respondents to identify strengths and weaknesses of the legacy Surveyor software used for electronic reporting to aid development of initial requirements for the online Centurion instrument.

The 2015 ASM provided a full-scale pilot of the new online Centurion collection instrument for single-establishments (SU) using the response-driven design being built for the 2017 Economic Census. Since paper questionnaires were not available, more than 95% of SU respondents reported online, while only a few reported their data by some other method, such as telephone. Additionally, respondent debriefings were conducted and Web paradata were analyzed to further evaluate the response-driven instrument and to recommend improvements.

The 2015 ASM also integrated research testing the value and effectiveness of targeted subsampling, to provide evidence for decisions about implementing an adaptive design approach in the 2017 Economic Census. (See Section 3.)

Finally, MUs in the 2016 ASM/COS are using the new online Centurion collection instrument, which features much of the functionality developed for the 2017 Economic Census, such as importing/exporting spreadsheets, the ability to toggle between “form” views and spreadsheet views of the questions, and questionnaire preview worksheets to aid data gathering. Respondent debriefings and Web paradata analysis will be conducted to further evaluate the instrument prior to conducting the 2017 Economic Census.

Results and recommendations from each component and testing activity were provided to subject matter specialists, managers and developers of data collection procedures, and software developers and programmers. Recommendations were prioritized, negotiated, and revised, as necessary, to facilitate implementation. Nevertheless, some recommendations were not fully adopted due to resource and technical limitations.

All studies are documented in internal Census Bureau reports; a few of the studies are

also reported publicly in conference papers or presentations.

Components of the research and testing, along with selected highlights, follow:

Component	Activity	Dates	Number of Cases	OMB Generic Clearance Date	Description, Selected Results and/or Recommendations
Single Unit Businesses	Task analysis	July 2014	8	June 2014	Response process is relatively straightforward for SUs; however, they rely heavily on paper questionnaires for gathering the data.
	Usability testing Round 1	October –November 2014	10	October 2014	Tested a low-fidelity prototype and provided recommendations for a response-driven design for implementation in 2014 ASM. Respondents preferred less information on the screen and skipping questions that were not relevant to them.
	Usability testing Round 2	October 2015	9	October 2014	Tested the 2015 ASM response-driven instrument prior to release in December 2015
	Respondent debriefings for 2015 ASM full-scale implementation of response-driven design	February – June 2016	23	January 2016	Respondents liked response-driven design, answering questions “one at a time.” Respondents wanted a worksheet preview, but did not see this feature prior to beginning.
	Analysis of Web paradata from 2015 ASM full-scale implementation of response-driven design	April 2016	NA	NA	Identified troublesome screens/questions; reviewed use of features/functions; evaluated indicators of burden. Prioritized changes for 2016 ASM.
Multi-Unit (MU) Businesses	Respondent debriefings for 2012 Economic Census	February 2014	11	January 2014	Debrief respondents about their use of the Census Surveyor instrument in order to identify features that should be maintained and/or improved for the 2017 Economic Census electronic data collection instrument.
	Usability testing Round 1	November 2014	10	November 2014	Tested a low-fidelity prototype and gathered requirements for a functioning prototype, and tested initial spreadsheet designs.
	Usability testing Round 2	May - June 2015	24	November 2014	Performance of SharePoint prototype inadequate and solution abandoned. Feedback on proposed MU instrument interface generally positive for main menu design, introductory screens, and step-by-step instructions for instrument features. Feedback on alternative spreadsheet designs generally positive for incorporating questions’ instructions and improved visual design.
	Focus group with NPC Register Analysis Branch analysts	April 2015		NA	Discussed analysts’ experience assisting respondents with Census Surveyor and obtained feedback on plans for revising the MU instrument.
Multi-Unit (MU) Businesses, continued	Debriefing interviews with 2014 COS/ASM respondents	April – June 2015	30	March 2015	Evaluated use of Census Surveyor features / functionality with respondents at small and mid-size multi-unit companies. Corroborated past research: Small MUs preferred within-instrument functions; Mid-size / larger MUs preferred Excel spreadsheet functions; Full capabilities of Census Surveyor overlooked by Rs; Downloaded / printed documents critical for gathering data from other company personnel
	Usability testing Round 3	September 2015	17	June 2015	Evaluated basic functionality of MU instrument prototype in Centurion - spreadsheet download/upload, form view, and database that communicates to spreadsheet and form view; navigation and usability of pages; effectiveness of content, especially instructions about reporting procedures; navigation and usability of spreadsheet template; gathered requirements for worksheet/reporting guide
	Analysis of Census Surveyor paradata from 2015 ASM	July – September 2016	NA	NA	Although the legacy Census Surveyor instrument was a powerful tool with useful features, Rs were not aware of / did not use quite a few of them.
North American Product Classification System (NAPCS)	Record-keeping study	November 2014	9	August 2014	Explored availability of product line data in respondents’ business records.
	Usability testing Round 1	October 2015	9	June 2015	Tested alternative presentations of product lists for SUs; recommended 2-screen design where Rs first select relevant products from a list, then second screen displays only the selected products and requests receipts for each one.
	Usability testing Round 2	October 2015	7	June 2015	Tested with Construction SUs and MUs: Prefer 2-screen design over spreadsheets; “Building” and “Non-building” categories helped Rs focus on their construction type
	Usability testing Round 3	April 2015	12	August 2014	Investigated the utility of a search feature for looking up products. Rs preferred “write-in” over search functionality, as tested.
	Usability testing Round 4	November 2015	9	August 2014	Tested orientation of MU NAPCS spreadsheets; decision to replicate Census Surveyor version with products listed in columns and locations in rows.
	Field test of alternative NAPCS question designs and write-ins	January – March 2017	892	December 2016	This will be a field test using an experimental design to examine the effectiveness of two alternative designs and lay-out options for obtaining descriptions of products not explicitly provided in the product lists. Results are expected in late March, and will be used to recommend a design strategy for “write-ins” in the final Economic Census instrument.
MU/SU Centurion Instrument	Usability testing for 2016 ASM/COS Round 1	July 2016	18	June 2016	Tested a limited-functioning prototype. The redesigned instrument performed well overall, but additional instructions and clearer labeling of buttons would be helpful to respondents
	Usability testing for 2016 ASM/COS Round 2	October 2016	28	June 2016	Tested a more fully functioning protoype. There were some features and functionality that were not clear to participants, e.g. how to update their locations.
	Usability testing for 2017 Economic Census	January – March 2017	TBD	TBD	This will be the first usability testing of complete fully functioning Centurion data collection instrument for 2017 EC.
Web Portal Design and Functionality (eCorrespondence)	Usability testing	July 2016	18	June 2016	Joint with ASM/COS instrument usability testing. Early results did not show major difficulties with using the new Portal.
	Usability testing	October 2016	19	September 2016	Overall, the account creation process and linking a survey to that account was not problematic.
Worksheet / Question Preview	2015 COS/ASM Worksheet Debriefings	February – March 2016	25	January 2016	Evaluated use of form/PDFs in current systems, and obtained feedback on worksheet prototype: Form/PDFs play critical role in data collection Most Rs used them (or would have, had they known about them) Positive feedback on prototype worksheet

Previous economic censuses of island areas also have been the subject of evaluation studies that have examined methodology, conceptual issues, and related statistical questions. Cognitive testing was conducted for the 2017 questionnaires intended for use in Guam, the Commonwealth of the Northern Mariana Islands, and Puerto Rico. Approximately 50 companies were interviewed, and the information obtained from the testing influenced the questions asked on the collection instruments. Additionally, substantial testing of content has been done by the stateside economic census. As a result, the procedures used by the 2017 Economic Census of Island Areas are well tested.

5. Contacts for Statistical Aspects and Data Collection

The table below lists the names of the individuals responsible for the collection, analysis, and statistical methodology aspects of the 2017 Economic Census of Island Areas.

Table 5: Census Bureau Contacts for the 2017 Economic Census

Contact	Position	Phone	Responsibility
William C. Davie, Jr	Methodology Director for the Business Register, Economic Census & Related Surveys, Economic Statistical Methods Division	301-763-7182	Development of statistical methodology
Diane Willimack	Methodology Director for Methodology and Response Improvement for Economic Programs, Economic Statistical Methods Division	301-763-3538	Pretesting of data collection instruments and methods
William Samples	Assistant Survey Director for the Economic Census, Economy-Wide Statistics Division	301-763-7175	Overall supervision

Attachments

1. Electronic Questionnaire Instrument Path Numbers and Titles

2. Draft Previews of Questionnaires

3. Questionnaire Information Sheets

4. Drafts of Initial Contact and Follow-up Letters

5. Summary of Changes to Questionnaires

6. Consultations with Representatives of the Island Areas, Persons and Organizations Contacted

7. Formal Requests of 2017 Economic Census of Island Areas

8. Electronic Instrument Selected Screen Shots

9. Quality Standard Waiver

1 The term used to collectively refer to Puerto Rico, the U.S. Virgin Islands, Guam, the Commonwealth of the Northern Mariana Islands, and American Samoa in the Federal Register on June 7, 2011 was Island Areas. Island Areas is common terminology across the Census Bureau.

2 Noise infusion is a method of disclosure avoidance in which values for each firm are perturbed prior to table creation by applying a random noise multiplier to the magnitude data (i.e., characteristics such as receipts, payroll, and number of employees) for each company. Disclosure protection is accomplished in a manner that results in a relatively small change in the vast majority of cell values. For the 2017 Economic Census of Island Areas, each published cell value will have an associated noise flag, indicating the relative amount of distortion in the cell value resulting from the perturbation of the data for the contributors to the cell. The flag for ‘low noise’ (G) indicates the cell value was changed by less than 2 percent with the application of noise, and the flag for ‘moderate noise’ (H) indicates the value was changed by 2 percent or more but less than 8 percent. Cells that have been changed by 8 percent or more are suppressed from the published tables. Additionally, other cells in the table may be suppressed for additional protection from disclosure or because the quality of the data does not meet publication standards. Though some of these suppressed cells may be derived by subtraction, the results are not official and may differ substantially from the true estimate.

3 A reporting unit is an entity from which data are collected. The economic census uses two types of reporting units. The first is the establishment, which is an economic unit usually at a single, physical location where business is conducted or where services or industrial operations are performed. Most businesses report data for the economic census at the establishment level. The second type of reporting unit employed by respondents to answer the economic census is referred to as an “Alternative Reporting Unit (ARU).” ARUs are generally a consolidation of establishments owned by the same company. ARUs are typically used by firms engaged in networked industries such as finance, insurance, or utilities, to facilitate reporting of revenue and expense data.

4 A returned questionnaire includes receipt of an electronic submission authorized by the respondent, receipt of an acceptable response during targeted telephone follow-up calls, or, under special circumstances, respondent-authorized submission by some other means.

5 The tabulating unit houses the data for estimation and tabulation. For the Economic Census, the tabulating unit is the establishment.