IA Supporting Statement B_jp(3)

IA Supporting Statement B_jp(3).docx

2017 Economic Census of Island Areas

OMB: 0607-0937

Document [docx]
Download: docx | pdf

Department of Commerce

United States Census Bureau

OMB Information Collection Request

2017 Economic Census of Puerto Rico, the U.S. Virgin Islands, Guam,

Commonwealth of the Northern Mariana Islands, and American Samoa—

Collectively Referred to as Island Areas1

OMB: 0607-0937



Part B. Collections of Information Employing Statistical Methods


1. Universe and Respondent Selection


The 2017 Economic Census of Island Areas is a complete enumeration of establishments with payroll in 2017 located in Puerto Rico, the U.S. Virgin Islands, Guam, the Commonwealth of the Northern Mariana Islands, and American Samoa.


The universe of establishments is obtained from the U.S. Census Bureau's Business Register. To be included in the Economic Census of Island Areas, an establishment must be: (i) classified in an industry in-scope to the Economic Census of Island Areas; (ii) active with payroll in 2017; and (iii) located in Puerto Rico, the U.S. Virgin Islands, Guam, the Commonwealth of the Northern Mariana Islands or American Samoa.


The estimated size of the universe for the 2017 Economic Census of Island Areas is approximately 51,100 establishments. This estimate is based on the most recent data available from the Census Bureau’s Business Register. Table 1 below shows the sectors covered and the estimated universe size for 2017.


2. Statistical Methods


A. Required Accuracy and Estimation Procedures


The Economic Census of Island Areas is a complete enumeration of establishments located in the Island Areas. Therefore, the accuracy of tabulations is not affected by sampling error.


The accuracy of all census data is influenced by non-sampling errors, such as those affecting coverage, administrative records, questionnaire design, reporting, processing, and tabulation. Although we make no direct measurement of non-sampling errors, we take precautionary steps in all phases of planning, report form development, data collection, processing, and tabulation to minimize their influence.


The census uses periodic (5-year) data collection, as required by Title 13 U.S.C., Section 131. The Census Bureau uses administrative data from the IRS for payroll data of non-response cases. Subsequently, other data items are imputed using ratios from response data of other establishments in the same industry. Tabulations are simple summations of data from all in-scope establishments using reported data collected from the Economic Census plus imputed data for item and unit nonresponse.





B. Disclosure


In accordance with federal law governing census reports (Title 13 of the United States Code, Section 9), no data are published that would disclose the operations of an individual establishment or business. However, the number of establishments in a kind-of-business classification is not considered a disclosure; therefore, this information may be released. Cell values that have a potential for revealing confidential information must be suppressed or have their values perturbed by using the "Noise Infusion" technique.2


C. Response Metrics


Definitions and Formulae

For the 2017 Economic Census of Island Areas, the Census Bureau will calculate official measures of response rates according to OMB’s standards (Office and Management and Budget Standards and Guidelines for Statistical Surveys, September, 2006). For recent past economic censuses, the check-in rate was used to measure, monitor, and manage data collection and response. The check-in rate is calculated as the ratio of the number of reporting units3 returning4 a questionnaire to the number of reporting units mailed a request to complete a questionnaire:





The check-in rate is readily available in “real-time” during data collection, but does not incorporate information on the quality of the response data and the ability to consider them as respondent-reported data in summary statistics. The check-in rate for the 2012 Economic Census of Island Areas was 77%.

The OMB standards indicate that response rates must be computed using standard formulas to measure the proportion of the eligible sample that is represented by the responding units in each study, as an indicator of potential nonresponse bias. Both unweighted and weighted response rates should be calculated. Additionally, weighted response rates should incorporate the probability of selection or, in the case of establishment surveys, the proportion of key characteristics that is represented by the responding units.

In previous Economic Censuses, the method of calculating response rates did not conform with the OMB standard. In preparation for the 2012 Economic Census, the Census Bureau agreed to calculate future response rates in a manner consistent with the OMB standard, starting with the 2017 Economic Census. Accordingly, the Census Bureau’s Methodology and Standards Council approved a one-time waiver for the calculation of response rates according to Census Bureau Standards for the 2012 Economic Census (and 2012 Economic Census of Island Areas) with the condition that the Economic Census provide standard response metrics with the release of the 2017 Economic Census (and 2017 Economic Census of Island Areas) results. A copy of the approved waiver is provided in Attachment I. The OMB desk officer for the 2012 Economic Census ICR verbally agreed with the proposed waiver.

The OMB Unit Response Rate (URR) calculation uses only valid responses in the numerator of the ratio, and eliminates known ineligible reporting units from the denominator:



The URR is the proportion of reporting units, based on unweighted counts, that were eligible and provided valid responses or were of unknown eligibility (expressed as a percentage). For the 2017 Economic Census of Island Areas, a valid response is defined to be a unit that reported total receipts (i.e., value of shipments/receipts/revenue/sales ≥ $0) matching the definition of a valid response used by the rest of the 2017 Economic Census.

It is not possible to accurately measure the URR during data collection, because eligibility and response validity are frequently not determined until after collection is completed, during the editing phases. To monitor and manage response during data collection, a proxy URR will be calculated, as follows:



Research using 2012 Economic Census of Island Areas data has shown that the proxy URR appears to better reflect the URR than the check-in rate.

The OMB standards also define a weighted item response rate, where the item of interest is a quantity of primary interest for the survey. The Census Bureau defines the Quantity and Total Quantity Response Rates (QRR and TQRR) as item-level indicators of the "quality" of each estimate. Both are weighted response measures that take the size of the tabulating unit5 into account as well as the associated sampling parameters. The QRR measures the weighted proportion of an estimate obtained directly from the respondent for the survey. The TQRR expands the rate to include data substituted from secondary sources considered to be equivalent-in-quality to reported data, such as businesses’ annual reports or data provided by respondents for other Census Bureau surveys.

For a data item t, the QRR and TQRR are defined below:

,

where

wi is the design weight of tabulating unit i,

rti is the indicator variable for reported data for tabulating unit i and data item t,

qti is the indicator variable for equivalent quality data sources for tabulating unit i and item t,

ti is the data value for tabulating unit i, and

NT is the total number of eligible tabulating units.

The TQRR(t) and QRR(t) are weighted item response rates for data item t. QRRs and TQRRs will be calculated post-collection for the following data items: receipts, payroll, and number of employees.

As with the URR, quantity response measures cannot be calculated during data collection, because the final source of the tabulated data is not determined until after collection is completed, during the post-collection processing. Additionally, quantity response measures are based on tabulating units instead of reporting units. Therefore, proxy QRR and TQRR metrics that use an appropriate measure of size characteristic, such as administrative annual payroll, will be calculated periodically during data collection to monitor the proportion of total administrative payroll represented by respondents, defined according to the same criteria used for the Proxy URR.

Projections for the 2017 Economic Census

Due to major changes in data collection procedures – e.g., increased reliance on electronic Web-based collection – and the implementation of these new response metrics, it is difficult to confidently project URR and TQRRs for the 2017 Economic Census of Island Areas.


Since data are published on an island-by-island basis, a nonresponse bias study will be conducted at an island level for any key variable with a TQRR below 70%. For this purpose, key variables are identified as receipt and payroll totals. The TQRR was chosen because it is better than the URR at reflecting the skewness of the population and the use of administrative data and other data sources when calculating response metrics. If a nonresponse bias study is warranted, a description of the findings will be included in the 2022 Economic Census OMB information collection request.



3. Methods to Maximize Response


This information collection will maximize response through the following means:


  1. public awareness campaign to assist businesses and the public in understanding the importance of the 2017 Economic Census of Island Areas by providing news stories through trade and professional associations, chambers of commerce, and business and general media;

  2. assignment of account managers to large businesses to provide personalized assistance;

  3. redesigned census questionnaires and information sheets with standard formats and terminology that try to simplify reporting and minimize response burden;

  4. the use of electronic reporting capabilities that will allow all businesses to report using the Internet;

  5. mailing materials that emphasize the mandatory and confidential nature of economic census reports, as provided by Title 13 U.S.C.;

  6. toll-free assistance and/or a web-based help desk for any business that has questions about completing the Economic Census of Island Areas; and

  7. systematic mail follow-up for nonresponse, supplemented by telephone follow-up for selected firms. Through these and other response improvement strategies, we expect to maintain or improve the 77 percent check-in rate obtained by the 2012 Economic Census of Island Areas. This level of response when combined with data review and corrections will yield accuracy and reliability that are adequate for intended uses of economic census data; and

  8. respondent portal where respondents can obtain answers to frequently asked questions, send secure emails to Census Bureau staff, report electronically and perform other self-service options without having to pick up the phone.





4. Testing of Procedures


This information collection will use procedures that are based on a considerable body of experience with the Economic Census and surveys along with research conducted for the 2017 Economic Census of Island Areas. Re-engineering economic census data collection instruments for implementation via the Web required a substantial multi-year, multi-method program of research, development, testing and evaluation.


Activities included task analyses, record-keeping studies, exploratory early stage scoping interviews, cognitive testing, usability testing, analyses of electronic paradata, respondent debriefings, and field tests, along with early implementation of designs within other Census Bureau establishment surveys, such as the annual Company Organization Survey (aka Report of Organization) and the Annual Survey of Manufactures. Different research components, in turn, focused on different subpopulations, different instrument components, or different question(naire) content, and results and recommendations were used to aid design decisions.


Of particular note is the use of the annual ASM/COS as a testing and proving ground for many new techniques planned for the Economic Census, as these two surveys collect data at the establishment level like the Economic Census. This began by conducting debriefing interviews with 2014 ASM/COS multi-unit establishment (MU) respondents to identify strengths and weaknesses of the legacy Surveyor software used for electronic reporting to aid development of initial requirements for the online Centurion instrument.


The 2015 ASM provided a full-scale pilot of the new online Centurion collection instrument for single-establishments (SU) using the response-driven design being built for the 2017 Economic Census. Since paper questionnaires were not available, more than 95% of SU respondents reported online, while only a few reported their data by some other method, such as telephone. Additionally, respondent debriefings were conducted and Web paradata were analyzed to further evaluate the response-driven instrument and to recommend improvements.


The 2015 ASM also integrated research testing the value and effectiveness of targeted subsampling, to provide evidence for decisions about implementing an adaptive design approach in the 2017 Economic Census. (See Section 3.)


Finally, MUs in the 2016 ASM/COS are using the new online Centurion collection instrument, which features much of the functionality developed for the 2017 Economic Census, such as importing/exporting spreadsheets, the ability to toggle between “form” views and spreadsheet views of the questions, and questionnaire preview worksheets to aid data gathering. Respondent debriefings and Web paradata analysis will be conducted to further evaluate the instrument prior to conducting the 2017 Economic Census.


Results and recommendations from each component and testing activity were provided to subject matter specialists, managers and developers of data collection procedures, and software developers and programmers. Recommendations were prioritized, negotiated, and revised, as necessary, to facilitate implementation. Nevertheless, some recommendations were not fully adopted due to resource and technical limitations.


All studies are documented in internal Census Bureau reports; a few of the studies are

also reported publicly in conference papers or presentations.




Components of the research and testing, along with selected highlights, follow:

Component

Activity

Dates

Number of Cases

OMB Generic Clearance Date

Description, Selected Results and/or Recommendations

Single Unit Businesses

Task analysis

July 2014

8

June 2014

Response process is relatively straightforward for SUs; however, they rely heavily on paper questionnaires for gathering the data.

Usability testing Round 1

October –November 2014

10

October 2014

Tested a low-fidelity prototype and provided recommendations for a response-driven design for implementation in 2014 ASM. Respondents preferred less information on the screen and skipping questions that were not relevant to them.

Usability testing Round 2

October 2015

9

October 2014

Tested the 2015 ASM response-driven instrument prior to release in December 2015

Respondent debriefings for 2015 ASM full-scale implementation of response-driven design

February – June 2016

23

January 2016

  • Respondents liked response-driven design, answering questions “one at a time.”

  • Respondents wanted a worksheet preview, but did not see this feature prior to beginning.

Analysis of Web paradata from 2015 ASM full-scale implementation of response-driven design

April 2016

NA

NA

  • Identified troublesome screens/questions; reviewed use of features/functions; evaluated indicators of burden.

  • Prioritized changes for 2016 ASM.






Multi-Unit (MU) Businesses

Respondent debriefings for 2012 Economic Census

February 2014

11

January 2014

Debrief respondents about their use of the Census Surveyor instrument in order to identify features that should be maintained and/or improved for the 2017 Economic Census electronic data collection instrument.

Usability testing Round 1

November 2014

10

November 2014

Tested a low-fidelity prototype and gathered requirements for a functioning prototype, and tested initial spreadsheet designs.

Usability testing Round 2

May - June 2015

24

November 2014

  • Performance of SharePoint prototype inadequate and solution abandoned.

  • Feedback on proposed MU instrument interface generally positive for main menu design, introductory screens, and step-by-step instructions for instrument features.

  • Feedback on alternative spreadsheet designs generally positive for incorporating questions’ instructions and improved visual design.

Focus group with NPC Register Analysis Branch analysts

April 2015


NA

Discussed analysts’ experience assisting respondents with Census Surveyor and obtained feedback on plans for revising the MU instrument.

Multi-Unit (MU) Businesses, continued

Debriefing interviews with 2014 COS/ASM respondents

April – June 2015

30

March 2015

Evaluated use of Census Surveyor features / functionality with respondents at small and mid-size multi-unit companies. Corroborated past research:

  • Small MUs preferred within-instrument functions;

  • Mid-size / larger MUs preferred Excel spreadsheet functions;

  • Full capabilities of Census Surveyor overlooked by Rs;

  • Downloaded / printed documents critical for gathering data from other company personnel

Usability testing Round 3

September 2015

17

June 2015

Evaluated basic functionality of MU instrument prototype in Centurion - spreadsheet download/upload, form view, and database that communicates to spreadsheet and form view; navigation and usability of pages; effectiveness of content, especially instructions about reporting procedures; navigation and usability of spreadsheet template; gathered requirements for worksheet/reporting guide

Analysis of Census Surveyor paradata from 2015 ASM

July – September 2016

NA

NA

Although the legacy Census Surveyor instrument was a powerful tool with useful features, Rs were not aware of / did not use quite a few of them.






North American Product Classification System (NAPCS)

Record-keeping study

November 2014

9

August 2014

Explored availability of product line data in respondents’ business records.

Usability testing Round 1

October 2015

9

June 2015

Tested alternative presentations of product lists for SUs; recommended 2-screen design where Rs first select relevant products from a list, then second screen displays only the selected products and requests receipts for each one.

Usability testing Round 2

October 2015

7

June 2015

Tested with Construction SUs and MUs:

  • Prefer 2-screen design over spreadsheets;

  • “Building” and “Non-building” categories helped Rs focus on their construction type

Usability testing Round 3

April 2015

12

August 2014

Investigated the utility of a search feature for looking up products. Rs preferred “write-in” over search functionality, as tested.

Usability testing Round 4

November 2015

9

August 2014

Tested orientation of MU NAPCS spreadsheets; decision to replicate Census Surveyor version with products listed in columns and locations in rows.

Field test of alternative NAPCS question designs and write-ins

January – March 2017

892

December 2016

This will be a field test using an experimental design to examine the effectiveness of two alternative designs and lay-out options for obtaining descriptions of products not explicitly provided in the product lists. Results are expected in late March, and will be used to recommend a design strategy for “write-ins” in the final Economic Census instrument.







MU/SU Centurion Instrument

Usability testing for 2016 ASM/COS Round 1

July 2016

18

June 2016

Tested a limited-functioning prototype. The redesigned instrument performed well overall, but additional instructions and clearer labeling of buttons would be helpful to respondents

Usability testing for 2016 ASM/COS Round 2

October 2016

28

June 2016

Tested a more fully functioning protoype. There were some features and functionality that were not clear to participants, e.g. how to update their locations.

Usability testing for 2017 Economic Census

January – March 2017

TBD

TBD

This will be the first usability testing of complete fully functioning Centurion data collection instrument for 2017 EC.







Web Portal Design and Functionality (eCorrespondence)

Usability testing

July 2016

18

June 2016

Joint with ASM/COS instrument usability testing. Early results did not show major difficulties with using the new Portal.

Usability testing

October 2016

19

September 2016

Overall, the account creation process and linking a survey to that account was not problematic.







Worksheet / Question Preview

2015 COS/ASM Worksheet Debriefings

February – March 2016

25

January 2016

Evaluated use of form/PDFs in current systems, and obtained feedback on worksheet prototype:

  • Form/PDFs play critical role in data collection

  • Most Rs used them (or would have, had they known about them)

  • Positive feedback on prototype worksheet



Previous economic censuses of island areas also have been the subject of evaluation studies that have examined methodology, conceptual issues, and related statistical questions. Cognitive testing was conducted for the 2017 questionnaires intended for use in Guam, the Commonwealth of the Northern Mariana Islands, and Puerto Rico. Approximately 50 companies were interviewed, and the information obtained from the testing influenced the questions asked on the collection instruments. Additionally, substantial testing of content has been done by the stateside economic census. As a result, the procedures used by the 2017 Economic Census of Island Areas are well tested.















5. Contacts for Statistical Aspects and Data Collection


The table below lists the names of the individuals responsible for the collection, analysis, and statistical methodology aspects of the 2017 Economic Census of Island Areas.


Table 5: Census Bureau Contacts for the 2017 Economic Census

Contact

Position

Phone

Responsibility

William C. Davie, Jr

Methodology Director for the Business Register, Economic Census & Related Surveys, Economic Statistical Methods Division

301-763-7182

Development of statistical methodology

Diane Willimack

Methodology Director for Methodology and Response Improvement for Economic Programs, Economic Statistical Methods Division

301-763-3538

Pretesting of data collection instruments and methods

William Samples

Assistant Survey Director for the Economic Census, Economy-Wide Statistics Division

301-763-7175

Overall supervision

Attachments


  1. Electronic Questionnaire Instrument Path Numbers and Titles

  2. Draft Previews of Questionnaires

  3. Questionnaire Information Sheets

  4. Drafts of Initial Contact and Follow-up Letters

  5. Summary of Changes to Questionnaires

  6. Consultations with Representatives of the Island Areas, Persons and Organizations Contacted

  7. Formal Requests of 2017 Economic Census of Island Areas

  8. Electronic Instrument Selected Screen Shots

  9. Quality Standard Waiver


1 The term used to collectively refer to Puerto Rico, the U.S. Virgin Islands, Guam, the Commonwealth of the Northern Mariana Islands, and American Samoa in the Federal Register on June 7, 2011 was Island Areas. Island Areas is common terminology across the Census Bureau.

2 Noise infusion is a method of disclosure avoidance in which values for each firm are perturbed prior to table creation by applying a random noise multiplier to the magnitude data (i.e., characteristics such as receipts, payroll, and number of employees) for each company. Disclosure protection is accomplished in a manner that results in a relatively small change in the vast majority of cell values. For the 2017 Economic Census of Island Areas, each published cell value will have an associated noise flag, indicating the relative amount of distortion in the cell value resulting from the perturbation of the data for the contributors to the cell. The flag for ‘low noise’ (G) indicates the cell value was changed by less than 2 percent with the application of noise, and the flag for ‘moderate noise’ (H) indicates the value was changed by 2 percent or more but less than 8 percent. Cells that have been changed by 8 percent or more are suppressed from the published tables. Additionally, other cells in the table may be suppressed for additional protection from disclosure or because the quality of the data does not meet publication standards. Though some of these suppressed cells may be derived by subtraction, the results are not official and may differ substantially from the true estimate.

3 A reporting unit is an entity from which data are collected. The economic census uses two types of reporting units. The first is the establishment, which is an economic unit usually at a single, physical location where business is conducted or where services or industrial operations are performed. Most businesses report data for the economic census at the establishment level. The second type of reporting unit employed by respondents to answer the economic census is referred to as an “Alternative Reporting Unit (ARU).” ARUs are generally a consolidation of establishments owned by the same company. ARUs are typically used by firms engaged in networked industries such as finance, insurance, or utilities, to facilitate reporting of revenue and expense data.

4 A returned questionnaire includes receipt of an electronic submission authorized by the respondent, receipt of an acceptable response during targeted telephone follow-up calls, or, under special circumstances, respondent-authorized submission by some other means.

5 The tabulating unit houses the data for estimation and tabulation. For the Economic Census, the tabulating unit is the establishment.


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitlePart B
Authorhill0033
File Modified0000-00-00
File Created2021-01-22

© 2024 OMB.report | Privacy Policy