1130_BB_BusinessSurvey_partB_ss_020410

3060-1130

February 2010

B. Collections of Information Employing Statistical Methods:

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When item 17 on the Form OMB 83-I is checked, "Yes," the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons)in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The target population for this study is all businesses, both for-profit and non-profit, in the 50 states of the United States. The basic unit in this population is the business organization, which it is a one location or a multiple location business. For multiple location businesses, the sample will attempt to target the headquarters or main office of the business. This definition is closest to the definition of firms in the 2006 Statistics of U.S. Businesses.

The SUSB distributions of those firms by number of employees in 2006 is:

Table 1
Employee size	Firms
TOTAL	6,022,127
0-4	3,670,028
5-9	1,060,787
10-19	646,816
20-99	535,865
100-499	90,560
500+	18,071

This design excludes businesses with less than 5 employees. This is not to ignore the group, but it is a reflection of the practical realities of such businesses. The available sampling frames for businesses with fewer than 5 employees are rapidly outdated as such businesses are established, grow or fail to survive. These small businesses are very flexible in terms of changing location or telephone numbers as businesses needs change. This flexibility translates into a sample frame that produces few usable, working businesses of this employee size that can actually be reached.

Based on recent experience, this survey will achieve a response rate of between 10 and 20 percent using the appropriate American Association of Public Opinion Research (AAPOR) response rate definitions. This response rate is solidly in the higher range of response rates based on telephone surveys with constrained interviewing periods.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

As directed by the FCC, the survey is designed to achieve appropriate national estimates of population parameters of plus or minus 2 percentage points at a 95 confidence level.

The business sample frame will be the Dun & Bradstreet database. Sample will be drawn by Survey Sampling International (SSI) which has access to the D&B data according to PSRAI specifications.

The sample will be drawn to represent all US companies with 5 or more employees and will cover all 50 states. All SIC codes will be included except for those in major groups 91-99 which are government establishments. Sample will be drawn from single-location companies or the headquarters of multiple-location companies.

The sample frame will be divided into four strata based on the company size as measured by the number of employees company-wide. Sample will be disproportionately across strata to ensure that enough small, medium and large companies are in the sample. Table 2 shows the sample frame distribution across. The target sample distribution is the projected distribution of the final sample of 3,500 interviews. These are not quotas and the numbers will differ slightly from those in the tables. We will try and keep the distribution as close as possible to the distribution in this table.

Table 2: D&B Sample Frame Counts
	Company size….
	Small	Medium	Large	X. Large	Total
	(5-25 emps)	(26-100 emps)	(101-500 emps)	(501+ emps)	(5+ emps)
Agriculture, Forestry, Fishing (01-09)	76,509	7,640	898	146	85,193
Mining (10-14)	7,634	1,769	514	204	10,121
Construction (15-17)	245,470	36,996	5,424	586	288,476
Manufacturing (20-39)	162,471	49,711	16,384	5,008	233,574
Transportation, Communication, Electric, Gas, Sanitation (40-49)	92,174	20,572	4,621	1,267	118,634
Wholesale Trade (50-51)	153,439	27,419	5,687	1,091	187,636
Retail Trade (52-59)	424,779	70,847	8,971	1,856	506,453
Finance, Insurance, Real Estate (60-67)	169,613	26,943	6,580	2,057	205,193
Services (70-89)	1,032,005	156,987	39,760	11,677	1,240,429
Total	2,364,094	398,884	88,839	23,892	2,875,709

Table 3: Projected Sample Sizes, Design Effects and Margins of Error
	Projected Sample Size	Projected Design Effect	Projected Margin of Error
Total	3500	1.29	1.9%
Company size…
Small	2000	1.00	2.2%
Medium	700	1.00	3.7%
Large	400	1.00	4.9%
X. Large	400	1.00	4.9%
Industry…
Agriculture, Forestry, Fishing (01-09)	85	1.11	11.2%
Mining (10-14)	15	1.52	30.9%
Construction (15-17)	307	1.15	6.0%
Manufacturing (20-39)	382	1.57	6.3%
Transportation, Communication, Electric, Gas, Sanitation (40-49)	156	1.35	9.1%
Wholesale Trade (50-51)	222	1.25	7.4%
Retail Trade (52-59)	555	1.18	4.5%
Finance, Insurance, Real Estate (60-67)	255	1.32	7.0%
Services (70-89)	1523	1.31	2.9%

The distributions in the D&B database vary from those in the SUSB for a variety of reasons, including the definition of the business unit. The D&B is the best available database covering the entire business community and thus was chosen for this survey.

The average expected length of the interview will be 16 minutes.

The survey will be conducted in English or Spanish, at the respondent’s discretion.

Sample Weighting

The data will be weighted to correct for the disproportionate sample design. Weight will be computed so that the four strata are represented in their proper proportions. Table 2 shows the weights by strata based on the sample design outlined above.

Table 4: Sample Weights
Strata (# of employees)	D&B Frame Dist'n	Target Sample Dist'n	Sample Weight
Small (5-25)	82.2%	57.1%	1.44
Medium (26-100)	13.9%	20.0%	0.69
Large (101-500)	3.1%	11.4%	0.27
Large (501+)	0.8%	11.4%	0.07

Effect of Weighing on Sample Estimates

Disproportionate sampling requires analysis procedures that reflect departures from simple random sampling. PSRAI calculates the effects of these design features so that an appropriate adjustment can be incorporated into tests of statistical significance when using these data. The so-called "design effect" or deff represents the loss in statistical efficiency that results from disproportionate sampling and systematic non-response.

PSRAI calculates the composite design effect for a sample of size n, with each case having a weight, wi as:

formula 1

In a wide range of situations, the adjusted standard error of a statistic should be calculated by multiplying the usual formula by the square root of the design effect (√deff ). Thus, the formula for computing the 95% confidence interval around a percentage is:

formula 2

where is the sample estimate and n is the unweighted number of sample cases in the group being considered. We estimate that the total sample design effect will be 1.29 which will lead to a total sample margin of sampling error of approximately +-1.9 percentage points.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The survey will be conducted in accordance with best practices of the survey industry. Trained and experienced interviewers will be briefed on the project, its goals and the specifics of the questionnaire. A full 15-call design will be implemented for the telephone survey. This means that the field house will make up to 15 calls to each telephone number in the sample in an effort to complete an interview. A telephone number will be removed from the active sample only when a terminal disposition has been achieved for that number (completed interview, non-working number, hard refusal, etc.) Refusal conversion will be attempted at least once on all soft refusals.

Respondents will be offered the option of taking the survey online, should they prefer that mode. For this population, particularly those in the medium and larger sized businesses, internet access is nearly universal and many of the more technically oriented managers will prefer to take the survey online. Offering this option should improve the response rate, at least for those who do not work in small businesses.

Plan to Examine Potential Non-Response bias

In order to quantify and measure potential non-response bias on survey results, we propose investigating two types of non-response. First, we will look at unit non-response. This is non-response at the survey level caused by not contacting potential respondents or not gaining cooperation once a potential respondent is reached. We will also investigate item non-response. This is non-response at the question-level and is caused by people not answering specific questions during the interview.

The approach mirrors the plan for the investigation of non-response bias in the National Consumer Survey being conducted for the FCC now, with OMB Control Number: 3060-1128.

Unit non-response

The first step will segment the completed interviews by the level of effort that was required to get the interview.

For this survey, we will segment the completed interviews into five categories based on the amount of “effort” it took to complete the interview. “Effort” will be measured as a function of the number of calls and refusal conversions necessary to gain cooperation. The business survey introduces a new factor into the analysis of effort: the availability of the online option to complete the survey.

We will start with the following definitions, developed for this project, which may be modified once the data is available:

• Least effort. Online complete without any reminder telephone calls or telephone complete on first call.

• Online complete with one or two reminder calls.

• Online or telephone complete with two to five calls, whether interview attempts or reminder calls.

• Online or telephone complete with five to ten calls, whether interview attempts or reminder calls.

• Most effort. Online or telephone complete with more than ten calls, whether interview attempts or reminder calls.

These effort measures will then be used to compare and contrast key survey results. For example, we will see if the harder-to-reach respondents are less likely to have the highest-speed connections to the internet and less likely to use online applications.

PSRAI will obtain all available parameters for each piece of sample from the D&B database. Using the unweighted data, an analysis of the non-response by each available parameter will be performed. A second wave of analysis will look at the impact of interviewing effort in relationship to each of the parameters.

Item non-response

Item non-response is due to respondents not answering specific questions during the interview. We will investigate unit non-response by identifying which questions have the highest and lowest levels on “DK/Ref” response. We will compare item results with item non-responders both included and excluded and see what effect this has on reported results. We will identify questions that have different levels of non-response by the sample strata.

We can also see if there is a link between item and unit non-response. We will investigate whether respondents who are harder to reach have different levels of item non-response than those who are easier to reach. This analysis will involve comparing levels of item non-response across different kinds of companies we identify in the unit non-response analysis.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

A field test will be held to confirm the questionnaire design, question wording and CATI programming of the surveys.

If the field test is held after OMB approval has been provided, the survey will be field tested in the following manner:

1. The questionnaire will be programmed in the CATI system and the programming checked for accuracy.

2. A small group of interviewers will be briefed on the study.

3. These interviews will then complete up to 20 interviews with actual respondents from the appropriate populations.

4. The interviews will be closely monitored by field house supervisors and debriefed at the end of each shift.

   1. A focus will be on question clarity and assuring that the respondent understands each question.

   2. In addition, the flow of the questionnaire will be judged, based on how respondents deal with changes in topics throughout the survey.

   3. The interviewers will be checked for proper performance and adherence to the survey instructions.

   4. The proper functioning of the CATI system will be checked in real time and the appropriate log files examined for any problems.

Should the field test take place in advance of OMB approval, the field house will conduct the field test using its employees as test respondents. Other than field house staff, the only other individuals who may be contacted for the field test are staff members of PSRAI or the FCC. Other than the change in source of the respondents and a reduction in the total number of interviews, the test will proceed as if members of the public were being interviewed.

PSRAI will prepare a short report on the field tests for the FCC.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

John B. Horrigan, PhD

Consumer Research Director

National Broadband Task Force

202-418-1553 (office)

443-904-2709 (cell)

8