Part B

B. Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the Form OMB 83-I is checked, "Yes," the following documentation should be included in the Supporting Statement to the extend that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The study will implement a sampling strategy designed to produce a nationally representative probability sample of 140 housing authorities. From each selected housing authority, a sample of properties will be selected for a total of 550 properties. Information on conversion, modernization, or demolition strategies for each selected property will be collected. Within each property approximately four buildings and four units will be inspected. The inspected units will be contained in one or more of the four inspected buildings.

To define the sampling universe, the study team used information from HUD’s 2008 public housing inventory file obtained in early June 2008. The file contained 1,205,198 units in 7,404 Asset Management Projects (AMPs). A number of important exclusions were made to the universe file.

• Because the study is intended to estimate the capital needs of developments likely to remain in the stock, units with proposed and approved demolition/disposition plans, completed demolitions/dispositions, or approved HOPE VI implementation grants as of June 2008 were removed from the universe file.

• To eliminate prohibitively expensive data collection costs, the study universe includes only developments located in the contiguous 48 states, the District of Columbia, and Puerto Rico. Therefore, units in Alaska, Hawaii, Guam, and the U.S. Virgin Islands were excluded.

• For the same reason, the study team has decided to exclude units in developments identified in the HUD PIC system as scattered-sites and contain fewer than 1.5 units per building because such developments would be expensive to inspect for the number of units they contain.

• Finally, as was done in the 1998 study, units in Turnkey developments were also removed from the study universe.

Therefore, the adjusted sampling universe for this study includes 1,079,561 units in 6,744 AMPs.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

B2.1 Sampling Plan

The overall sampling approach for this study is a multiple-stage probability sample based on probability-proportional-to-size (PPS) sampling, where size is indicated by the number of units available in the housing authority. Properties, defined in this study as the development components according to pre-asset management reform definition that make up each AMP, also play a central role in the sampling strategy.

Definitions:

• Housing development is defined as it was in the pre-asset management reform world.

• Asset Management Project (AMP) is the grouping of units and buildings that form a new “development”. In many cases old developments have been merged into a single AMP, and in some cases developments have been split into a number of AMPs.

•Property is the component of an old development that moved into a new AMP. For a single development that moved into a single new AMP the development, property and AMP are all the same. For an AMP that is comprised of a number of old developments, the properties in the new AMP are the old developments. In cases where old developments were split into a number of AMPs, the property is the portion of the old development that moved into the AMP.

The selected sample consists of a base sample of 140 housing authorities and a replacement sample of 5 housing authorities. The replacement sample is intended to compensate for both ineligible housing authorities and non-responses.¹² All 145 housing authorities will be pursued, anticipating that the final study sample may be slightly more or less than 140 depending on the actual number of ineligibles and non-responses.

Housing authorities were selected with PPS sampling in multiple stages. There are a total of 2,046 housing authorities in the sampling universe. In the first stage, all the properties with 500+ units were identified. Based on recommendations from HUD staff, the study team further restricted this set of properties to the AMPs where their “lead contributing property” contains 500+ units. The purpose of such restriction was to ensure that developments with 500+ units in both the AMP and pre-AMP definition would be included among the certainty sites.³ There are a total of 162 such developments, spreading across 27 housing authorities. These housing authorities were therefore selected with certainty.

The remaining 123 housing authorities were selected next by defining a new sampling universe of developments by removing the 162 developments identified above as selected with certainty.⁴ Housing authorities were then selected with PPS based on the new sampling universe. However, because some of the large housing authorities would be selected in this stage again with certainty due to their size and they would inevitably overlap with the 27 certainty authorities already selected, the target number of selected housing authorities for this stage could not be determined a priori. Rather, it was determined by a “trial and error” and iterative approach. After a series of trials, a sample of 135 housing authorities at the second stage was found to yield an overall sample of 145 housing authorities (including the 27 housing authorities selected in the first stage). As described above, the target was 145 rather than 140 HAs to allow for some attrition of HAs (due to having ineligible projects, scheduling difficulties or refusals).

In order to ensure representativeness along dimensions of interest to HUD, selection of the non-certainty housing authorities at the second stage were done using 18 sampling strata: 4 Census regions (Northeast, Midwest, South, and West), 4 housing authority size categories (less than 250 units, 250-1249 units, 1250-6600 units, and 6600+ units), New York City Housing Authority, and Puerto Rico Housing Authorities.

Exhibit B-1 compares the sampling universe of housing authorities with the selected sample. As expected, it shows that large and extra-large housing authorities are over-represented in the sample. This result is consistent with the expected probability proportional to size sampling plan.

Exhibit B-1. Description of Sampling Universe and Sample of Housing Authorities

	Sampling Universe		Sample
	Number	Percent	Number	Percent
Housing Authorities by Region
Northeast	413	13.6	36	24.8
Midwest	899	29.5	32	22.1
South	1,510	49.6	64	44.1
West	224	7.4	13	9.0
Total	3,046	100.0	145	100.0
Housing Authorities by Authority Size
Less than 250 units	2,289	75.2	35	24.1
250 – 1,249 units	633	20.8	48	33.1
1,250 – 6,600 units	113	3.7	51	35.2
More than 6,600 units	11	0.4	11	7.6
Total	3,046	100.0	145	100.0

Notes: Excludes housing authorities in Alaska, Hawaii, Guam, and the U.S. Virgin Islands. Unit counts exclude approved and proposed demolitions, those with HOPE VI implementation grants, scattered-sites with fewer than 1.5 units per building, and Turnkey developments.

Sampled properties were selected in a multi-stage framework. The study includes all properties with 500+ units with certainty. The study team has identified a total of 162 such developments in the sampling universe. Over half of these properties are located in New York City (NYC). These properties constitute the certainty portion of the sampled properties and will be inspected according to the standard protocol.

The selection for the remaining 388 non-certainty properties cannot be done based on AMPs because that would yield a sample with more than 388 “properties” to inspect, which would be beyond the cost scope of the study. (For the very small housing authorities, each will be consolidating all their developments into a single AMP in the post-asset management reform world, and many large HAs consolidated former developments.) Nor can the sample focus exclusively on 388 old “developments,” because all future data collection and management at HUD/housing authorities will be at the AMP level. Therefore, for sampling purposes, the selection for the remaining developments will be based on the “properties” discussed above. The properties identify the components of old developments that moved into the new AMPs.⁵

To carry out the selection, the 162 certainty properties were removed from the sampling universe. Next, the sample was restricted to the 135 housing authorities identified at the second stage of the housing authority selection. Developments for each housing authority were sorted by AMP and within each AMP by property. To account for non-responses, the study team determined that a sample of 404 properties was required, which would yield a sufficient number of replacement properties. This implies that picking on average 3 properties from each of the 135 housing authorities. However, given that some of the small housing authorities have fewer than 3 properties, other housing authorities in the list were over-sampled to ensure a final sample of 404 distinct properties. The properties were selected with probability proportion of size.⁶

To summarize, the final list of sampled properties contains 162 certainty AMPs and 404 non-certainty properties, arriving at a total of 566. This allows for a replacement sample of 16 properties.

Exhibit B-2 compares the sampling universe with the weighted sample in unit terms (where the weights are equal to the inverse of the selection probability). It shows that the sample represents the universe of public housing stock very well.

Exhibit B-2. Description of Sampling Universe and Weighted Sample, in Unit Terms

	Sampling Universe		Weighted Sample
	Number	Percent	Number	Percent
Housing Units by Region
Northeast	404,909	37.5	400,380	37.1
Midwest	205,008	19.0	205,121	19.0
South	387,356	35.9	389,288	36.1
West	82,288	7.6	84,771	7.9
Total	1,079,561	100.0	1,079,561	100.0
Housing Units by Authority Size
Less than 250 units	205,085	19.0	216,789	20.1
250 – 1,249 units	304,211	28.2	293,067	27.2
1,250 – 6,600 units	267,439	24.8	269,134	24.9
More than 6,600 units	302,826	28.1	300,571	27.8
Total	1,079,561	100.0	1,079,561	100.0

Notes: Excludes housing authorities in Alaska, Hawaii, Guam, and the U.S. Virgin Islands. Unit counts exclude approved and proposed demolitions, those with HOPE VI implementation grants, scattered-sites with fewer than 1.5 units per building, and Turnkey developments.

Within each selected property approximately four buildings and four units will be sampled at random. (The four sampled units will come from one or more of the sampled buildings). For each selected building/unit, back-ups for inspection will also be sampled to ensure that all inspected buildings/units are selected at random. It is assumed that the randomly selected buildings and units represent all buildings and units in the property. In properties with multiple building types (high-rises, garden apartments, walk-ups, etc.), the sample will include at least one building of each type in the property, which will improve the reliability of the estimates.

B2.2 Estimation Procedures

On-site inspections of physical condition and detailed repair/replacement cost data will be used to estimate capital needs at the sampled properties. These estimates will then be used to provide national estimates of capital need and estimates for particular categories of HAs.

The information recorded by inspectors will be analyzed to produce estimates of repair/replacement costs in two broad categories:

1. Physical needs backlog costs (the costs of the backlog of currently needed repairs and replacements).

2. Estimated accrual costs, the future accrual of major repair and replacement costs.

The estimation of backlog repair costs based on the development inspections will involve seven steps:

1. conducting a physical inspection of the overall site and on average 4 buildings and 4 units within each project in the sample, according to the Observable Systems Methodology;

2. generating a system‑level cost file providing, for each of the over 150 systems inspected, a cost associated with the possible action levels for that system;

3. calculating system‑level costs for the site and inspected units and buildings;

4. inferring costs for uninspected units and buildings from inspected the ones, and using these to generate property‑level costs;

5. regionally adjusting the property ‑level costs;

6. estimating needs for the full stock and for various subgroups of HAs by weighting the cost estimates for the inspected properties; and

7. adding cost estimates for groups of properties and categories of need not included in the observation-based estimation.

Accrual costs are those a development will need to cover expected repairs and replacements for each observable system over each of the next 10 years.

A detailed description of the methodology summarized here is available in the report on Capital Needs of the Public Housing Stock in 1998 and Abt’s 1993 Assessment of the HUD-Insured Multifamily Housing Stock.⁷

B2.3 Justification of Level of Accuracy

This study will generate estimates of the repair needs of the public housing stock that likely will be used by policy makers and HAs for years to come. In order to provide an approximation of the likely range of reliability of the estimates, Exhibit B-4 presents the data from the 1998 study on the mean, standard error and 95 percent confidence interval of the estimates of modernization needs and average annual accrual needs per unit for the stock as whole and various subsections of the stock. Exhibit B-5 presents similar estimates for the overall measures of needs.

These estimates provide an approximation of the expected reliability of the study-generated estimates. It is important to note that the confidence intervals presented in these tables account for the sampling weights, the study sample size, and the size of the public housing stock. They do not account for the complex sample design (i.e. sampling clusters, strata, certainty sites etc). Relevant data elements to calculate the impacts of sampling design were not kept in the analysis file from the 1998 study. The actual estimation of confidence intervals for the current study will fully account for the complex design, and thus would likely yield slightly wider confidence intervals. Similarly, the 1998 sample included 684 developments, while the current sample includes 550 properties. Given that the overall stock size has remained roughly the same, it is expected that the current study will yield slightly wider confidence intervals if the mean estimates and standard errors are similar.

Exhibit B-4. 1998 Average Per-Unit Capital Needs Estimates

Direct Estimates of the Existing Modernization Needs, per Unit
Housing Authority Size	Point Estimate	Standard Error	Standard Error as % of Point Estimate	[95% Conf. Interval]
<250 Units	$13,868	$990	7%	$11,925	$15,812
250-1,249 Units	$17,631	$855	5%	$15,952	$19,311
1,250-6,600 Units	$18,875	$1,043	6%	$16,826	$20,923
6,600+ Units	$21,462	$1,688	8%	$18,147	$24,777
NYC	$23,074	$1,885	8%	$19,374	$26,774
Chicago	$26,184	$2,276	9%	$21,715	$30,653
Puerto Rico	$14,601	$1,965	13%	$10,742	$18,460
All (except NYC, Chicago, PR)	$17,720	$547	3%	$16,646	$18,793
Average Annual Accrual Years 1-20, per Unit
Housing Authority Size	Point Estimate	Standard Error	Standard Error as % of Point Estimate	[95% Conf. Interval]
<250 Units	$1,821	$54	3%	$1,715	$1,926
250-1,249 Units	$1,640	$41	2%	$1,560	$1,721
1,250-6,600 Units	$1,645	$43	3%	$1,561	$1,730
6,600+ Units	$1,554	$74	5%	$1,409	$1,698
NYC	$1,918	$58	3%	$1,803	$2,032
Chicago	$1,346	$64	5%	$1,220	$1,472
Puerto Rico	$1,260	$59	5%	$1,143	$1,377
All (except NYC, Chicago, PR)	$1,668	$25	1%	$1,620	$1,717

Note: Estimates for Puerto Rico differ from those reported in the 1998 report because of additional adjustment made in the report.

Exhibit B-5. 1998 Total Capital Needs Estimates

Direct Estimates of the Existing Modernization Needs
Housing Authority Size	Point Estimate (in millions)	Standard Error (in millions)	Standard Error as % of Point Estimate	[95% Conf. Interval] (in millions)
<250 Units	$2,740.0	$303.0	11%	$2,150.0	$3,330.0
250-1,249 Units	$6,040.0	$493.0	8%	$5,070.0	$7,000,0
1,250-6,600 Units	$5,500.0	$476.0	9%	$4,570.0	$6,430.0
6,600+ Units	$2,610.0	$361.0	14%	$1,900.0	$3,310.0
NYC	$3,610.0	$573.0	16%	$2,480.0	$4,740.0
Chicago	$843.0	$201.0	24%	$449.0	$1,240.0
Puerto Rico	$776.0	$152.0	20%	$478.0	$1,070.0
All (except NYC, Chicago, PR)	$16,900.0	$627.0	4%	$15,600.0	$18,100.0

Average Annual Accrual Years 1-20
Housing Authority Size	Point Estimate (in millions)	Standard Error (in millions)	Standard Error as % of Point Estimate	[95% Conf. Interval] (in millions)
<250 Units	$360.0	$34.8	10%	$291.0	$428.0
250-1,249 Units	$562.0	$39.7	7%	$484.0	$639.0
1,250-6,600 Units	$479.0	$34.5	7%	$412.0	$547.0
6,600+ Units	$189.0	$22.6	12%	$144.0	$233.0
NYC	$300.0	$41.9	14%	$218.0	$382.0
Chicago	$43.3	$9.8	23%	$24.0	$62.6
Puerto Rico	$66.9	$10.3	15%	$46.7	$87.2
All (except NYC, Chicago, PR)	$1,590.0	$42.7	3%	$1,510.0	$1,670.0

Note: Estimates for Puerto Rico differ from those reported in the 1998 report because of additional adjustment made in the report.

B2.4 Unusual Problems Requiring Specialized Sampling Procedures

The transition to an AMP-based public housing inventory (which changes the way developments are defined at many HAs) has significant impacts on sampling and on the analysis for this study. Numerous developments have been merged into single AMPs as part of the conversion to an AMP-based public housing inventory. Many HAs have consolidated old “developments” into new, larger AMPs (asset-management projects), and in a few cases have separated old “developments” into a number of new AMPs.

These changes have significant impacts on the sampling for the study, on the process for estimating capital needs and on the process for comparing needs in 1998 with current needs. The main impact of the transition on the estimation of capital needs results from the impact on sampling—instead of sampling full developments (one site, with a sample of buildings and units), the sample may now include an entity that is a portion of a new development (termed “property” for this study). Thus, the sample will not be used to generate “development-level” estimates, but rather HA- and stock-wide estimates of need.

B2.5 Any Use of Periodic (less frequent than annual) Data Collection Cycles to Reduce Burden.

Not applicable to this study as it requires data to be collected just once.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The target response rate for the inspection portion of this study is to inspect 550 individual properties and for the survey, to complete 140 interviews with HA staff. The selection of HAs and properties for participation in this study was done in a manner that ensures the data collected will be representative of the national public housing stock, as described above.

The contractor’s experience with the similar study in 1998 yielded data from 199 housing authorities out of a selected sample of 205 HAs. Thus, for this study, a replacement sample of five HAs has been drawn to allow for any ineligible housing authorities, or non-response.

In the event that key respondent(s) to the HA survey are unable to answer particular items on the HA survey, missing values will be imputed. In the inspection process the forms will contain a check box to indicate that an item is either not applicable or not observable.

To maximize the response to each data collection component, a complete pre-test of the entire protocol will be done in September 2008 (as described in Section B4 below). The pretest ensures that the data collection forms are easy to use, that the survey questions are clearly worded, and that the burden estimates are real. In addition to pre-testing the entire protocol, each sampled HA will receive a letter from the HUD GTR requesting that they cooperate with the study. This letter will be followed with one from the contractor explaining the study requirements in more detail and also seeking their cooperation.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

The entire set of data collection instruments—both on-site inspection forms and the HA survey—will be tested in September 2008 before the data collection begins. The contractor will pretest the entire inspection process at two different types of developments at each of three HAs. The pre-test sites are Alexandria, VA, Providence, RI and Athens, GA.

This pretest will encompass every step of the process—from inspection through costing reports. The pretest and revision process will be an iterative one. As each pretest is completed, instruments will be revised if necessary before the next pretest. This will enable the project team to make any procedural changes determined to be necessary to improve the process or the quality of the data collected.

A summary of the pretest findings will be presented in the Revised Request for OMB Clearance.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

HUD has contracted with Abt Associates, Inc. to conduct the data collection. The data collection procedures will be similar to those used in other studies involving capital needs assessments conducted by Abt Associates and its partners. HUD also contracted with Abt Associates to design the survey and establish the sample design. The HUD Government Technical Representative (GTR) reviewed all the proposed procedures and had them reviewed by other subject matter experts at HUD. If there are any questions about this submission, please call either Mr. Harold Katsura the HUD GTR at 202-402-3042 or the Abt Associates Project Director Meryl Finkel at 617-349-2380

1 For example, housing development with AMP code AL004000001 is created from two separate developments—AL004001 and AL004006—based on the pre-asset management reform definition. The first development contributes 164 units while the second development contributes 100 units. Therefore, in the sampling scheme, this AMP consists of two properties: 164 units from AL004001 and 100 units from AL004006.

2 In some instances, a sampled HA may turn out to be ineligible after data collection begins. Reasons a HA may be deemed ineligible after the study begins include all the HA’s properties being scheduled for demolition in the current FY or all of the HA’s stock is scattered site. In the 1998 study for example, a small number of HAs originally sampled had HOPE VI plans in place for all their units so they were not eligible for that study.

3 For example, if AMP code NY005000001 (510 units) consists of 5 separate properties: NY005001 (110 units), NY005002 (100 units), NY005003 (100 units), NY005004 (100 units), and NY005005 (100 units),the “lead contributing property” sampling strategy ensured that it would not be considered a certainty selection for the study, given that the number of AMPs that have more than 500 units is significantly higher than in 1998, and higher than the Congressionally mandated sample size. However, each individual property could be selected in the second stage among the non-certainty properties.

4 The study team did not exclude the 27 HAs selected above for this second stage selection in order to potentially study the other developments in these HAs aside from the 162 with 500+ units.

5 One implication of this sampling approach is that statistically reliable “development-level” estimates cannot be generated, but rather HA- and stock-wide estimates of capital needs. There are exceptions. For the 162 certainty developments and any other developments that have not changed configuration after the AMP transition, generating statistically reliable development-level estimates is feasible.

6 The selection probability for the certainty developments was equal to 1. When selecting the properties among the certainty housing authorities, the selection probability was equal to the number units at the property, divided by 2007.307. The threshold was determined in an iterative calculation process. Selection probability for the remaining properties was (6125.648/2007.307)*number of units at the property/number of units at the housing authority (after removing the certainty developments).

7 M. Finkel et al., Capital Needs of the Public Housing Stock in 1998, Formula Capital Study (HUD, Office of Public and Indian Housing March 2000) and James Wallace et al., Assessment of the HUD-Insured Multifamily Housing Stock, Final Report Volume I: Current Status of HUD-Insured (or Held) Multifamily Rental Housing, Appendix C (Cambridge, MA: Abt Associates, Inc., September 1993).

8