Supporting Statement A SF Survey

Supporting Statement for Paperwork Reduction Act Submissions

General Instructions

A Supporting Statement, including the text of the notice to the public required by 5 CFR 1320.5(a)(i)(iv) and its actual or estimated date of publication in the Federal Register, must accompany each request for approval of a collection of information. The Supporting Statement must be prepared in the format described below, and must contain the information specified in Section A below. If an item is not applicable, provide a brief explanation. When Item 17 of the OMB Form 83-I is checked "Yes", Section B of the Supporting Statement must be completed. OMB reserves the right to require the submission of additional information with respect to any request for approval.

Specific Instructions

A. Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The purpose of this Annual Customer-Service Survey of PBCAs is to enable HUD to decide which PBCAs have earned an Annual Incentive Fee for customer service. This will be the first such survey of the newly awarded PBCAs regarding their monitoring and oversight of HAP contracts.

The information collection for which this OMB approval is requested will consist of a customer satisfaction survey of tenants and property owners that have been overseen by the 11 PBCAs. The surveys will collect information specifically on aspects of PBCAs’ conduct of their monitoring and oversight responsibilities. Those surveyed in each State will consist of: 1) 100 property owners, drawn from a sample of property owners with HAP contracts; and 2) 150 tenants sampled from a list of all Section 8 tenants of selected properties. The responses to survey questions will be analyzed and compiled to form a basis for determining whether each PBCA is eligible to receive an Annual Incentive Fee.

The information collected from these interviews and the results of this analysis will only be used for the purposes of this study, and will not be shared with any other agencies or entities outside of HUD or used for any other purposes.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The purpose of this Annual Customer-Service Survey of PBCAs is to enable HUD to decide which PBCAs have earned an Annual Incentive Fee for customer service. This will be the first such survey of the newly awarded PBCAs regarding their monitoring and oversight of HAP contracts.

The information collection for which this OMB approval is requested will consist of a customer satisfaction survey of tenants and property owners that have been overseen by the 11 PBCAs. The surveys will collect information specifically on aspects of PBCAs’ conduct of their monitoring and oversight responsibilities. Those surveyed in each State will consist of: 1) 100 property owners, drawn from a sample of property owners with HAP contracts; and 2) 150 tenants sampled from a list of all Section 8 tenants of selected properties. The responses to survey questions will be analyzed and compiled to form a basis for determining whether each PBCA is eligible to receive an Annual Incentive Fee.

The information collected from these interviews and the results of this analysis will only be used for the purposes of this study, and will not be shared with any other agencies or entities outside of HUD or used for any other purposes.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The survey of tenants will be conducted via telephone, using contact information obtained from lists of tenants in sampled properties provided by owners or managers of those properties. Since reasonably up-to-date telephone contact information is likely to be available for these tenants, a telephone survey is the most efficient and cost-effective means of reaching this population. Many of the low- and moderate-income tenants sampled likely will not have convenient access to the Web, thereby making Web surveys impractical. Furthermore, with no email addresses likely for tenants, there is no low-cost, practical means of alerting them to a Web Survey. Mail surveys, either with or without inclusion of a URL for Web administration of the survey would achieve a low response rate (estimated at less than 10 percent) and might have appreciable non-response bias.

The survey of property owners will be will be conducted by Web; property owners will be notified of the Web surveys first by a pre-notification letter and then by email from lists of contact information available from PBCAs and/or HUD. Property owners who do not respond also will be followed up by telephone. As business entities, owners and manager access to the Web should be near universal. Their incentive to respond should be high, and with email and telephone reminders, sufficient to minimize non-response bias. Note that, in some cases, the property owner may not be the individual that deals directly with the PBCA. In these cases, we plan to send out the link to the most appropriate individual, such as the property manager or agent. This will be determined when we compile the sampling frame for the survey.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

No prior studies have been conducted for this purpose or similar purposes. HUD and its survey

contractor are not aware of any surveys of owners and managers to assess their satisfaction with the

PBCAs. While surveys of tenant satisfaction have been conducted, they have been for substantively

different purposes.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

Property owners contacted for the survey in their State may be small businesses. Their participation is

essential to achieve sufficient representation of property owners within each State. Sampling

procedures are designed to select small business in proportion to their representation of all property

owners.

There are no known sources of information on the representation of small businesses among these

property owners. Property owners who do respond to the survey may be asked if they are small

businesses, using established Small Business Administration criteria based on revenues or number of

employees.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The information collection proposed for this study consists of one interview for each sampled property

owner or tenants. If these interviews are not conducted, HUD will not have a critical source of

information for determining the eligibility of PBCAs for the Annual Incentive Fees. Owners provide the

best overall source of information on PBCA performance. Since PBCAs are ultimately responsible for

tenant health, safety, and maintenance issues, the inclusion of tenants in the survey is essential.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

There are no special circumstances. The collection of information is conducted in a manner consistent with the guidelines in 5 CFR 1320.5.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, required by 5 CFR 1320.8(d) soliciting comments on the information collection prior to submission to OMB.

N/A Included in Generic Collection.

HUD does not plan any consultation from outside of the agency.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

N/A

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

All respondents in the study will be given assurance that any information they provide is confidential and their participation will have no impact on their receipt of HAP contracts or payments (for property owners) or rental subsidies or ongoing occupancy for tenants. Interviewees will also be assured that no individually identifiable information will be included with any response and the identity of respondents will not be provided to HUD. Further, the key that links any response to any individual will be maintained by HUD’s contractor in a locked file (hard copy) or encrypted (computer file) and destroyed at the conclusion of the study pursuant to applicable HUD regulations. All Insight Policy Research and ICF data collection staff are required to sign a data confidentiality pledge associated specifically with this study.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument for tenants includes selected demographic questions related to housing for background purposes, some of which could be considered sensitive to respondents. These include race/ethnicity, employment status, household size, unit size (# of bedrooms), length of residence at this property, length of time in Section 8 housing (consecutive), and total number of Section 8 housing units lived in (not necessarily consecutive). These will be analyzed to determine if there are any relationships with measures of tenant satisfaction and for categorization purposes.

The survey instrument for property owners will include questions on the business size (number of employees, revenues) and participation in Section 8 (number of properties with Section 8 units, total number of units in this property, total number of units in all section 8 properties, length of participation in Section 8, length of ownership/management of this property) for categorization purposes. Some of these questions could be considered sensitive by certain respondents. The survey instrument is provided in Attachment B.

As part of the consent process, respondents will be assured that they may choose not to answer any specific question(s).

12. Provide estimates of the hour burden of the collection of information.

The burden estimates for respondents are shown in Table A.12.1. The contract covers a single annual survey in each of 11 States. This data collection effort involves a total of 1,650 surveys of tenants (150 in each of 11 States), each of which will take 10 minutes, including screening questions, for a total burden of 275 hours for completed responses, plus 20.65 hours for 413 attempted responses of 3 minutes each.

For property owners, the plan is to conduct 100 surveys per State of 15 minutes each, for a total of 1,100 completed responses from property owners in the 11 States with PBCAs, plus 13.75 hours for 275 attempted responses of 3 minutes each. The table below illustrates the burden level for a single year’s surveys in 11 States.

Table A.12-1. Estimated Hour Burden of the Data Collection

Respondent Type	Type of Instrument	Number of respondents	Frequency of Response	Total Annual Responses	Hours per Response	Total Annual Hour Burden
Tenants	Telephone survey questionnaire	1650 (completed)	1	1650	.167	275.50
		413 (attempted)	1	413	.050	20.65
Property Owners	Web survey questionnaire/ Telephone follow up	1100 (completed)	1	1100	.25	275.00
		275 (attempted)	1	275	.050	13.75
TOTAL		2750	1	2750		584.90

Table A.12.2 below illustrates the estimated annualized cost to respondents for the hours of burden for this data collection. The hourly wage rate used for tenants is the mean national wage rate in May 2010 of 11 selected occupations¹ (slightly more than $11.99) rounded to the nearest whole dollar ($12.00). At an hourly wage rate of $12.00, the total cost for tenants would be $3,306.60 for completed surveys and $247.80 for attempted surveys. For property owners, with an average hourly wage rate of $49.12², the total cost burden would be $13,508.00 for completed surveys and $675.40 for attempted surveys. The table below illustrates the maximum possible cost burden for each of tenants and property owners.

Table A.12-2. Annualized Cost to Respondents

Type of respondents	Type of Instrument	Number of respondents	Hours per Response	Frequency of Response	Hourly Wage Rate	Respondent Cost
Tenants	Telephone	1650 (completed)	.167	1	$12.00	$3,306.60
		413 (attempted)	.050	1	$12.00	$247.80
Owners	Web	1100 (completed)	.25	1	$49.12	$13,508.00
		275 (attempted)	.050	1	$49.12	$675.40
TOTAL		3850		1		$17,737.80

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

No capital/startup or ongoing operation/maintenance costs are associated with this information

collection.

14. Provide estimates of annualized costs to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies may also aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is $426,651. This total includes include the cost of design, instrument development, technical development of survey forms, information collection, analysis, reporting, and presentation/publication of the results. The average annualized cost is $213,325.50.

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I.

This is a new collection of information, so there are no changes to items 13 and 14.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Respondents will be asked to rate, on a 6-point scale, their satisfaction with the PBCA’s overall performance as well as their specific satisfaction on three key customer service functions (health, safety and maintenance). Overall satisfaction scores will be computed for each PBCA in aggregate. Tabulations will be conducted for the percentage of respondents reporting satisfaction ratings (5 or 6 on the 6 point scale) overall and broken down by tenant vs. owners. If sample sizes allow, we also plan to analyze the results to determine whether key variables affect differences in customer-service rating (e.g., urban/rural location, size and age of the property and characteristics of the tenants, such as percent under 100% of the Federal Poverty Level, FPL).

The schedule for data collection, analysis, and reporting is shown in Table A.16-1 below.

Table A.16-1. Project Time Schedule

Activity	Expected Activity Period
Develop Data Collection Instruments	November to December 2011
Receive OMB approval	May 2012
Conduct Surveys	September to October 2012
Data analysis	November 2012
Submit Final Report to HUD	December 2012

.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

N/A

18. Explain each exception to the certification statement identified in Item 19, "Certification for Paperwork Reduction Act Submissions," of OMB Form 83-I.

The agency plans to display the expiration date of OMB approval on all forms/questionnaires associated with this information collection.

B. Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the Form OMB 83-I is checked, "Yes," the following documentation should be included in the Supporting Statement to the extend that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The purpose of this section is to document the statistical procedures to be used for the HUD PBCA Customer Satisfaction Survey. The sampling plan for the PBCA Satisfaction Survey will be probability based so that study findings can be used to make statistically defensible inferences about the entire population of 1) Section 8 property owners (e.g., those who receive rental subsidies through the “project-based” Section 8 rental assistance program) that are administered under the 11 PBCAs awarded for FY2011 and 2) Section 8 tenants of these property owners. Section 8 authorizes a variety of "project-based" rental assistance programs, under which the owner reserves units in a building for low-income tenants, in return for a Federal government guarantee to make up the difference between the tenant's contribution and the rent specified in the owner's contract with the government. Note: Currently, nine States, Puerto Rico, and the U.S. Virgin Islands have awarded PBCA contracts in place, yielding a total of 11 PBCAs.

The sample design for this study will be based on the satisfaction score, which is defined as the average satisfaction scores and the percent of owners and tenants who are almost, or completely, satisfied with the PBCA (e.g., score of 5 or 6 on a 6 point scale). The goal of designing the sample is to permit accurate statements regarding the overall satisfaction levels of building owners and tenants within each PBCA territory, so that results can be compared across the individual PBCAs. Thus, the sampling procedures in this memorandum are designed to measure this overall satisfaction within each PBCA with – at most – a 6.2 percentage point error and a 95 percent confidence level.

The sample for the study was designed to achieve the following goals:

• The use of a three-stage sample design that produces a linkable analytical dataset of property owners, properties and their section 8 tenants;

• The development of a customer satisfaction ratings with a 95-percent, two-tailed confidence intervals of between 2.7and 6.2 percentage points by obtaining a combined sample size of 250 owners and tenants per PBCA (e.g, 100 owners and 150 tenants).³

Steps involved in the sample design are briefly described below.

B.1.1 Target Population. The target population for this survey includes all property owners and

Section 8 housing tenants residing in their properties in the 11 PBCA “states” (e.g., Iowa,

Maine, Minnesota, Montana, New Hampshire, North Dakota, South Dakota, Vermont,

Wyoming, Puerto Rico, and the U.S. Virgin Islands) who have received rental subsidies through

the “project-based” Section 8 rental assistance program under the Housing Assistance

Payments (HAP) Contracts that were administered by the 11 PBCAs in the current fiscal year

(e.g., between October 1, 2011 and September 30, 2012). The 11 PBCAs are the:

• Iowa Finance Authority/EPS;

• Maine State Housing Authority;

• Minnesota Housing Finance Agency;

• Montana Department of Housing (PBS8 Housing);

• New Hampshire Housing Finance Authority;

• North Dakota Housing Finance Agency;

• Puerto Rico Housing Finance Agency;

• South Dakota Housing Development Authority;

• Vermont State Housing Authority;

• Virgin Islands: NTHDC (North Tampa Housing Development Corporation)/CGI; and

• Wyoming: Housing Authority of the City of Cheyenne

B.1.2 Survey Eligibility. This study will include only those property owners and tenants who have

received rental subsidies through the “project-based” Section 8 rental assistance program

for which payment had been made under the Housing Assistance Payments (HAP) Contracts

that were administered by the 11 PBCAs between October 1, 2011 and August 31, 2012

of the past fiscal year. Note: The lists must be collected by August 31^st, in order to conduct

the surveys during the month of October. Managing agents may serve as proxies for the

property owners if they have had the most contact with the PBCA.

B.1.3 Sampling Frame. The sampling frame will be developed in three phases, as described

below.

Stage 1: Property Owner Sampling Frame. First, the sampling frame of property owners

will include a list of all property owners who have received rental subsidies

through the “project-based” Section 8 rental assistance program for which payment had been made under the Housing Assistance Payments (HAP) Contracts administered by the 11 PBCAs between October 1, 2011 and August 31, 2012 (including both owners who have had direct contact and owners who have not had direct contact with the PBCA in the past year). The frame will include contact information for each property owner, including email address, and will indicate the date(s) of contact, if applicable, by a PBCA. The files will be obtained both from internal files maintained by the HUD regional offices and the PBCAs themselves. Note: Each property owner will be listed once on the frame, so that each owner will have the same probability of selection for the survey of property owners.

Stages 2 and 3: Tenant Sampling Frame. For each of the sampled property owners identified in

Stage 1 above, we will obtain lists of each of the properties that they own for which they had received rental subsidies through the “project based” Section 8 rental assistance program for which payment had been made under the Housing Assistance Payments (HAP) Contracts that were administered by the 11 PBCAs between October 1, 2011 and August 31, 2012. This is called Stage 2 of the frame development process. These lists will contain property addresses and the number of tenants in each property. We plan to obtain these lists both from internal files maintained by the HUD regional offices and the PBCAs themselves. Note: This step is necessary for compiling a sampling frame for tenants, as it is not feasible to obtain tenant lists from all property owners. Each of the sampled owner’s properties will be listed once; as such, owners may have multiple properties selected for the tenant sample (see sampling procedures in Section B.1.4 below).

In Stage 3 of the frame development process, for each of the sampled properties identified in Stage 2 above, we will obtain lists of all Section 8 tenants residing in each of the sampled properties. To collect these lists, property owners whose properties were selected in the second stage of sampling will be asked to provide a list of their Section 8 tenants.⁴ The lists of tenants must contain accurate and up-to-date contact information for each tenant.

B.1.4 Statistical Methodology for Stratification and Sample Selection.

Stage 1: Property Owner Sample Selection. The property owners will be stratified into

eleven primary strata based on PBCA. Next, each of the eleven primary strata will be further divided into 3 substrata. The three substrata will be defined by whether the property owner has less than 75 section 8 assistance units, 76 to 150 section 8 assistance units or more than 150 section 8 assistance units. This will yield a total of 33 substrata (11 PBCAs times 3 site categories), or two substrata per PBCA.

The sample size will be allocated equally among the PBCAs to achieve the desired precision of the estimates in each of the 11 PBCAs. Thus, with an overall sample size of 1,375, the sample size allocated to each PBCA would be 125 property owners.

Prior to selecting the sample within each PBCA, a sample allocation program will be run to determine the sample sizes within each of the substrata. Property owners will be allocated to each substratum in proportion to the size of that substratum (defined by the sum of all property owners in that substratum). The benefits of this procedure include the fact that all weights are exactly the same; as such, there is no ‘oversampling’ of certain strata causing variation in the weights. As a result, the variance of the overall satisfaction estimates for owners is smaller than would be otherwise. Note: if a PBCA has less than 125 property owners, all owners in the PBCA will be contacted for the study.

After the sample size of 125 is allocated within each PBCA substratum, the property owners will be sorted within substratum by zip code and total section 8 assistance units before sampling to ensure a representative sample within these groups. We will then perform systematic sampling within strata. This method involves numbering the property owners in the population from 1 to N (N= total records in population). To select a sample of n owners, we take an owner at random from the first k owners and every k’th owner thereafter until the appropriate number of property owners is achieved in the stratum. In this way, each property owner on the sampling frame will be given a known, nonzero probability of selection so that weighted inferences can be made about the entire population of property owners.

Assuming an 80 percent response rate and a 100 percent eligibility rate among selected property owners, we plan to select approximately 1,375 property owners to participate in this survey (or 125 property owners per PBCA). Based on the anticipated response rate, this will yield 1100 property owners (or 100 property owners per PBCA).

Stage 2: Property Sample Selection. For the second stage of selection, a list of individual

properties will be compiled for each of the 1,375 selected property owners selected in Stage 2. Within each BPCA, the properties will be selecting using Probability Proportionate to Size (PPS) sequential sampling procedures. Properties will be ordered and selected using the total number of Section 8 tenants in the property as the measure of size. PPS sampling methods will ensure that the larger properties will be included in the tenant sample. This procedure will be used to select a sample of 110 properties for a more detailed survey of Section 8 tenants (a total of 10 properties in each of the 11 PBCAs). After sampling, we will run the same sampling program a second time to select a sample of “replacement” properties, where each “replacement” property is matched to a “primary” property. The replacement property will be substituted for the primary property if an owner refuses to provide a listing of the section 8 tenants for sampling below.

Stage 3: Tenant Sample Selection. For the third stage of sample selection, a list of tenants

will be compiled for each of the 110 properties selected in Stage 3. Within each

PBCA, the tenants will be sorted by property before sampling to ensure a representative sample across all properties in the PBCA. To ensure that tenants have equal probabilities of selection across both Stage 2 and Stage 3, a sampling “rate” will be developed for each property, so that tenants in large properties will be selected at lower rates than tenants in smaller properties. This rate will be computed separately for each property as the desired sample size for tenants (in the PBCA) times the weighted proportion of tenants in the property (e.g., number of tenants times Stage 2 property weight over the sum of this weighted proportion for all sample properties). We will then randomly sample tenants within each property by selecting a tenant at random from the first k tenants and taking every k^th tenant thereafter until the appropriate number of tenants is achieved in the property. In this way, each tenant in the survey will be given an equal, nonzero probability of selection.

Assuming an 80 percent response rate from selected tenants and a 95 percent occupancy rate of section 8 tenants, we plan to select approximately 2,200 tenants to participate in this survey (or 200 tenants per PBCA; or approximately 20 tenants per property).

B.1.5 Response Rates. Our goal is to achieve an overall response rate of 80 percent for both owners

and tenants. We feel that this is a likely response rate for this survey as it uses a proven data collection methodology (telephone survey with follow-up of non-respondents), contains a salient subject matter and reduces the respondent burden by keeping the questionnaire length to a minimum.

B.1.6 Reliability of Estimates. With an overall respondent sample of 2,750, (1100 owners or 100 per

PBCA and 1,650 tenants assuming 150 per PBCA) we propose a sample size of 250 respondents for each of the 11 PBCAs. With this sample size, 95-percent confidence intervals (CIs) are expected to be less than ± 6.2 percent for percentages generated for each PBCA. For example, assuming that the respondent sample size is 250 for any one State and the percentage of property owners who were satisfied with their communication is 50 percent, then using a 95 percent confidence interval, in 95 out of 100 samples like the one selected the results should be no more than 6.2 percentage points above or below this figure.

In addition to making survey comparisons between the PBCAs, estimates of satisfaction will also be computed for property owners within each BPCA. Obtaining a sample size of 100 completed property owner interviews within each PBCA, except for the Virgin Islands and Wyoming, which contain 12 and 59 properties respectively), will yield an average coefficient of variation (CV) of 10 percent for estimates of “percent of owners satisfied” by property owners; and 95-percent, two-tailed confidence intervals of between 4.3 to 9.8 percentage points.⁵

We plan to select approximately 3,575 sample cases (1,375 property owners and 2,200 tenants) to reach this goal. Follow up CATI methods will be used to ensure that the response rate goal of 80 percent is achieved.

B.1.7 Estimation Procedures. Following data collection, sample weights for property owners will be: 1)

prepared based on the initial probability of selection, 2) adjusted to compensate for owner nonresponse, and 3) edited to remove multiple selection opportunities. The end product will be final analysis weights suitable for use in analysis of property owners satisfaction scores. This weighting scheme inflates the respondents' data to represent the entire universe of property owners in the PBCAs.

Following the final sample weights for property owners, the sample weights for tenants will be prepared based on 1) the initial probability of selection for the owners, 2) the initial probability of selection for the properties, 3) the initial probability of selection for the tenants and adjusted to compensate for tenant nonresponse. The end product will be final analysis weights suitable for use in analysis of tenant satisfaction scores. This weighting scheme inflates the respondents' data to represent the estimated universe of tenants across the PBCAs.

Although data management and simple cross-tabulations will be conducted using SAS v9.2, we will use SUDAAN v9.0.1 for standard errors and tests of significance. SUDAAN provides the correct computations for the standard errors by accounting for the design of the sample. Various multivariate and descriptive statistical techniques will be used to analyze the data, including cross tabulations and frequency distributions, t-tests, chi-square tests, and regression analyses including logit, multinomial logit and least squares methods. Direct variance estimates that reflect the sample design will be computed for each analysis variable, and will be used in all analytic comparisons of final results.

2. Describe the procedures for the collection of information.

The surveys will be administered to two target customer groups including property owners and tenants. Each of these is described below.

B.2.1 Data Collection for Property Owners. For property owners, a web-based satisfaction survey will be

conducted in each of the 11 PBCAs to obtain information on the satisfaction of owners with the PBCAs. We believe that Web administration is the most efficient means to reach this group for a few reasons. First, Web-based surveys are ideally suited for persons in management/ownership positions managers/owners have identifiable email addresses as they are used to administer the HAP contracts themselves. Second, Web-based surveys have the advantage of reducing measurement error and resulting bias. The absence of an interviewer reduces the possibility of obtaining socially desirable responses—the tendency on the part of the respondent to give what they interpret as the socially correct answer. Third, properties owners may be more motivated to respond because of their regular interactions with the PBCA.

Our plan is to send the email notifications to all owners selected for sample. To make certain that owners do not ignore the initial email request (or accidentally delete it), we will alert them to its arrival with a pre-notification letter from HUD for delivery about 1 week prior to the email. We will prepare this letter on stationary with an official HUD logo obtained from HUD; we will then print and mail the letter in envelopes with a similar logo. Attachment A illustrates a copy of this letter. Then, we plan to contact any nonrespondents for telephone follow-up to ensure that a minimum of 100 completed interviews is obtained.

About 1 week after receipt of the pre-notification letter, we will send an email to the potential survey respondents identified by the HUD as an owner/manager that includes a customized link. This link will include an embedded password which may be used to complete the survey once. Email notifications are used as an initial contact with potential respondents, to explain the purpose of the survey, to elicit cooperation, and to communicate a secure link as well as a unique ID and password that the respondent will use to complete the survey. Specifically, the initial email will include the following items:

• The purpose of the survey,

• A statement of how the results will be used,

• A request for the respondent’s participation,

• A statement of promised confidentiality or anonymity,

• Detailed instructions for accessing the survey including a hypertext link address and a unique password for each respondent,

• The cut-off date for responses,

• Instructions to decline participation,

• A phone number and email address to use for technical support (provided by ICF Macro), and

• A phone number and email address for the HUD contact if the respondent has questions about the study’s validity.

Respondents may either click on the URL directly from the message or “copy and paste” the address into their Internet browser. The password feature serves several functions, specifically it:

• allows participants to begin the survey, suspend it, and re-enter the survey later at the point where they left off; the responses already entered will be saved;

• protects a participant’s data against power or network interruption, since responses are saved after each screen and participants can simply re-enter the survey using their password and begin where they left off;

• ensures that only targeted respondents can complete the survey;

• safeguards the survey process against “ballot-box stuffing” by preventing multiple surveys (i.e. since each password can be used to complete only one survey).

The survey will be posted to a secure Web site owned and maintained by ICF Macro so that owners can submit survey responses electronically. ICF Macro has its own state-of-the art Internet data collection software and complete sample management system, which includes automated email invitations and reminders, including SPSS Dimensions. This software suite allows the programmer to quickly and easily create professional-looking surveys that are easy to navigate and flexible enough to make quick modifications as needed. SPSS Dimensions has extremely high levels of customizability to allow for the programming of any skip patterns and survey logic. The web servers are available 24 hours a day, 7 days a week, so respondents may complete the survey at any time that is convenient for them. Respondents who have questions about the survey are offered up to three methods of support: 1) Frequently Asked Question (FAQ) pages (which will be included in the questionnaire) and technical support to ensure that participants can access the system and complete the Web survey as intended; 2) a dedicated email address that is accessed by multiple team members to ensure timely response; and 3) a toll-free phone number (with voicemail) for participants to call to speak directly with a help-desk team member.

Once participants reach the Web survey via the secure link contained in their email invitation, they can begin to complete the survey.

Follow up Methods - Reminder Emails and CATI. We plan to use proven methods to increase response rates for the survey, including multiple contacts, personalization, incentives, and guarantees of confidentiality. If respondents do not complete the Web survey within a specified timeframe, additional reminder emails are sent. We will issue an initial email invitation, followed by up to two email reminders to nonrespondents, to maximize response rates. The reminder emails are very similar in content to the initial email invitation, restating the survey cut-off date and further emphasizing the importance of everyone’s participation. We will work jointly with HUD to develop the appropriate email invitation text for a schedule of reminder emails that works with the overall timeline available for Web survey fielding. Any respondent who completes the survey via Web is identified as complete and removed from subsequent follow-up contact.

If there is still no response within approximately 1 week of the second email, we will begin telephone follow-up. We will make up to three telephone calls to the owner or manager, requesting them to complete the survey. If we do not reach the sampled person in these attempts, we will leave a voicemail reminder. If we do reach the respondent live and can convince that person to complete the survey by telephone at that time, we will administer the survey by phone and enter the response into the Web survey form directly. If owners/management agents prefer to use the Web version, they will still have the option to do so. Insight estimates that

approximately 40 percent of the owners/management agents will respond to the survey after the pre-notification letter, an initial email, and reminder emails. The telephone follow-up will double the estimated response rate for a total response of 80 percent.

B.2.2 Data Collection for Tenants. The proposed tenant survey is designed as a telephone survey using

Computer Assisted Telephone Interviewing (CATI) with non-response telephone follow-up to obtain information on the satisfaction of tenants with the PBCAs. Tenants may have little or no awareness of the PBCA, but they are in a position to provide feedback about their interactions with the owners/building managers and their operation of the buildings. Since the PBCAs provide standards for the contracts they administer, they are ultimately responsible for the performance of those they monitor. We recommend utilizing CATI for this target population for several reasons. CATI is an efficient way to reach a substantial number of respondents, where the sampling frame is sufficiently large and the contact information is adequate to provide a reasonably high response rate. Phone numbers are likely to be accurate given our assumption that owners/management agents maintain current telephone contact information for their tenants. Still, we acknowledge that cell-phone numbers, which may be the primary telephone of many of these tenants, are less stable and more likely to change than landlines. However, based on our experience with low-income populations; we find that those living in medium-to-small communities, which dominate in these PBCA States, tend to change cell-phone numbers much less frequently than those living in larger communities. When we find that telephone contact information is inaccurate, we plan to utilize standard locating and “skip tracing” procedures as available. The use of CATI offers several advantages that can shorten the data collection period. For example, call attempts can be scheduled to maximize the chances of reaching the intended respondent, and interviewers can often obtain immediate locating information when the contact information on file is incorrect.

We believe that, overall, CATI would yield a higher response rate for this study than other modes of data collection, with an anticipated response rate of 80 percent for the final results. In addition, CATI will improve the quality of the data by ensuring that the most knowledgeable respondent is interviewed for the survey. The data collection methodology is as follows:

• The CATI instrument will be developed, tested and programmed to assign interim and final status codes to track refusal, ineligible, and unlocatable cases;

• A survey management system will be programmed to track completed cases, partially completed cases, call history, and locating history;

• A training program will be developed and interviewers will be thoroughly trained on all aspects of the study.

• Screening questions will be used to located the most knowledgeable respondent for the survey (for example: the adult in the household who most often deals with maintenance, rental issues, etc.);

• Tracing efforts using commercial locating databases will be done to obtain updated phone numbers for non-respondents

• Response rates will be monitored and analyzed by completed cases by time of day and days of the week to optimize calling times; and

• Refusal conversion calls will be made by specialists trained in refusal conversion.

Follow-up Methods. We propose a multipronged strategy for ensuring strong response rates, including 1) obtaining the most current contact information from property administrative records; and 2) use of respondent-locating techniques⁶ as needed. Insight will design the materials and interview scripts to convince sample members that the survey feedback will provide some benefit to them as tenants and that their rental assistance benefits will not be affected by their responses; we will also design the scripts so they are not too lengthy. Our call center interviewers are trained in refusal-conversion techniques and will utilize a wide range of methods to minimize nonresponse and maximize the complete data available for analysis. Procedures to maximize the response rate include the following:

• 7-attempt protocol on different days/at different times of day. Research shows that the incremental increase in response rates diminishes beyond seven calls. Messages will be left for recipients to call a toll-free number to complete the survey. After seven attempts to reach a number, a replacement number will be used if available.

• Call rotation and flexibility. The CATI system can schedule calls to rotate among various times throughout the day and evening during callbacks. The system allows respondents to call in to complete a survey or continue a survey over multiple sittings. Interviewers can also schedule appointments so that respondents can participate at a time convenient to them.

• Refusal conversion. ICF Macro will work with Insight to implement refusal conversion appropriate to the needs of the project. The level of conversion will be communicated to interviewers as part of the training.

The survey is cross-sectional. No future contacts are planned after a completed questionnaire is returned and/or the interview is completed by phone.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The methods described above have been proven in methodological research to yield response rates of 80 percent when the survey is of reasonable length and sample members consider the topic salient. The following strategies will be used to help achieve this response rate for both owners and tenants, unless otherwise noted:

• Personalized pre-notification letters (owners only);

• Strategically scheduled follow-up attempts;

• Survey sponsorship by a recognized Federal agency;

• A brief introduction that underscores the saliency of the survey topic for sample members;

• Interviewer training that addresses potential obstacles in reaching or communicating with owners and

tenants and offers strategies for overcoming these obstacles;

• A toll-free number for respondents with questions.

• Locating efforts using commercial locating databases and directory assistance in an effort to

obtain updated phone numbers for unreachable sample members.

The pre-notification letter will be printed on HUD letterhead and will briefly explain the purpose of the study and the reasons why sample members should volunteer their time. The letter will also include the estimated completion time of the survey, and assurances of confidentiality. Stating the sponsorship of the survey helps to engage sample members by providing immediate assurance that the survey is legitimate and not an attempt to sell them something. The likelihood of acceptance is greatly increased when sample members are told early why the survey is being conducted and why their responses are important.

Prior to data collection, efforts will be made to identify the most appropriate respondent. For owners, the selected respondent should be the person most knowledgeable about the owner’s experience with the PBCA. The most knowledgeable person could be a property manager or agent. A set of screener

questions will be added to the beginning of the questionnaire to identify which person is the most appropriate respondent. For tenants, the selected person will be the adult with most interaction concerning rental issues.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

A pretest was conducted with potential respondents in Virginia, during which the questions were tested and additional issues identified. Many of the changes in instructions, and question wording were made as a result of these interviews. The telephone data collection procedures themselves have been well-tested on tenants.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Sid Groeneman, Ph.D.

Groeneman Research and Consulting, Inc.

9205 Laurel Oak Drive

Bethesda, MD 20817

301-469-0813

[email protected]

1 Healthcare support, security guard, building/grounds cleaning and maintenance, food preparation and serving, personal care and service, cashier, retail salesperson, office clerk, store clerk, construction trade helpers, and taxi drivers

2 The mean hourly wage for owners was determined using national data for “Managers, All Other” as reported for May 2010 by the Bureau of Labor Statistics.

3 The lower bound of this range reflects the 95-percent confidence interval when the population mean of a binary variable is 10 or 90 percent; the upper bound when it is 50 percent.

4 The second and third stages of the frame development process are for the tenant survey only.

5 The loss of effectiveness by the use of cluster sampling of tenants, instead of simple random sampling across all properties in the PBCA, may preclude us from estimating results for tenants within the PBCA. To estimate the approximate effect on variances, we calculated a design effect (e.g., the ratio of the actual variance, under the sampling method actually used, to the variance computed under the assumption of simple random sampling). The magnitude of the design effect depends on the size of the clusters and on the internal homogeneity of the clusters (intra-class correlation). For our calculations, we have assumed an average building size of 50 assistance units and an intra-class correlation of 0.05. Obtaining a total of 150 completed tenant interviews within each PBCA will yield an average coefficient of variation (CV) of 14.0 percent for customer-satisfaction ratings for tenants in the State and 95-percent, two-tailed confidence intervals of between 6.1 to 15.0 percentage points.

6 We will use locating databases such as LexisNexis and residential telephone listings to locate sample members based on names, addresses, current or former telephone numbers, and/or other identification numbers.