ICR 2528 Part A revised 8-19-08

Cambridge, MA

Lexington, MA

Hadley, MA

Bethesda, MD

Chicago, IL

Assessing Quality of Life Issues in FEMA’s Alternative Housing Pilot Program (AHPP)

Final

OMB Paperwork Reduction Act Submission for

Household Outcomes Surveys

May 14, 2008

Revised: 8-19-2008

Prepared for

Harold Holzman, GTM

US Department of Housing and Urban Development, PD&R

451 Seventh Street, SW, Suite 8140

Washington, DC 20410

Prepared by

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138

T able of Contents

Part A Justification 1

A1 Circumstances That Make the Collection of Information Necessary 1

A2 How and by Whom the Data Will Be Used 2

A2.1 Project Overview 2

A2.2 Purpose of the Data Collection 2

A2.3 Who Will Use the Information 2

A2.4 Instrument Item-by-Item Justification 2

A3 Use of Improved Technologies 6

A4 Efforts to Avoid Duplication 6

A5 Involvement of Small Entities 6

A6 Consequences of Less Frequent Data Collection 7

A7 Special Circumstances 7

A8 Consultations Outside the Agency 7

A9 Payments to Respondents 8

A10 Arrangements and Assurances Regarding Confidentiality 8

A11 Sensitive Questions 8

A12 Estimate of Annualized Burden Hours 8

A13 Estimated Record Keeping and Reporting Cost Burden on Respondents 9

A14 Estimated Cost to the Federal Government 9

A15 Reasons for Changes in Burden 9

A16 Tabulation Plan, Statistical Analysis, and Study Schedule 9

A17 Expiration Date Display Exemption 11

A18 Exceptions to Certification 11

Part B Statistical Methods 12

B1 Potential Respondent Universe 12

B2 Statistical Methods 12

B2.1 Sampling Plan 12

B2.2 Justification of Level of Accuracy 14

B3 Maximizing Response Rates 15

B4 Tests of Procedures or Methods 16

B5 Statistical Consultation and Information Collection Agents 17

Appendix A: Household Outcomes Survey – Experimental Version

Appendix B: Household Outcomes Survey – Non-Experimental Version

Appendix C: Household Outcomes Survey Item by Item Justification – Experimental Version

Appendix D: Household Outcomes Survey Item by Item Justification –Non-Experimental Version

Appendix E: Household Outcomes Study Advance Letter and Tracking Letter

Appendix F: Federal Register Notice for Proposed Information Collection

Part A
Justification

A1 Circumstances That Make the Collection of Information Necessary

More than two years after the devastating Hurricanes Katrina and Rita in 2005, thousands of families in the Gulf region are still living in travel trailers or other temporary housing provided by the Federal Emergency Management Agency (FEMA). The Alternative Housing Pilot Project (AHPP) provided FEMA funds to four states (Alabama, Louisiana, Mississippi, and Texas) for demonstration programs to test alternative housing types that could be used in place of FEMA trailers or mobile homes following a future disaster. AHPP grantees will offer displaced households living in FEMA-assisted housing a variety of types of alternative housing across the four states. Recipients may be able to place the housing unit on land they own, or it may be located in a group site, a multifamily structure, or a subdivision. Some grantees will offer the option to purchase the unit after a period of time, while for others the housing will strictly be temporary.

FEMA has provided funds to the Department of Housing and Urban Development (HUD) to develop and implement a Household Outcomes Survey of AHPP participants. The study will allow HUD and FEMA to assess the affect of the alternative housing on participants’ quality of life, including changes in the quality of their housing, health and mental health outcomes, and economic and employment situations. The findings will inform future decisions about post-disaster housing strategies.

The Alternative Housing Pilot Program is authorized under Section 2403 of Public Law 109-234, enacted on June 15, 2006. This data collection activity is authorized under [12.U.S.C. 1701z-1], which reads as follows:

TITLE 12--BANKS AND BANKING

CHAPTER 13--NATIONAL HOUSING

Sec. 1701z-1. Research and demonstrations; authorization of

appropriations; continuing availability of funds

The Secretary of Housing and Urban Development is authorized and

directed to undertake such programs of research, studies, testing, and

demonstration relating to the mission and programs of the Department as

he determines to be necessary and appropriate.

A2 How and by Whom the Data Will Be Used

A2.1 Project Overview

The AHPP Quality of Life (QOL) study will assess the outcomes of a demonstration program to test alternative approaches to providing housing after a disaster. Baseline surveys will be administered by grantees or HUD during each grantee program’s start-up period, using an instrument developed by HUD and previously approved by OMB (OMB Control Number 2528-0248). A Household Outcomes Survey will be administered twice to participants in each state between 2008 and 2010 to capture data on outcomes. The survey will be conducted using a mixed mode approach, telephone interviews with in-person follow-up for non-responses or respondents that prefer a face-to-face interview.

A2.2 Purpose of the Data Collection

The goal of the QOL study is to provide FEMA and HUD with a rigorous evaluation of the quality of life outcomes for AHPP program recipients in the four states that received AHPP grants. Over the four-year study, the evaluation will assess a range of outcomes, including households’ economic and employment situations, physical and mental health status, and housing quality and satisfaction.

In Louisiana, the evaluation will use an experimental design because the number of applicants is expected to be substantially higher than the number of AHPP units produced. Louisiana applicants will be randomly selected to receive the units through a lottery. For the survey sample, we will select a random sample of people who receive a unit in the lottery (the treatment group) and a random sample of people who apply, but do not receive a unit in the lottery (the control group). This experimental design will allow us to compare outcomes for households that are offered AHPP assistance compared to otherwise similar households that are not offered assistance. This design provides the most definitive estimates of the impact of receiving an AHPP unit on the recipients’ quality of life. In Mississippi, Texas, and Alabama, there is not enough excess demand for the program (i.e., not substantially more eligible applicants than units) to permit a random assignment study. In these states, the study will use a pre-post approach; that is, we will look at outcomes for participants after their AHPP participation compared to their pre-AHPP situations.

A2.3 Who Will Use the Information

FEMA and HUD will use the study’s findings to inform future decisions about housing strategies following disasters.

A2.4 Instrument Item-by-Item Justification

Two survey instruments have been developed: one for subjects in the experimental study and one for those in states where the non-experimental, pre-post design will be implemented. The survey instruments are attached in Appendices A and B.

Exhibit 1 shows the domains covered in the two versions of the Household Outcomes Survey and the justification for the questions in each domain. The majority of questions on the two instruments are the same and the exhibit indicates when there are differences between the experimental and non-experimental versions of the survey. The primary differences between the experimental and non-experimental versions involve minor wording changes.

More detailed justifications of the survey questions are contained in the appendices. Appendix C provides item-by-item justifications for the questions in the Experimental Version of the Household Outcomes Survey. It shows not only the content and reason for including each question, but also the source of the survey question. Appendix D shows the same detail for the Non-Experimental Version of the Household Outcomes Survey.

Exhibit 1. Survey Domain Justification

Survey Section	Domains Covered	Justification	Difference Between Experimental and Non-Experimental
A: AHPP Unit	Housing quality	This section asks about the respondents AHPP unit. After establishing when the participant moved into the unit and if the household is still living in the unit, all respondents are asked specific questions about the impact of their AHPP unit on their quality of life and about the AHPP unit features that are important to them. If the participant has already left their AHPP unit, it asks about the reasons for moving out and the quality of the AHPP unit. It is important to ask questions about the quality of the AHPP unit for those who already moved out to make sure the information on AHPP units covers both people who live in them a short time and people who live in them for a long time.	The experimental version skips control group members out of this section after A2.
B: Current Housing and Housing Quality	Housing quality Mobility Permanent housing plans Housing security Rent/mortgage payments and hardship Utility payments and hardship Unit Maintenance and Repair Household composition	This section allows us to measure the participant’s satisfaction with their current unit (AHPP or non-AHPP), identify households where overcrowding is an issue, examine financial hardship issues related to rent/mortgage and utility payments and how they differ for those living in an AHPP unit and those who do not. It includes the housing quality questions that were used in HUD’s baseline survey so that pre-post AHPP comparisons can be made. We also have questions about routine maintenance and repairs to understand how well the units are managed and maintained. To measure whether or not receipt of an AHPP unit allows families to ‘reconnect’ for all current household members we ask if they were living with the respondent at the time of the hurricane and at the time they applied. This allows us to track household member movements over time in a simple format to understand the role of AHPP on household composition.	In the experimental version, where questions ask specifically about AHPP units, CAPI will either substitute wording changes to ask about current unit for control group members or skip them out of that question.
C: Quality of Life	Physical health	The physical health questions focus primarily on the aspects of physical health that are likely to be affected by a housing unit—physical disabilities and ease of accessibility and breathing problems, such as asthma. The breathing problem series is asked of the head of household, other adults and any children in the household. We restrict the series to whether there was a diagnosis, when the diagnosis occurred, and if there was improvement after AHPP unit receipt. It also asks about smoking habits because of the relationship between smoking and physical health and stress. Finally, we ask height and weight for the head of household to compare to baseline measures. Height and weight can be used to measure obesity.	Both versions determine when the diagnosis was received in relation to the AHPP unit occupancy. The Experimental version adjusts for AHPP versus Current unit by asking treatment group members about AHPP unit and controls about current unit.
	Emotional health	This section contains the same emotional health questions that were in HUD’s baseline survey so pre-post AHPP comparisons can be made. We also use the PTSD PCL module to measure Post Traumatic Stress Disorder to measure PTSD and the change in PTSD for recipients of AHPP. This 17 question module is the shortest module that yields reliable results. We use the full module to ensure that the results retain the high level of reliability and leave open the opportunity to compare results against national norms.	No differences between instruments.
	Employment, education, and income	The questions in this section ask about employment status, reasons for unemployment, amount of total monthly household income, sources of income, reliance on charity or friends and family as income sources, and level of educational achievement. It is important to understand how the pre-post AHPP differences in employment and income. Education level will be used for analysis to determine if AHPP has affects on different subgroups.	Slight wording differences
	Child well-being	Here we capture information on the willingness of the children to go to school, the parental involvement in the school, and respondent assessment of whether they are more involved in school since receiving an AHPP unit.	Slight wording differences.
	Neighborhood and Social Interactions	This section includes the series of question on satisfaction with the neighborhood, perceptions of safety, and criminal victimization that were used on HUD’s baseline survey. This will enable us to measure the affect of AHPP on safety. We also ask respondents to report on how safe they feel from floods, high winds and hurricanes. This is important as AHPP units are required to be built outside the 100-year flood plain and thus may be in safer locations or built to higher wind-resistant standards than pre-AHPP housing.	No differences between instruments.
C: Quality of Life (cont.)	Social Support Module	The social support module focuses on the respondent’s access to social supports in different areas first at the time of the hurricane and then at the time of the interview. This allows us to see if there are any changes in the types of support available to participants after the disaster.	No differences between instruments.
	Secondary contacts	This section assists us in future locating efforts.	No differences between instruments.
	Debriefing Module	This section allows us to check that the respondent is not upset by our survey questions and connect them with someone who can help if they are.	No differences between instruments

A3 Use of Improved Technologies

The surveys will be administered using computer-assisted personal interviewing (CAPI) technology. Data collected during interviews are entered into data files that are stored within the computer system. The CAPI questionnaire format is user friendly and easily guides interviewers through the survey. Abt has successfully used this technology on past projects, including Moving to Opportunity Interim Evaluation (MTO), the Effects of Housing Choice Vouchers on Welfare Families Evaluation (Voucher Family Study), and the HOPE VI Panel Study.

A4 Efforts to Avoid Duplication

The Household Outcomes Survey will build on the baseline survey developed by HUD to capture information on AHPP participants prior to receiving AHPP assistance. Other than HUD’s survey, we are not aware of any other data collection being undertaken for AHPP participants or participants in any other similar program.

A5 Involvement of Small Entities

The Household Outcomes Survey will not involve data collection from small entities. Respondents will be heads of households.

A6 Consequences of Less Frequent Data Collection

The AHPP is a time-limited demonstration program. While some AHPP units may be converted to permanent housing after the demonstration period, some units will be demobilized in as little as 12 months. It is thus important to capture information on outcomes during the relatively brief demonstration period, and while participants’ perceptions and experiences are recent and still fresh in their minds.

The timing of the first household outcomes survey data collection period will take into account the differing AHPP implementation timelines in each state. Because the AHPP programs are getting underway at different times, the first survey will be conducted in two waves. In Mississippi where implementation has been underway since spring 2007, the first wave of the survey will occur in July 2008 or as soon thereafter as possible following OMB approval. The second wave will include the three remaining states where program implementation is more recently underway (Alabama, Louisiana, and Texas) and will occur in July 2009. The second round of household outcomes data collection for all four grantees will occur in July 2010.

The data collection schedule allows us to collect information on outcomes such as health, mental health, economic and employment circumstances, and housing situations when participants have been in their AHPP housing for at least six months and while experiences and perceptions are fresh in participants’ minds. The findings would be much less reliable if respondents were interviewed either too soon (when they had spent only a short period in their AHPP unit) or too late (when they would have to recall their experiences and perceptions over a long period of time).

A7 Special Circumstances

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6 (Controlling Paperwork Burden on the Public—General Information Collection Guidelines). There are no special circumstances that require deviation from these guidelines.

A8 Consultations Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, the Department of Housing and Urban Development published a notice in the Federal Register in March 2008 (V73, No.56, pp. 15168-69) announcing the agency’s intention to request an OMB review of data collection activities for the Household Outcomes Surveys for the Assessment of Quality of Life Issues in FEMA’s Alternative Housing Pilot Program (AHPP). The notice provided a 60-day period for public comments. A copy of the Notice is in Appendix F.

HUD’s contractor, Abt Associates, developed the household outcomes data collection instrument in consultation with staff from HUD, FEMA and Abt’s subcontractors (Amy Jones and Associates Inc. and The Urban Institute). Pre-testing of the survey instrument was conducted in April 2008.

A9 Payments to Respondents

Each respondent will receive a $25 incentive payment for completing the first household outcomes survey and a $25 incentive for completing the second household outcomes survey.

A10 Arrangements and Assurances Regarding Confidentiality

Our contractor, Abt Associates, takes seriously the responsibility to protect the subjects they interview. The data collection plan, informed consent form, and survey instruments were reviewed and approved by Abt’s Institutional Review Board (IRB) to help ensure appropriate protection.

To protect the privacy of respondents, the findings from the study will be reported only at the aggregate level and responding study participants will not be identified in the study reports.

A11 Sensitive Questions

The Household Outcomes Survey does contain some questions that may be sensitive to some respondents, particularly given the traumatic experiences that respondents may have had during and following the hurricanes. Potentially sensitive domains include questions related to mental and physical health issues.

We have taken steps to address the issue of sensitive subjects in both the survey design and administration. First, we have used questions that have been tested and found effective in other large-scale studies. In addition, we will be using local interviewers to conduct the interviews. These local interviewers will have lived through many of the same events as survey respondents and can be empathetic to respondents’ concerns. We will also offer respondents the option of responding to the survey either by telephone or in person. For particularly sensitive items, we will offer respondents alternative ways of indicating their responses, other than saying their answer out loud.

These questions are important to the study. Eliminating them would adversely affect our ability to truly measure quality of life changes for those who received an AHPP unit and hinder the study results. The hypothesis that receiving a better quality and more permanent housing unit helps to eliminate stress is expected to correlate with decreases in mental health issues. It is also likely to show improvements in physical health. In selecting the questions to ask, we have been careful to select questions that have worked well and did not result in high rates of missing data on past studies.

A12 Estimate of Annualized Burden Hours

Exhibit 2 provides information on the estimated time necessary to complete the data collection for each household outcome survey. Total burden for data collection over three years for this survey is estimated at 1,680 hours. The average annual burden is 560 hours.

Exhibit 2

Response Burden Summary

A	B	C	D	E
Survey Wave	Number of Respondents	Burden per Respondent (Minutes)	Total Respondent Burden (Minutes)	Total Respondent Burden (Hours)
			(C*E)	D/60
First Household Outcomes Survey in Mississippi in 2008	320	45	14,400	240
First Household Outcomes Survey in Other Three States in 2009	800	45	36,000	600
Second Household Outcomes Survey in all Four States in 2010	1120	45	50,400	840
Total	2240	45	100,800	1680
Average Annual Burden	747	45	33600	560

A13 Estimated Record Keeping and Reporting Cost Burden on Respondents

There is no cost to respondents other than the time required to respond to the survey. They will receive a stipend for completing the survey ($25 for first follow-up survey and $25 for second follow-up survey). These costs are reflected in A14 (Estimated Cost to the Federal Government).

A14 Estimated Cost to the Federal Government

The total contracted cost to the federal government for two waves of the first survey and one wave of the second survey is $1,057,354.

A15 Reasons for Changes in Burden

This submission to OMB is a new request for approval; there is no change in burden.

A16 Tabulation Plan, Statistical Analysis, and Study Schedule

Responses to AHPP survey questions will be stored in a database to analyze the impacts of the AHPP and derive recommendations about lessons from the demonstration. Once the survey data are compiled, cleaned and prepared for analysis, analysts will link the survey data to baseline data collected by the states or HUD and provided to Abt, and survey statisticians will prepare non-response adjustment weights so that the completed observations will represent all AHPP study members at all grantee sites. We will then prepare analysis data containing outcome variables, explanatory covariates, any necessary weighting variables, and subgroup identifiers.

Outcomes. The goal of the data analysis is to measure the impact of the AHPP units on the quality of life for people who move into them. We will assess outcomes in several major quality of life domains, including: housing conditions, neighborhood quality, behavioral health, and physical health.

In Mississippi, Alabama, and Texas, we will use a pre-post comparison of outcomes and questions on changes to the respondent attributes that are related to AHPP to measure the impact of the AHPP program. A pre-post comparison is an analysis that compares respondents’ quality of life prior to living in the AHPP unit (pre-AHPP) as reported on the baseline survey to their quality of life after living in AHPP unit (post-AHPP) on the first and second household outcomes surveys. To supplement the pre-post comparison, we are also directly asking respondents whether or not they would attribute changes in their quality of life to AHPP. A limitation of these attribution questions is that respondents may not know the reason for the change or it may due to multiple reasons that cannot be distinguished in a short interview.

In Louisiana, where we expect to implement random assignment, we will look at whether households who are offered an AHPP unit fare better in the quality of life domains than similar households who are not offered an AHPP unit. To improve the precision of these estimates, we will use regression analysis to control for any chance differences between the treatment and control groups at baseline and to increase the explained variance observed in the outcomes. We will use ordinary least squares (OLS) for continuous outcomes and logit models for binary outcomes. We anticipate that most outcomes will be prevalence (binary) outcomes, such as whether health is good or excellent, whether children are attending school regularly, and whether the family has more than one person per room.

In an experimental design, the difference between the treatments and controls measures the “intent-to-treat” (ITT) impact. This answers the question of “What is the average benefit (on a particular outcome) of offering this type of housing to displaced hurricane survivors?” We anticipate that policymakers will be even more concerned with the question, “What is the average benefit (on a particular outcome) to displaced hurricane survivors who live in this type of housing?” Therefore, we will also calculate the “treatment-on-the-treated” (TOT) impact estimate for outcomes. The TOT estimate is derived from the ITT estimate through the Bloom adjustment mathematical formula.¹

Key Subgroups for Analysis. The impact of the AHPP program may be different for different groups of people. To investigate this, we will conduct some analysis separately based on the characteristics of the AHPP participant and their household. We expect the key characteristics for subgroup analysis to be: (i) whether the participant owned or rented their home prior to the hurricanes; (ii) whether the household has children; (iii) whether the household has an elderly or disabled person; (iv) and living situation (such as FEMA travel trailer) prior to AHPP. Some analysis will be conducted separately by state and within the state, and we may also want to conduct separate analysis by the type of site (either geographic location or group versus infill site) if sample sizes permit.

Schedule. The first round of data collection is scheduled to begin in Mississippi in July 2008 or as soon thereafter as possible after receiving OMB approval. The second wave of the first round of data collection will occur in Alabama, Texas, and Louisiana in July 2009. Analysis of the Mississippi survey data will begin in October 2008 and a draft interim report on the findings from the Mississippi data collection will be submitted in January 2009. A final report on Mississippi will be submitted in March 2009. Analysis of survey data from Alabama, Texas and Louisiana will begin in September 2009 with a draft report submitted to HUD in January 2010 and a final report submitted in February 2010. The second round of data collection for all four grantees will occur in July 2010 and analysis will begin in October 2010. A draft and final report of the findings from all sites will be submitted in early 2011.

A17 Expiration Date Display Exemption

All data collection instruments will prominently display the expiration date for OMB approval.

A18 Exceptions to Certification

This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

Part B
Statistical Methods

HUD has contracted with Abt Associates to conduct the household outcomes survey of AHPP participants and, in Louisiana, a comparison group of households who were found eligible for AHPP but did not participate in the program.

B1 Potential Respondent Universe

The AHPP demonstration implementation is still underway, so the final number of units to be produced – and thus households to be served – is unknown. We expect the total number of units by state will be approximately as follows:

• Alabama – 100 units

• Louisiana – 500 units

• Mississippi – 3,500 units

• Texas – 100 units

This would result in a total potential respondent universe for people receiving AHPP units of 4,200 households. In Louisiana, where more applicants are expected to apply than there are available units, we will also survey people who applied for the program but were not randomly selected to receive a unit (control group members). The size of this group is expected to be at least 1,000 people, but this will not be known until the application process is complete sometime in 2008.

B2 Statistical Methods

B2.1 Sampling Plan

Household Outcomes Survey Sample Allocation. We plan on selecting a sample of 1,400 households for the survey sample. In allocating the sample, we tried to balance the goals of measuring impacts for each state and taking advantage of the more definitive estimates of impact with the treatment-control group comparison in Louisiana. We also took into account that Mississippi has the largest program and was awarded more than 70 percent of the AHPP funding and will be the first site on which outcomes are available. Based on these considerations, Exhibit 3 shows the planned allocation of the sample for the first household outcomes survey. A discussion of the sample and precision of the estimates for each state follows in Section B2.2.

Exhibit 3. Planned Samples Sizes for the Household Outcomes Survey

State	Sample Size for First Household Outcomes Survey	Number of Completed Surveys (assuming 80% response rate)
Mississippi	400	320
Louisiana	800	640
Alabama	100	80
Texas	100	80
Total	1400	1120

Sample Selection. Our plan is to select all AHPP recipients in Alabama and Texas and samples in Mississippi and Louisiana. For each applicant selected, we will target the adult in the household who completed the application or the baseline survey.

In Mississippi, our plan is to select a simple random sample of AHPP recipients who moved into their AHPP unit at least six months prior to the start of the first survey period. A simple random sample means that every person eligible to be selected for the survey has the same chance of being selected for the survey. Even if the person has moved out by the time of the survey, they will remain in the sample and we will survey them about their experiences. We do not plan to survey people in Mississippi who applied for the program, but were determined to be ineligible, turned down an offered unit, or accepted a unit but never moved into the unit. For the first follow-up survey, this means that the sample will represent applicants in Mississippi who lived in an AHPP unit for at least one day at least 6 months prior to the start of the survey. (If the survey starts in July 2008, this means everyone who moved into their unit by the end of 2007.)

The first follow-up survey in Mississippi is not being scheduled for a later time period (such as the same time as the first follow-up survey in the other sites) for several reasons. First, a few of the temporary AHPP units have already been demobilized as some of the recipients have moved into permanent housing and all the temporary units are suppose to be demobilized by the end of March 2009. We want to interview as many people as possible while living in their AHPP unit (as long as they have lived there at least six months). Second, Mississippi’s first units were occupied in June 2007 and they are the only site with occupied units as of April 2008. We want to capture the affect of AHPP on their quality of life at approximately the same time in the housing cycle as the other sites. Third, both FEMA and Congressional staffers need to know how well the program is working as soon as possible so it can factor into decisions about disaster planning.

In planning for the second household outcomes survey in 2010, we plan to include households that received their AHPP unit in the survey sample because it is important to capture any differences in either the program or households receiving units in January 2008 or later. The Mississippi program is evolving as it matures. It is expanding into more counties, is negotiating with non-profits and housing agencies to develop group sites, and is also moving from a program that provides temporary housing for up to two years to a program that provides permanent housing. (Some of the earlier households that moved into what was called temporary housing will have their housing reclassified as permanent housing if the household needs it and the local city and county governments approve the change.) Thus, for the second follow-up survey in Mississippi, we plan on splitting the sample based on whether or not the AHPP unit was occupied before or after January 2008.² The split will be approximately 50-50 as that is the expect distribution of units occupied before and after January 2008. The pre-January 2008 sample will be a subsample of the households selected for the first follow-up survey.

In Louisiana, we plan to select controls and treatments in exactly the same way to ensure that the only difference between the groups is their receipt of an AHPP unit. The Louisiana AHPP staff have discussed up to six distinct sites where recipients may move in Louisiana: (1) Jackson Barracks, (2) a senior village in Baton Rouge, (3) a mixed-age group site in Baton Rouge, (4) a group site in New Orleans, (5) infill housing in New Orleans, and (6) a group site in Lake Charles. It is not likely that all these sites will work out, so there will be likely be fewer sites in the end. Our plan for the July 2009 survey is to select the Louisiana sample in proportion to the number of occupied units at each of the sites. However, in consultation with HUD and FEMA, we may exclude some sites from the sample selection so that we can have larger samples at the other sites. The decision on excluding Louisiana sites would be based on the need to have more precise estimates from the sites most important for setting future disaster response policies.

B2.2 Justification of Level of Accuracy

The justification of the sampling approach and level of accuracy the sample allocation presented in Exhibit 3 (above) is as follows:

Mississippi. A relatively large part of the sample was allocated to Mississippi because we wanted to obtain reasonable levels of precision for the largest program. With 320 completes, the 95 percent confidence interval for a yes/no question in which 50 percent of the people answer “yes” is 44.5 to 55.5 percent (i.e., +/- 5.5 percentage points).

Louisiana. In Louisiana, the sample will be evenly split between treatments and controls. This means that we plan to select 400 treatments and 400 controls and expect approximately 320 respondents from each group. The main analysis for Louisiana will be based on treatment-control comparisons. With 320 completes in each group, we will be able to detect true differences between the treatment and control group of 9.7 percentage points or larger.³

Alabama and Texas. In Alabama and Texas, we are proposing to target all the AHPP recipients— expected to be approximately 100 people per state—for the survey. By including every AHPP recipient in the sample, the estimates are extremely precise even though the size of the sample is relatively small. If we achieve a 100-percent response rate, there is no sampling error and thus estimate is the true value for the population. Some people will likely decline to participate in the survey or be unavailable during the survey field period, so the response rate will be less than 100 percent, but the sampling error will be as small as is possible with this size AHPP project.

B3 Maximizing Response Rates

The Household Outcomes Survey will be administered by phone with in-person follow-up for non-responses or respondents that prefer a face-to-face interview. We will maximize participation by sending an advance letter notifying potential respondents about the study and encouraging their participation (see Appendix E). We will offer respondents the flexibility to respond either by phone or with an in-person interview. We will also use local interviewers who may have a better rapport with respondents because they have been through similar experiences during and since the hurricanes. We will offer respondents a $25 incentive payment for the first household outcomes survey and a $25 incentive for the second household outcomes survey.

We will also undertake a number of steps to minimize study subject attrition in order to maximize response rates. A tracking database is being developed to allow for the systematic collection and management of participant location data. The AHPP participant tracking system will contain information on the household and its members such as key dates (e.g., date baseline survey was received, date of random assignment), family membership, telephone and address information, and secondary contacts.

The participant tracking database will be updated on a quarterly basis. Updates will come from a mix of active and passive tracking sources to maintain the location of AHPP participants over time. Active tracking requires direct contact with a program participant and allows us to confirm that our location data is accurate. Passive tracking relies on electronic comparisons of data from the tracking database and data from outside vendors. Passive tracking methods can provide three data elements critical to sample tracking: address updates, phone number updates, and deceased status.

Exhibit 4 presents a summary of the tracking sources we will use over the life of the project and their respective timing.

Exhibit 4. Summary of Proposed Tracking Activities for AHPP Participants

Source	Who will be Covered?	Timing
Passive Tracking Activities by Source
National Change Of Address Database	All heads of household	Quarterly
Telephone Matches	All heads of household	Quarterly
Deceased Database Searches	All heads of household	Annually
Other Passive Sources	All heads of household	Semi-annually following the first NCOA update
Active Tracking Activities by Source
Participant Tracking Letter	All heads of household	8 months after first participants enrolled; again at 22 months
Field Locating	All household heads sampled for the follow-up survey data collection efforts	July 2008 in MS; July 2009 in AL, TX, and LA; July 2010 in all sites
Data System Updates
	All passive and active tracking data	Quarterly, after passive results are processed.

Beginning in May 2008, quarterly National Change of Address (NCOA) updates will be requested. NCOA updates, along with telephone updates will be obtained from a vendor. Annually, we will expand the NCOA search to include a search against the Social Security Death Index. The death index searches will be done just prior to any active tracking or the survey data collection. Identifying sample members who passed away before survey data collection begins allows us to flag them in the databases so we do not attempt to interview them. To do so could place undue burden on the deceased member’s family.

We will also employ active tracking efforts, such as customized participant tracking mailings. All sample members will receive a tracking letter asking them to update or confirm their contact information. All updates obtained through the mailing are sent out to the NCOA vendor for address standardization. This process ensures that all new address data is posted to the data system in proper US Post Office format.

We will implement field locating efforts during the data collection field period. As interviewers work the sample to complete an interview, they utilize several locating tools such as directory assistance calls, online searches for updated information, and door-to-door locating. Updated information is stored in the respondent information booklets to aid interviewers in subsequent data collection efforts. The most recent update, or one that yields the correct address, is updated into the field management system.

B4 Tests of Procedures or Methods

Early drafts of the instrument have been reviewed by HUD personnel, Abt Associates staff, and their subcontractors in order to ensure that the instrument is clear, flows well, and is as concise as possible. In addition, the instruments submitted in this package were pre-tested in April 2008.

Abt interviewers pre-tested the non-experimental version of the survey with 5 people who had lived in an AHPP unit in Mississippi for at least 6 months. The interviews were done by telephone on paper. The paper survey added a bit of time to the pretest instruments because the interviewer had to manually follow skip patterns. In addition, questions that rely on existing data had to use drop sheets with key information listed on them in the pretest. However, once programmed in CAPI, such key information will be included in the program and displayed automatically, reducing the burden on the interviewer.

The pretest findings indicated that the questionnaire was well designed. Interviewers indicated that they had little to no problems in administering the interviews. Respondents did not indicate that the questions were difficult to answer or that the recall was troublesome.

The interviews varied in length from 38 to 70 minutes. As noted above, the length was likely affected a bit by the need to conduct the pretest on paper. Therefore, it is likely that our 45 minute estimate is within the right range. However, there are several factors that could cause this to change. First, and most important, is the respondent population itself. The study population is one that has undergone extreme trauma. In such situations it is very likely that respondents will digress, and want to share their story with the interviewers. We will train our interviewers to tactfully steer the respondents back to the questions at hand, but some respondents will increase the length of the interview by sharing their stories. Other factors for consideration are household size and the presence of children as there are questions in the instrument about other household members. If a respondent lives alone, such questions would obviously be skipped.

B5 Statistical Consultation and Information Collection Agents

HUD has contracted with Abt Associates, Inc. to conduct the data collection. The data collection procedures will be similar to those used in other surveys conducted by Abt Associates. HUD also contracted with Abt Associates to design the survey and establish the sample design. The HUD Government Technical Representative (GTR) reviewed all the proposed procedures and had them reviewed by other subject matter experts at HUD. If there are any questions about this submission, please call either the HUD GTR, Harold Holzman (202-402-5709) or the Abt Associates Principal Investigator, Larry Buron (301-634-1735).

1 TOT = ITT * (1/p), where p equals the percentage of the treatment group who move into the alternative housing that is offered by AHPP. Howard Bloom, “Accounting for No-Shows in Experimental Evaluation Designs,” Evaluation Review 8, April 1984, pp. 225-46.

2 This should not be an issue in the other three states. Although these programs do not have occupied units as of April 2008, they are smaller programs and our expected to be completed by the end of 2008.

3 This 9.7 percentage point minimum detectable effect (MDE) is based on a two-sided test at the 90 percent significance level with 80 percent power. This is the worst-case scenario for the MDE because it is based on 50 percent of the control group providing the same answer to a binary question such as a Yes/No question. If 90 percent of the control group responded “yes” to the binary question (or 10 percent said “yes” and 90 percent said “no,”) then the MDE decreases to 6.7 percentage points.

Abt Associates Inc. Table of Contents ii