2021NYCHVS_SupportingStatementB_final

OMB Information Collection Request

SUPPORTING STATEMENT B

U.S. Department of Commerce

U.S. Census Bureau

2021 New York City Housing and Vacancy Survey

OMB Control Number 0607-0757

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Description of the Universe

The 2021 NYCHVS sample will consist of about 12,000 housing units selected from the Census Bureau’s Master Address File. The sample will be matched to administrative data files to ensure adequate coverage of various types of housing programs.

2. Procedures for Collecting Information

We expect that the vast majority of regular NYCHVS interviews will be conducted through personal visit.  However, some respondents prefer to answer the questions over the phone and there will be a telephone number on the Advance Letter that respondents can call. To accommodate these respondents, the New York Regional office will do the interview by calling from the office and the unit will not be visited. In addition, we will conduct a phone match to retrieve phone numbers that match the sample addresses and may attempt to collect the data by telephone interview, if unable to complete the personal visit.

The primary sample frame will be the Census Master Address File (MAF), updated as of July 2020, whereas in prior cycles the primary sample frame was the latest decennial census. Each sample unit will have about a 1 in 275 chance of being selected for interview. There is no longer any cluster sampling as in past surveys. Instead, we will use a single-stage systematic sampling method stratified by types of housing units, including those receiving publicly funded assistance. The sample is designed to produce a standard error of the estimate of the "vacant available for rent" rate for the entire city of no more than one-quarter of 1 percent, if the actual vacancy rate is 3 percent or less.

A secondary frame is being considered to address potential MAF undercoverage. This frame would be developed using administrative files from the survey sponsor.

Our target for housing type estimates is a 15% coefficient of variation. The housing types include:

• Public Housing

• Rent Control

• Rent Stabilized

• Affordable Housing / Rental

• Affordable Housing / Owner

• Conventional owner-occupied 1-4 family homes

• Condominiums

• COOPs

• Mitchel-Lama Renter

• Mitchel-Lama COOP

The procedure employed to estimate housing unit and population characteristics will consist of three parts:  (a) inflating the sample results by the reciprocal of the probability of selection, (b) adjusting the sample results to account for non-interviews, and (c) using independent estimates of housing units and population in order to control the estimates in a manner that increases precision. We plan to expand the ratio adjustments applied with the production of the sample weights. Ratio adjustments are beneficial because they reduce bias and variability of the resultant estimates. Additional considerations include using ratio adjustments employing known totals by housing type and tenure. The table below highlights the changes in methodology from 2017 to 2021.

Weight	Component	2017 NYCHVS	2021 NYCHVS
Housing Unit	Nonresponse Adjustment	Factor calculated within static cells	Use logistic regression to form dynamic cells
		Only used variables from source files	Use ACS tract-level estimates to form noninterview adjustment cells
	Ratio Adjustment to Control Totals	Used frame, In Rem, and HU POP totals	Use additional control totals by type of housing programs

No changes are planned in the person weighting for the 2021 survey. As in 2017, when calculating person weights, the final housing unit weight will be used as the base weight for each person, then we will add a ratio adjustment to account for sampling variability and known coverage deficiencies for persons within interviewed households. We will compute this factor within each borough by age, race, Hispanic Origin, and sex (200 cells). This is described in more detail in the Attachment F, 2017 NYCHVS Sample Design, Weighting, and Error Estimation.

We are also planning an experimental building-level weight, which could be used to estimate the total number of buildings, as well as various building-level characteristics.

3. Methods to Maximize Response

Based on the 81 percent response rate of the 2017 NYCHVS, the Census Bureau expects the 2021 NYCHVS response rate to be 80 percent. If an occupant is reluctant to participate, the field representative (FR) informs the regional office staff, who sends a follow-up letter explaining the survey in detail and urging the occupant's cooperation. A Census Bureau FR or his/her supervisor will contact the occupant again.

4. Testing of Procedures

Pretest Methodology

In 2019, the NYCHPD conducted successive rounds of pretesting for the revised NYCHVS questionnaire that would be used in 2021. Two types of testing were conducted: intercept surveys and field tests. These pretest activities were designed to complement review done by subject matter experts at NYCHPD, including the survey translation team, focus groups conducted with hard to reach populations, and cognitive interviewing done by the Center for Behavioral Science Methods (CBSM) at the Census.

Information from the pretests was used to refine question wording, answer choices, and scripting as well as the sequencing of survey items within a given module. Often, the same module was tested multiple times as questions were rephrased or reordered. Tests often utilized a split-panel in which two (or more) variations were tested at the same time to determine which worked better. Pretesting was also used to determine if any item did not perform sufficiently to be included in the final questionnaire. New questions that were tested but ultimately excluded from the final instrument are not discussed in this working paper.

Each test included a subset of survey items that were administered in a face-to-face setting along with scripted probes that asked participants to provide qualitative information on their answers or secondary information that was used to assess the quality of answers provided. Some probes were asked concurrently, that is, after respondents answer specific questions; others were asked retrospectively, that is, at the end of the questionnaire section. Interviewers were able to ask probing questions at their discretion, based on respondent responses and behaviors, to develop a more comprehensive understanding of potential problems with questions or answers. Probes borrowed from cognitive interviewing techniques and were developed to assess potential problems with respondent comprehension, information retrieval/recall, judgment/estimation, and response selection/decision making.

Information captured included participant answers to the tested questions, qualitative notes from the interviewer during the test, and detailed summary of findings and recommendations gathered after the conclusion of testing from the interviewers, including debriefs conducted with the interview team and senior staff from the NYCHVS. Basic demographic data were collected to gather information on the breadth of participants reached. Tests were conducted in-person by experienced field staff who have conducted fieldwork in New York City and overseen by Sponsor staff who have experience in survey methodology and field operations.

All tests were conducted anonymously, such that no personally identifiable information was collected from any participant.

Intercept Surveys

A total of 325 intercept surveys were conducted across six sets of intercept surveys, including one or more days of testing. For each intercept survey, two or more trained interviewers were sent to a specified public location with printed forms and clipboards where they approached individuals in a public space and recruited them to participate. Though they worked in close physical proximity to one another, each interviewer recruited and interviewed respondents separately. Interviewers approached a non-random sampling of passers-by; potential participants were asked to give ten minutes of their time to help improve our survey. A recruitment pamphlet was provided that summarized the NYCHVS, the activity they were being asked to participate in, and how the information would be used. It also stated that the data would remain anonymous.

Intercept surveys were conducted in order to collect a meaningful number of responses in a relatively short time period with minimal resources. Replication of the same small number of survey items many times in quick succession gave interviewers the chance to gain expertise in administering the questions, develop an intuitive sense for what did or did not work, and generate recommendations quickly. The larger number of completed surveys over a short time period also facilitated a split panel approach whereby two or more versions of questions could be tested simultaneously. Intercept surveys were conducted in English and Spanish.

Field Tests

A total of 150 short-form interviews were completed at subjects’ homes using a Computer-Assisted Personal Interview; this approach was intended to mimic how the NYCHVS is fielded by the Census Bureau but with only a limited battery of questions.

Individual modules were programmed, tested, and loaded onto a tablet to be used by HPD staff. Additional questions from other modules, such as the household roster, were incorporated to the extent they were needed for survey logic. Field test locations were chosen to survey housing types that are representative of the buildings that will be included in the 2021 NYCHVS, including Mitchell-Lama, Public Housing (New York City Housing Authority/NYCHA), Rent Stabilized buildings, and Cooperatives.

Field interviewers worked in pairs, knocking on doors of units in selected buildings. If a person below age 18 came to the door, HPD staff asked to speak with an adult. Potential respondents were asked to give ten minutes of their time to help improve the survey. If they agreed, interviews were conducted on the spot at respondents’ homes. A recruitment pamphlet was provided that summarized the NYCHVS, the activity they were being asked to participate in, and how the information would be used. It also stated that the data would remain anonymous.

Field tests were used to assess the flow of an entire module and verify appropriate survey logic. Feedback from these tests included how well the module could be administered in CAPI format, whether more or less scripting should be included, and whether additional “as needed” information should be provided to field representatives, such as optional definitions for concepts or guidance for administering individual items.

Additional Testing and Review

Pretesting complemented other forms of testing and review, including expert review by subject matter experts at NYCHPD and elsewhere; full-length test interviews to assess content, survey logic, and timing; focus groups with hard to reach populations; and two rounds of cognitive interviewing conducted in New York City by CBSM.

Additional information about testing procedures can be found in Attachment E, NYCHVS 2020+ Redesign: Questionnaire Development and Testing.

5. Contacts for Statistical Aspects and Data Collection

The following individuals were consulted regarding the statistical data collection and analysis operation:

Tamara Cole

Associate Directorate for Demographic Programs

U.S. Census Bureau

(301) 763-4665

Statistical Design, Analysis, and Collection

James Farber

Demographic Statistical Methods Division

U.S. Census Bureau

(301) 763-1844

Statistical Design

LIST OF ATTACHMENTS

A. 2021 New York City Housing and Vacancy Survey Instrument Content

B. 2021 New York City Housing and Vacancy Survey Centralized Reinterview Questions

C. 2021 New York City Housing and Vacancy Survey Advance Letter, Form H-100(L)

D. Legal Citations of NYC

E. NYCHVS 2020+ Redesign: Questionnaire Development and Testing

F. 2017 NYCHVS Sample Design, Weighting, and Error Estimation

18