Memo to OMB on VDHS Pilot Study

MEMORANDUM

MEMORANDUM TO: Robert G. Sivinski

Office of Statistical and Science Policy

Office of Management and Budget

THROUGH: Melody Braswell

Clearance Officer

Justice Management Division

Jeffrey H. Anderson

Director

Bureau of Justice Statistics

Jeri Mulrow

Principal Deputy Director

Bureau of Justice Statistics

FROM: Barbara A. Oudekerk

Statistician

Bureau of Justice Statistics

Lynn Langton

Chief, Victimization Unit

Bureau of Justice Statistics

DATE: June 27, 2018

SUBJECT: BJS Request for OMB Generic Clearance to conduct the Victims with Disabilities in Homeless Shelters Pilot Survey (under OMB Number 1121-0325).

The Bureau of Justice Statistics (BJS) is requesting generic clearance to conduct the Victims with Disabilities in Homeless Shelters Pilot Survey, an establishment survey of emergency homeless shelters, designed to capture basic information about the tracking and record keeping related to clients with disabilities and with prior experiences with criminal victimization. The proposed pilot study includes three overarching research questions: (1) What information do shelters retain about disability status of sheltered individuals? (2) What information do shelters retain about victimization? and (3) Are shelters able to provide information on the victimization experiences of and service offered to homeless persons who experience victimization, particularly those with a disability?

Funded by the Office for Victims of Crime (OVC) and BJS, this pilot survey is an important component of a broader effort by BJS to understand the prevalence of victimization among persons with a disability and the criminal justice systems’ and its complementary agencies’ response to persons with disabilities who are crime victims. Under Title 34 USC 10132, BJS is authorized to collect and analyze indicators of crime prevalence and assess the correlates of crime (see attachment 1). In addition, the Crime Victims with Disabilities Awareness Act mandates that the Department of Justice study the nature and extent of crimes against persons with a disability, and the manner in which the justice system responds to crimes against persons with a disability (Pub. L. 105-301, see attachment 1). The proposed work will build upon BJS’ current efforts to report on victimization among persons with a disability by testing the feasibility of collecting data on victimization, disability status, and access to services for homeless persons residing in emergency shelters, a population largely missing in BJS’s annual National Crime Victimization Survey. This work aligns with BJS’s larger efforts to build a more complete picture of crime, victimization rates, and access to victim services by collecting information from agencies providing services to crime victims. This approach is especially important for capturing data on hard-to-reach populations or on victims who experience crime types that are difficult to measure through the National Crime Victimization Survey.

The current request for approval, under the BJS NCVS Generic Clearance (1121-0325), is for a pilot survey of 100 completed telephone interviews with emergency homeless shelters. The proposed sample will be diverse to ensure the survey can be administered with shelters across different geographic locations and shelter sizes based on bed numbers, and will include veteran and non-veteran shelters. The sample will be drawn from the Homeless Management Information System (HMIS) database, the most comprehensive database on shelters to date managed by Housing and Urban Development (HUD). The pilot will test the administration of the project instrument and the quality of the data on victimization and disability status that can be provided by homeless shelters, and examine the feasibility of using the HMIS to survey emergency shelters.

Purpose of the Research

To date, the National Crime Victimization Survey is the only national data collection that assesses victimization rates and the nature of victimization among persons with a disability. As mandated by the Crime Victims with Disabilities Awareness Act (see attachment 1), BJS routinely releases statistical tables on the victimizations experienced by persons with disabilities.¹ However, there are limitations of using the NCVS to fully understand victimization among persons with disabilities. First, the NCVS requires reporting either by the victim or through a proxy person, presenting challenges for collecting comprehensive information about victimization from persons with certain types of disabilities (e.g., cognitive impairments). Another limitation is that the persons with disabilities experience homelessness at a higher rate than persons without disabilities, and the NCVS excludes the homeless. Housing instability is a correlate of victimization, and victimization can be a leading cause of homelessness.² Therefore, the NCVS might be missing a number of victimizations experienced by homeless persons with a disability each year. Although the NCVS includes a sample of group quarters which might include some homeless shelters each year, the NCVS is not designed to represent homeless shelters.

Therefore, BJS is investigating methodological approaches to capturing information about victimization and responses to crime among persons with a disability that will supplement data gleaned from the NCVS. Because of the challenges in directly sampling and surveying persons who are homeless, the proposed pilot project will test the feasibility of collecting information from emergency shelters on the levels and nature of victimization among their clients, in general, and specifically among those with a disability. This approach is consistent with the BJS victimization unit’s practice of collecting multi-level data from both victims and the agencies serving victims to analyze patterns and integrate findings for a more complete picture of crime and victimization. The cornerstone of this work, the National Census of Victim Service Providers (NCVSP), was administered for the first time in 2017 and demonstrated success in being able to recruit and survey agencies that serve victims as their primary function or through dedicated programs. The NCVSP included homeless shelters that intend to serve victims as part of their mission, for example domestic violence or sexual assault shelters. Although the NCVSP did not collect information specific to disability status of victims, for the purposes of this pilot work, the NCVSP provided evidence that shelters designed to serve victims are capable of completing a survey instrument on this topic. As such, the proposed pilot study will exclude domestic violence shelters, and instead focus specifically on testing the administration of the pilot survey with homeless shelters that do not identify as victim service providers.³

Examining victimization among persons with disabilities in homeless shelters is especially timely due to recent changes in the Victim of Crime Act (VOCA) Victim Assistance funding guidelines which now state that transitional housing expenses can be paid for with victim assistance funding.⁴ Data from shelters can help to fill important gaps in understanding the prevalence of victimization among at-risk, homeless persons with a disability in the United States; about the capacity of emergency homeless shelters to identify victimization and provide services; and about the need to expand or modify how services are delivered to this population. These data could also have direct utility for OVC by identifying the extent to which emergency shelters are identifying and documenting victimization and providing services to victims, and could therefore utilize funding or other related resources (e.g., training).

As a first step to testing whether shelters can be an adequate source of information on victimization among persons with a disability, BJS and Westat, through a cooperative agreement, designed the proposed pilot test to capture critical information about: a) what information homeless shelters record related to disability status and victimization, b) the types of services that are available to persons and victims with disabilities residing in shelters, and c) characteristics of victims with disabilities. The results of this pilot study will inform BJS about the practicality and potential challenges of conducting a national study of emergency shelters to better understand the prevalence of crime, delivery of victim services, and victim characteristics among homeless persons with disabilities.

Study Design and Sample

The Victims with Disabilities in Homeless Shelters Pilot Survey is primarily focused on the following issues:

• Examining whether emergency shelters are willing to complete a survey on disability status and victimization, topics that are not central to their core mission of providing housing services;

• Testing the feasibility of administering a two-part survey that asks for 1) general information about the shelter and clients and 2) information about the longest-served client (see attachment 2);

• Determining how many emergency shelter staff are needed to complete the two-part instrument;

• Testing the feasibility of using the HUD HMIS homeless shelter frame for administering a survey to emergency shelters (e.g., is shelter contact information available and up to date, does the frame contain accurate descriptive information about the shelter locations, size of shelter, type of shelter, etc.).

The sampling approach is aimed at drawing a diverse sample of shelters, including shelters for veterans, shelters in both urban and rural areas, and shelters that are diverse in size (200+ beds vs. smaller shelters). This sampling approach ensures the pilot will provide information on the feasibility of conducting this survey with different types of emergency shelters. This is important given that shelters’ experiences encountering victims with disabilities, their resources available for providing services, and their record keeping practices and access to electronic case management systems might vary as a function of shelter size or geographical location.

The sample will be drawn from a publicly available dataset of homeless shelters provided by the Department of Housing and Urban Development (HUD) called the Homeless Management Information System (HMIS). The original file contained 6,945 emergency shelters in the 50 states or DC, but was reduced after applying a number of restrictions:

• 5,483 records remained after excluding shelters classified as domestic violence shelters; domestic violence shelters are known to be serving crime victims, and information on the types of services they offer and characteristics of victims they serve will be captured through BJS’ other victim services statistical collections;

• 5,410 records remained after 145 records involving duplicated identifiers and addresses were simplified to the unique records with the highest number of beds.

As summarized in Table 1, the sample will aim for completed interviews with 100 total shelters, distributed across four strata defined by characteristics of interest: 20 veteran shelters, 20 non-veteran shelters in rural counties (that is, counties outside metropolitan areas), 10 large shelters with at least 200 beds, and 50 non-veteran shelters in urban or presumed urban counties. The four strata are defined to avoid overlap; in particular none of the 200+ bed shelters in the HMIS frame are in rural counties or serve predominately veterans. Shelters will be presumed to be urban if the geocode provided by HMIS associates them with a specific city rather than a county.

Table 1. Targeted sample numbers by shelter stratum

Shelter Characteristics	Total shelters	Target # in final Sample
Total shelters	5,410	100
Stratum
Veteran shelters	205	20
Rural county shelters (non-veteran)	818	20
Shelters with 200+ beds/	244	10
Urban/presumed urban shelters	4,143	50

Because there is limited basis to predict the response rate, an initial sample of 200 shelters will be drawn in proportion to targets of interest. Backup samples of 50 percent of the size of the initial sample will also be drawn for each stratum. The backup sample will be released if response lags within the stratum. Additional backup samples can be readied for use if necessary. This approach is appropriate given that the study is designed as a test of the instrument rather than to produce national-level instruments, and this approach is timely and cost-effective for generating a diverse sample of completed surveys.

Instrument Design and Development

To develop the instrument, the project team consisting of Westat and BJS staff first consulted with a number of experts and practitioners to best understand how to structure a survey on disability status and victimization for emergency shelters. The team conducted site visits to four homeless shelters in Washington, D.C., Maryland, and Virginia. They met with shelter administrators and staff to learn about shelter policies and practices, including screening and assessment and record-keeping capacities. In addition, the team conducted two phone interviews with shelter directors in other locations in the U.S. to supplement knowledge gained from the site visits. These activities informed the development of the survey instrument by identifying key areas of interest and led to the development of the initial draft of the 2-part survey (see attachment 3 for the initial version of the questionnaire).

Specifically, part 1 of the instrument asks general, high-level questions about the characteristics of the shelter, record keeping on victimization and disability status, number of documented victims served, and basic services offered by the shelter. Part 1 is intended to be completed by a shelter administrator or program director that has a broad view of what the shelter does. Key domains include:

• Shelter characteristics (e.g., date established, type of shelter, target population, and number of beds;

• Services (e.g., mental health, crisis counseling, civil legal aid);

• Record keeping and reporting of about victimization experienced by clients (e.g., rape/sexual assault, physical assault, domestic violence, human trafficking, survivors of homicide victims);

• Shelter resources for victims (number of staff dedicated to serving victims and funding for victim services);

• Record keeping and reporting about sheltered persons with disabilities (e.g., types of client disabilities, methods for finding out about disabilities, methods for storing information about disabilities).

Part 2 of the instrument focuses specifically on gathering information on the prevalence of victimization and characteristics of victims and their victimizations, when possible specifically for sheltered persons with a disability. This section is intended to be completed by a case manager or other shelter staff who can speak to the experiences of individual clients. The instrument asks for the total number of clients with a disability served in the past 30 days, then asks the case manager to provide information about the longest-served client with a disability or who experienced a victimization to assess:

• Basic client characteristics (but no personally identifying information);

• Type of victimization experience;

• Service needs of clients.

Two of Westat’s senior methodologists with expertise in survey development reviewed several draft versions of the survey instrument and provided their feedback. Once the instrument was completed, in June, 2017 the team conducted five cognitive testing sessions over the phone with shelter administrators and program managers to assess comprehension of the items, assess survey length and burden, and assess whether items were worded appropriately for a phone interview. Most participants’ cognitive interview feedback resulted in minor edits to question wording or tweaks to particular response items. However there were two major changes that helped refine the instrument to be used in the pilot test.

First, the survey item asking respondents to indicate whether any of their clients experienced a list of different types of victimization was burdensome due to its length and level of specificity (e.g., C1 in attachment 3). The study team revised these questions by shortening the list of victimization types; restructuring the question to first ask about which types of victimizations are generally documented in electronic case files or in some other way that allows for data reporting; and then asking which types of victimization had been documented for clients they served in the past 30 days. The revised, shortened victimization list also replaced the list of victimizations in all other sections of the survey to reduce burden.

Second, there were a few items in part 1 of the survey assessing whether shelters had any staff focused part-time or full-time on serving crime victims. The respondents said that many of their clients had been victims of crimes, and that all of their staff members were dedicated to serving victims when victimization as identified. This section of questions was revised and shortened to one item assessing whether any staff specialize in working with victims. Attachment 2 is the revised version of the questionnaire to be used in the pilot test.

Interview Procedures

The interviews will be conducted over the telephone during a 1-month time period beginning as soon as OMB approval is obtained, approximately July or August, 2017. The sample of emergency homeless shelters will receive a recruitment letter from Westat, which includes the rationale for the survey and a brief description of the survey, as well as the importance of participation (attachment 4). The letter states that Westat is collecting the data on behalf of the Bureau of Justice Statistics, notes that the collection is voluntary, and explains that Westat will be contacting the shelter to schedule the interview and complete the survey. The letter will also ask the facility administrators whether they will be the contact person or if another staff member will participate in the interview. A one-page FAQs sheet will also be mailed with the introductory letter (attachment 5).

Shortly after the facility administrators receive the recruitment letter, the Westat interviewers from Westat’s Telephone Center (TRC) will contact them to:

• Invite them to complete the survey;

• Identify appropriate individuals who will complete the survey;

• Determine the schedule for survey completion; and

• Answer any questions that the facility administrators or staff pose to the interviewers.

Administrators will be given the option of completing both Parts 1 and 2 or completing Part 1 and recommending a colleague, such as a case manager, who has regular interactions with the client, to complete Part 2 of the survey.

Language

The pilot survey will be conducted in English.

Burden Hours for Pilot Survey

We request a total of 58.3 hours for the pilot survey (35 minutes x 100 interviews).

Analysis Plan

The analysis plan consists of analysis of survey responses and compilation of paradata. The survey data will be converted from the CATI database to a SAS dataset. Descriptive analysis will be conducted on all survey questions to determine the following:

• Variables with missing data (means and ranges);

• Distribution of responses by question; and

• Differences in responses by shelter type.

Westat will also analyze the paradata from the pilot. Analyses will yield:

• Average administration time (and ranges);

• Rate of completion for the sample and by strata;

• Percent of respondents that checked records to respond to questions (and how this varied by strata);

• Descriptive statistics on completion codes (e.g., completions, partial completions, refusals) for the entire sample and by strata;

• Comparison of completion codes for the sample and by strata;

• Reasons for refusals;

• Percent of the survey completed by one staff member and staff type (e.g., administrators, program managers, case workers);

• Percent of survey instruments completed by two staff members (e.g., administrators, program managers, case workers);

• Accuracy of the contact information in the HMIS database.

Informed Consent, Data Confidentiality and Data Security

The pre-survey letters will include an introduction to the survey. The letter provides the purpose of the survey, the voluntary nature of the study, and a number to call with questions about the study. The letter also explains that no personally identifying client information will be collected. The introduction to the survey and the script read to respondents on the telephone will repeat much of this information. The letter and introduction to the survey will both announce the estimated length of the interview in advance, allowing the participant an opportunity to decline if the burden would be unacceptable.

The Bureau of Justice Statistics (BJS) is authorized to conduct this data collection under 34 U.S.C. § 10132. BJS will protect and maintain the confidentiality of personally identifiable information (PII) to the fullest extent under federal law. BJS, its employees, and its contractors (Westat staff) will only use the information obtained for statistical or research purposes pursuant to 34 U.S.C. § 10134, and will not disclose information in identifiable form to anyone outside of the BJS/Westat project team without a respondent’s consent. All PII collected under BJS’s authority is protected under the confidentiality provisions of 34 U.S.C. § 10231. Any person who violates these provisions may be punished by a fine up to $10,000, in addition to any other penalties imposed by law. Further, per the Cybersecurity Enhancement Act of 2015 (6 U.S.C. § 151), federal information systems are protected from malicious activities through cybersecurity screening of transmitted data. For more information on how BJS and its contractors will use and protect information, go to https://www.bjs.gov/content/pub/pdf/BJS_Data_Protection_Guidelines.pdf.

Access to Westat’s secure computer systems is password protected and data are protected by access privileges, which are assigned by the appropriate system administrator. All systems are backed up on a regular basis and are kept in a secure storage facility. To protect the identity of the respondents, no identifying information will be kept on the final data file. Identifying information includes the name of the sampled organization, address, and telephone number. The identifying information of respondents (e.g., name, telephone number, shelter name) will be deleted once the analysis file has been created and the link is no longer needed. We estimate destruction of data to be one month after the project has ended. Once Westat completes the summary report, all copies of the data will be destroyed.

With respect to personnel, all members of the research team are required to sign a pledge of confidentiality. This pledge requires employees to maintain confidentiality of project data and to follow the above procedures when handling confidential information.

Protection of human subjects

This is an establishment survey that will not collect personal information from respondents; however, the human subjects protection procedures been reviewed and approved by Westat’s Institutional Review Board (IRB), which has Federal-wide assurance.

1 For recent publications, see: https://www.bjs.gov/index.cfm?ty=tp&tid=924

2 Meinbresse, M., et al. (2014). Exploring the experiences of violence among Individuals who are homeless using a consumer-led approach. Violence and Victims, (29).

3 The HMIS dataset includes a flag variable indicating which shelters identify as domestic violence shelters, allowing for these to be excluded from the pilot study.

4 VOCA Victim Assistance Program Final Rule (81 FR 44528, August 8, 2016)

7