ACF SSB_NHTH_Nov2019_clean OMB comments_revised_clean

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

The National Human Trafficking Hotline Program: Assessment of Utility and User Satisfaction

OMB Information Collection Request

New Collection

Supporting Statement

Part B

NOVEMBER 2019

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Mary G. Mueggenborg

Part B

B1. Objectives

This proposed information collection (IC) is part of the ACF Office of Planning, Research, and Evaluation’s (OPRE’s) Evaluation of the National Human Trafficking Hotline (NHTH) Program. OPRE, in collaboration with the Office on Trafficking in Persons (OTIP), seeks approval to conduct two multi-mode surveys, gathering information from individuals who seek assistance from the NHTH. OPRE has contracted with RTI International to conduct this work.

Study Objectives

The NHTH connects victims¹ of sex and labor trafficking, and others who seek assistance on behalf of a victim of human trafficking, with resources and services to get help. The NHTH also receives tips about potential cases of human trafficking and facilitates the reporting of that information to local, state, and federal law enforcement authorities.² The Evaluation of the National Human Trafficking Hotline (NHTH) Program is supporting ACF in efforts to: 1. Describe the activities, procedures and organization of the NHTH program; 2. Describe the customer services provided by the NHTH; 3. Describe the short-term outcomes of NHTH contact for hotline users; and 4. Develop performance measures for objectively assessing NHTH activities and accomplishments. The data collection proposed under this request will help to accomplish objectives 2, 3, and 4.

Together, the two surveys are designed to provide data that will increase understanding of contactors’³ satisfaction with NHTH services and the utility of the information and referrals provided during the contact.

Generalizability of Results

This study is intended to describe contactors’ experiences with the NHTH Program, not to promote statistical generalization to other programs or services.

Appropriateness of Study Design and Methods for Planned Uses

Administering a self-report survey to individuals who seek assistance from the NHTH after their interactions with the hotline is essential for understanding NHTH contactors’ perceptions of and experiences with the service. Data will be used to produce estimates of contactors’ satisfaction with the services they received during their contact with the NHTH and the utility of the information and resources provided by NHTH advocates.

This is the first systematic effort to collect data from NHTH contactors about their experiences with and perceptions of the services provided by the NHTH. Because data have not been collected among this population, it was necessary to make assumptions regarding sample sizes, responses rates, and standard deviations.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The possible universe of respondents for the Immediate Survey (Instrument 1) is all adults 18 years of age and older who contact the NHTH. Approximately 30,000 annual contacts (about 2,500 per month) are made with the NHTH.

During the study period, all contacts 18 years of age and older are eligible for inclusion in the Immediate Survey (Instrument 1), regardless of mode of or reason for the contact. The survey will be administered during an estimated four-month period, with approximately 10,000 contactors eligible for participation in the Immediate Survey. Although surveys conducted by other hotline services have typically excluded contactors who are transferred directly to a service agency by the hotline advocate, the Immediate Survey is designed to enable participation from this group because of the importance of understanding the utility of referrals for the critical victim group of contactors.

The possible universe of respondents for the Two-Week Follow-Up Survey (Instrument 2) is all respondents who complete the Immediate Survey. All who complete the Immediate Survey will be invited at the end of the survey to participate in the Two-Week Follow-Up Survey. If they indicate interest in participating in this second survey, they will be asked to provide their telephone number or e-mail address to receive a survey invitation two weeks later. A detailed description of data collection is presented in section B4.

Sampling

All contactors to the NHTH during the four-month data collection period will be eligible to participate in the survey at the end of their interaction with the hotline. Because there is no reason to assume that the nature of calls varies throughout the year, contactors during the data collection period are expected to be representative of all contactors during the year. Assuming the monthly average of 2,500 contacts per month during the estimated four-month data collection period, a response rate of 10.5% is necessary to meet the target sample size of 1,031 respondents.⁴ Further discussion on estimated response rates is included in section B5.

Because there have not been prior efforts to collect data from this population, generalized assumptions were used to calculate the target sample size. The target sample of 1,031 will give a margin of error of +/- 3% for a conservative estimate of approximately 50% of respondents responding positively (a selection of a 4 or 5 on a 1-to-5 scale) to the various measures of satisfaction with the hotline contact.

The Two-Week Follow-Up Survey will be delivered to all contactors who complete the Immediate Survey and agree to voluntarily participate in the Two-Week Follow-Up Survey. Assuming the target sample size for the Immediate Survey is met, the expected number of respondents to the Two-Week Follow-Up Survey is 310. This will give a margin of error of +/-6% on an estimate of 50% of respondents feeling positively about the utility of the contact with the hotline.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument(s)

The Evaluation of the NHTH Program will use two instruments for this data collection (see Instrument 1: NHTH Immediate Survey and Instrument 2: NHTH Two-Week Follow-Up Survey). Table 1 presents the instruments and survey items by research question.

Table 1. Research Questions by Instrument

Instrument	Item Number
RQ1. To what extent do contactors feel that their call/SMS text/live online chat session with the NHTH was helpful?
NHTH Immediate Survey	Item 1
NHTH Two-Week Follow-Up Survey	Items 4 and 5
RQ2. To what extent do contactors feel supported during their call/text/chat session?
NHTH Immediate Survey	Item 2
RQ3. What are contactors’ perceptions of NHTH advocates’ knowledge and skills (i.e., how confident are users in those offering hotline services)?
NHTH Immediate Survey	Item 3
RQ4. Overall, how satisfied are contactors with their call/text/chat session with the NHTH?
NHTH Immediate Survey	Items 5 and 6
RQ5. To what extent are NHTH contactors’ needs met during their call/text/chat session?
NHTH Immediate Survey	Item 4
NHTH Two-Week Follow-Up Survey	Items 1, 2, 3, 4, 5 and 15
RQ6. To what extent do contactors feel that the assistance or information they received from the NHTH was useful?
NHTH Immediate Survey	Items 7a, 8a, 7b and 8b
NHTH Two-Week Follow-Up Survey	Items 7, 8, 16, 17 and 18
RQ7. Do contactors’ knowledge (e.g., of human trafficking, how the hotline can help, safety planning, available resources) increase as a result of their call/text/chat session?
NHTH Two-Week Follow-Up Survey	Items 6 and 18
RQ8. Do contactors use the resources and referrals they are provided during their call/text/chat session?
NHTH Immediate Survey	Items 7a, 8a, 7b, 8b
NHTH Two-Week Follow-Up Survey	Items 9, 10, 11, 12, 13 and 14

The questions for the Immediate Survey and Two-week Follow-up Survey were developed based on a review of survey questions in published evaluations of hotlines and helplines (see Table 2). The constructs measured in these previous studies were mapped to the research questions specific to this evaluation. The survey items associated with relevant constructs were compared and assessed for appropriateness with the context of the NHTH and the respondent population. Additionally, we incorporated input from Polaris staff (including Polaris’ survivor engagement advisor), ten non-federal experts in the fields of human trafficking and victim services, research, and hotline services and evaluation (four university researchers, four RTI research scientists, one service provider, and one survivor expert) (see Supporting Statement Part A, Section A8) and findings from the study’s earlier formative evaluation activities (see Supporting Statement Part A, Section A2).

Table 2. Evaluation Instruments Reviewed

Hotline	Publication
Rape, Abuse, & Incest National Network (RAINN)	Finn, J., Garner, M. D., & Wilson, J. (2011). Volunteer and user evaluation of the national sexual assault online hotline. Evaluation and program planning, 34(3), 266-272.
RAINN	Finn, J., & Hughes, P. (2008). Evaluation of the RAINN national sexual assault online hotline. Journal of Technology in Human Services, 26(2-4), 203-222.
City of Chicago Domestic Violence Help Line	Fugate, M., George, C., Haber, N., & Stawiski, S. (2005). Providing a Citywide System of Single Point Access to Domestic Violence Information, Resources, and Referrals to a Diverse Population: An Evaluation of the City of Chicago Domestic Violence Help Line. National Institute of Justice.
Telephone crisis services	Kalafat, J., Gould, M. S., Munfakh, J. L. H., & Kleinman, M. (2007). An evaluation of crisis hotline outcomes. Part 1: Nonsuicidal crisis callers. Suicide and Life-threatening behavior, 37(3), 322-337.
National Domestic Violence Hotline and loveisrespect hotline	McDonnell, K.A., Nagaraj, N.C., Mead, K.H., Bingenheimer, J.B., Stevens, H., Gianattasio, K.Z., & Wood, S.R. (2018). “An Evaluation of the National Domestic Violence Hotline and loveisrespect. A report from the Accomplishments of the Domestic Violence Hotline, Online Connections, and Text Project.” Prepared for the Administration for Children & Families, U.S. Department of Health and Human Services.
Dutch AIDS STI Helpline	Mevissen, F. E., Eiling, E., Bos, A. E., Tempert, B., Mientjes, M., & Schaalma, H. P. (2012). Evaluation of the Dutch AIDS STI information helpline: Differential outcomes of telephone versus online counseling. Patient education and counseling, 88(2), 218-223.
Illinois’ domestic violence crisis line	Riger, S., Bennett, L., Wasco, S. M., Schewe, P. A., Frohmann, L., Camacho, J. M., & Campbell, R. (Eds.). (2002). Evaluating services for survivors of domestic violence and sexual assault. Sage Publications.

B4. Collection of Data and Quality Control

Data collection will begin after OMB approval and continue for approximately four months.

Immediate Survey

During the data collection period, NHTH advocates will inform adult NHTH contactors of the Immediate Survey at the completion of their call, SMS text session, or live online chat session (see Appendix A: Recruitment Script for NHTH Advocates).

• Calls: Advocates who interact with NHTH callers will read the telephone recruitment script at the completion of their call. They will then transfer the caller to an IVR telephone survey before disconnecting from the call.

• Texts or Chats: Advocates who interact with NHTH contactors through the NHTH’s SMS text line or live online chat mode will copy and paste a standardized script and a unique survey Web link (i.e., URL) into the text or chat screen. RTI will provide a password-protected spreadsheet to the NHTH that contains a list of pre-generated unique survey Web links. Advocates will access the shared spreadsheet (stored in the NHTH’s secure Cloud) to pull a unique survey Web link to provide to the text or chat contactor. After selecting a survey Web link, advocates will indicate that the link was distributed and specify the date and time the text or chat session ended beside the given Web link.

• Transfers to Third-Party Service Provider: Some NHTH callers are transferred directly to a third-party service provider by the NHTH advocate. So as to not interrupt their receipt of services, NHTH advocates will offer callers who are connected directly to a service provider the opportunity to participate in the survey by offering to provide the toll-free telephone number that the caller can contact later to complete the survey (via IVR) and receive a $10 gift card. Callers who agree to receive the information will be given the survey IVR telephone number by the advocate, as well as a 4-digit personal identification number (PIN) to use to enter the survey when they call the toll-free survey telephone number. RTI will provide a password-protected spreadsheet to the NHTH that contains a randomly generated PINs. Advocates will access the shared spreadsheet (stored in the NHTH’s secure Cloud) to pull a unique PIN to provide to the caller. After selecting a PIN, advocates will indicate that the PIN was distributed and specify the date and time the call ended beside the given PIN.

The recruitment script includes scripted answers to potential anticipated questions the NHTH advocates may receive from NHTH contactors about the survey to reference when and if necessary.

IVR Survey Responses: After respondents are connected to the IVR survey, either by direct transfer by the NHTH advocate or by calling into the IVR line and successfully entering their PIN, they will first hear the pre-recorded consent language. The consent language will emphasize that the survey is voluntary, respondents’ answers will be kept private to the extent permitted by law, and that they can skip answers or end the survey at any time. Respondents will press a specified number on their keypad to indicate their consent to participate. Respondents will hear the survey questions and use their phone keypad to respond. At the end of the survey, respondents who called in to the IVR survey (i.e., those who had been transferred directly to a third-party service provider by the NHTH and called to complete the survey later) will be prompted to enter a cell phone number that can receive SMS text messages or an e-mail address to receive their $10 gift card.

Web Responses: Individuals who receive a Web link to the Immediate Survey via text or chat will see the consent language on the first page of the Web-based Immediate Survey after clicking their unique Web link. After indicating agreement to participate by clicking the begin button on the screen, respondents will complete the survey questions (see Instrument 1: NHTH Immediate Survey for consent and survey questions).

Two-Week Follow-Up Survey

All respondents who complete the Immediate Survey via IVR or Web will be invited at the end of their survey to participate in the Two-Week Follow-Up Survey (see question 9 in Instrument 1: NHTH Immediate Survey). If they indicate interest in participating in this second survey (by selecting “yes”), they will specify their preference for how to be contacted (telephone, text, or e-mail), provide their telephone number or e-mail address (depending on the mode they choose), and create a 4-digit PIN. Respondents who choose telephone or text will specify their time zone, the best time of day to be safely contacted, and if leaving a voicemail is permissible (telephone mode only).

Individuals who elect to participate in the Two-Week Follow Up Survey will be contacted by RTI through their preferred mode (phone, text, or e-mail) approximately two weeks from the date on which they completed the Immediate Survey.

• Phone Contact: Respondents who elect to be contacted by phone will be contacted by a trained telephone interviewer (TI) at the RTI Call Center during their preferred time of day in their time zone (based on the information they provided at the end of their Immediate Survey). When the TI begins a telephone conversation with a person to complete the Two-Week Follow Up Survey, the TI will first determine whether the person is the correct respondent and able to talk at that time (see Appendix B: Recruitment Script for Two-Week Follow-Up). After the respondent confirms and provides the correct PIN, the TI will read the consent (see Instrument 2: Two-Week Follow-Up Survey for consent and questions. Following consent to participate, the TI will read the survey questions and enter the respondent’s answers in the CATI system. At the end of the survey, the TI will collect a cell phone number that can receive SMS text messages or an e-mail address to which to send phone respondents’ $10 gift card.

• E-mail or SMS Contact: Respondents who elect to be contacted by e-mail or SMS will be sent a unique Web link to access the Two-Week Follow-Up Survey (see Appendix B: Recruitment Script for Two-Week Follow-Up). After clicking their unique Web link, respondents will be prompted to enter their PIN. After entering the correct PIN, the respondent will be taken to the first page of the Web-based Two-Week Follow-Up Survey on which they will see the consent language. After indicating agreement to participate by clicking the begin button on the screen, respondents will complete the survey questions (see Instrument 2: Two-Week Follow-Up Survey). Web-survey respondents will receive their $10 gift card via the same mode they received the survey, either via SMS text or e-mail.

We will implement several precautions to protect the safety and privacy of all contactors who choose to participate in the study. TI training will cover study recruitment and data collection protocols; privacy guidelines and procedures; protocol for responding to participant distress; and procedures for data security and monitoring. When contacting telephone respondents for the Two-Week Follow-Up Survey, TIs will be trained to safely confirm that the person they are talking to is the person who completed the Immediate Survey, without revealing that the survey pertains to the NHTH. Voicemail messages will only be left for respondents who have given permission and the term “Customer Satisfaction Survey” will be used when referring to the survey or study in all respondent communications, including e-mails and text messages.

Immediate Survey respondents who call in to the survey and do not know their PIN will be prompted press a designated number on their telephone keypad to be transferred to a survey representative. Two-Week Follow-Up Survey respondents will be prompted to contact the toll-free survey phone number if they forget their PIN or have questions. When a respondent is transferred to or calls the toll-free number for assistance, the RTI survey representative (i.e., one of the trained TIs) will request their date of first contact and the number that they contacted or the service that is associated with the “Customer Satisfaction Survey.” Only if the respondent satisfactorily provides the requested information will the representative provide their PIN (in the case of Immediate Survey respondents and Two-Week Follow-Up Survey respondents who received a survey Web-link) or continue to the consent (with Two-Week Follow-Up Survey telephone participants). Additionally, the final screen of both Web-based surveys will provide information on how to clear the respondent’s search history in their internet browser. Participants may opt out of additional survey contacts by calling the toll-free survey number.

RTI will request a limited set of administrative data from Polaris comprised of information routinely collected by NHTH advocates about NHTH contacts.⁵ The data will include date and time (start and stop) of the NHTH contact; contactor location (state); whether the contactor was anonymous; whether the contactor was a first time contactor; contactor type (e.g., potential victim, friend of a potential victim, service provider), whether the contact was trafficking related; reason for the contact; whether the contactor was a minor; and whether (for callers only) the contactor was transferred to a victim service provider. These data on the nature and length of contact from the hotline calls will be used to provide context to survey findings as well as to assess bias in the types of contactors who completed the survey (see Section B5, Nonresponse).

Data Quality and Consistency

IVR, web-based surveys, and computer-assisted data collection improve survey data quality by eliminating routing errors, implementing logical range checks, and increasing response rates. Data collected by the TIs have the greatest risks to data quality and consistency. To establish standardized and high-quality data collection, we will develop detailed data collection procedures, conduct training for all NHTH hotline advocates focused on survey recruitment, and conduct training for TIs focused on correct implementation of the survey protocol. The data collection procedures will include scripts for TIs for study recruitment, respondent consent, obtaining information for the tokens of appreciation, and how to address frequently asked questions. The NHTH hotline advocate training will include instructions on how to transfer contactors to the survey, review of the script for individuals who would receive the first survey after a transfer to a service provider, and how to assign and manage PIN numbers. During the TI training, each TI will role-play survey administration and be evaluated on their performance until each TI is able to demonstrate appropriate and accurate data collection. The final component of the training will involve certification of TIs in key areas of consent form administration, survey administration, addressing questions, distress protocol, and safety measures.

During data collection, several data quality monitoring activities will be employed on the study, including regular monitoring of TIs and NHTH hotline advocates, and data frequency reviews (e.g., weekly review of number of respondents). RTI will provide training to NHTH advocates and NHTH advocate managers will monitor hotline advocates’ performance as part of their routine quality assurance monitoring. The RTI TI supervisor will monitor TI staff through live monitoring of select interviews to confirm protocol compliance, appropriate respondent/interviewer interaction, and proper coding of responses.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

A number of features are built into the survey methodology to maximize response rates and minimize respondent burden without compromising respondent safety or the voluntary nature of participation. First, we have developed a mixed-mode protocol for both surveys that uses technology to reduce respondent burden and enable completion of the survey in the potential respondent’s preferred mode. For the Immediate Survey, contactors who called the NHTH will be transferred to an IVR survey, enabling them to simply stay on the phone in order to complete the survey. Chat and text contactors will be similarly provided a unique Web-link that will enable them to complete the survey on the same device being used for contact with the NHTH. For the Two-Week Follow-Up Survey, respondents will have the option of receiving a call from an interviewer at a phone number and time that is convenient and safe, receiving an SMS text message with a link to the Web-based survey, or receiving an e-mail with a link to the Web-based survey. Second, NHTH advocates will be thoroughly trained on the brief protocol used for transferring contactors to the Immediate Survey and NHTH leadership impress on advocates the importance of carrying out the protocols as designed. Interviewers who will conduct the Two-Week Follow-Up Survey and/or communicate with respondents who do not remember their access PIN will be similarly trained on the survey protocols, on communicating with contactors in a compassionate manner, and on methods for ensuring respondents’ safety and privacy of data. Third, as described in B4, although respondents will be required to use a PIN to access the surveys if they are not directly transferred in, we have built in protocols to enable them to still complete the survey if they forget the PIN, assuming there are no concerns regarding security or duplication of responses. Fourth, the length of both surveys has been limited (approximately 3 minutes for the first survey and about 6 minutes for the second survey), asking the minimum number of questions needed to address the research questions, and the consent language has been kept as concise as possible to increase the likelihood of participation and reduce respondent burden and breakoffs. Finally, we are proposing to offer a $10 gift card token of appreciation upon completion to respondents who have to actively call into the Immediate Survey because they were directly transferred from the NHTH to a service provider, and to all respondents who complete the Two-Week Follow-Up Survey. Although the time burden is not substantial for either of these two surveys, the act of participating and utilizing a PIN to do so requires extra effort on the part of respondents and the token of appreciation is intended to help mitigate that effort. Without the use of tokens of appreciation for these NHTH contactors, we foresee the following challenges: (1) inability to recruit a sufficient sample of individuals who receive a direct transfer to a service provider from the NHTH (i.e., low response rate for an important sub-population of NHTH contactors), (2) inability to maintain continued participation in the two-week follow up (i.e., low response rate for longitudinal data collection), and (3) less representative sample of NHTH contactors who participate in both surveys (i.e., nonresponse bias).

Because this is the first effort to collect data from individuals who contacted the NHTH, there is limited data on which to base assumptions about response rates with this population. Estimated response rates are based largely on similar survey efforts conducted by other hotlines, coupled with formative research on the characteristics and circumstances of NHTH contactors and efforts to mitigate nonresponse through the survey protocols.

The National Domestic Violence Hotline (DV Hotline) conducted phone and chat exit surveys using similar questions as proposed for the NHTH Immediate Survey and obtained response rates of approximately 6% among eligible callers and 17% among eligible chat contactors.⁶ Approximately 20% of NHTH contactors use chat or text and a similar response rate of 17% is assumed for these contactors (n=340). Among callers who are immediately transferred to the Immediate Survey, we assume a slightly higher response rate of about 9% because the estimated time required for callers to complete the NHTH Immediate Survey (3 minutes) is less than half of the estimated burden for the DV Hotline exit survey (8.5 minutes).

No prior efforts that we are aware of have attempted to conduct a follow-up survey with hotline contactors who are transferred directly to a service provider, so response rates among this group are difficult to estimate. Despite offering a $10 gift card token of appreciation for those who are willing and able to call into and complete the survey, we anticipate that less than 5% of this relatively small but important group of contactors will participate. Response rates are expected to be lower among this group than the other contactors because respondents are being asked to proactively call into the survey, rather than being transferred. Additionally, respondents who are in crisis and seeking shelter and services are less likely to be able to complete a survey than other callers (Campbell, Adams, & Patterson, 2008).

A response rate of 30% is estimated for the Two-Week Follow Up Survey. Safeline, a suicide prevention hotline was able to obtain a 45% response rate to a follow-up survey of users administered 4-weeks after the contact (Gould, Kalafat, Munfakh, & Kleinman, 2007). Based on this finding, the DV Hotline estimated a response rate of 35% for their two-week follow-up survey effort. Although, we propose to offer a $10 gift card token of appreciation for completion of the Two-Week Follow-Up Survey in order to increase response rates and reduce nonresponse bias, given the transient nature of many of the NHTH contactors, we anticipate a slightly more conservative response rate among NHTH contactors.

Nonresponse

Participation in the data collection is voluntary and will include only those NHTH contactors who agree to participate. Participation in the surveys will not impact the services that contactors receive from the NHTH.

For the Immediate Survey, if contactors do not wish to be transferred to the survey or choose not to complete the survey after hearing or reading the consent information, they do not have to participate and no further attempt at obtaining a response will be made. Additionally, contactors who consent to participate in the Immediate Survey may choose to skip questions or end the survey at any point if they become distressed, have safety concerns, or do not want to continue for any other reason.

Respondents to the Immediate Survey who agree to participate in the Two-Week Follow-Up Survey by phone will specify a phone number and time of day at which contact is safe. They will not schedule a specific date but will be told that the follow-up will be in approximately two weeks. They will also be asked whether it is safe for interviewers to leave a voicemail message. If the individual does not answer the phone when the first call is placed, three nonresponse follow-up calls will be made at the same time of day on subsequent days. For those respondents who have specified that voicemail messages are permissible, a voicemail message will be left. The voicemail message will provide a telephone number the individual can use to call into the survey if he or she chooses to participate. No further follow-up attempts will be made after the third call.

Respondents to the Immediate Survey who agree to be contacted for the Two-Week Follow-Up Survey via text message or e-mail will be sent up to three messages. If the respondent has not accessed the link provided in original message within two days, a reminder message will be sent on day three and another reminder message on day five. The reminder messages will use similar language as the initial message but will be altered slightly to include language referring to them as the “first reminder message” and “final reminder message,” respectively. No further attempts at contact will be made. However, the Web survey link will remain open throughout the study period, enabling individuals to complete the survey at a later point, if they so choose.

We will monitor response rates on a weekly basis throughout the study period and if response rates are lower than anticipated in the first several weeks, the study team will work with the NHTH staff to examine reasons for non-participation and to strategize ways to increase response rates.

Once data collection is complete, the data will be analyzed using basic frequencies and crosstabs to assess levels and patterns of item missingness. Item non-response is expected to be small due to the short length of the survey, but because respondents are able to skip survey questions as needed, both surveys will contain missing data. Because the surveys will be programmed in IVR, CATI, and Voxco Web, it will be possible to control for other potential data quality issues such as incorrect skip patterns and out of range responses in the programming and these should not be an issue for data analysis.

A nonresponse bias analysis will also be conducted at the completion of data collection. Polaris will provide a limited set of administrative data collected from all NHTH contacts (calls, texts, online chats) during the data collection period. The variables will include date and time (start and stop) of the NHTH contact; contactor location (state); whether the contactor was anonymous; whether the contactor was a first time contactor; contactor type (e.g., potential victim, friend of a potential victim, service provider), whether the contact was trafficking related; reason for the contact; whether the contactor was a minor; and whether (for callers only) the contactor was transferred to a victim service provider. Unfortunately, basic demographic data, like age, race, and sex, cannot be used for nonresponse bias analysis because these fields are only collected when the contactor chooses to provide that information and have high levels of missingness.

Data from the Immediate Survey will be linked to the NHTH data using time stamps from both datasets. We will assign a unique ID number to each respondent to the Immediate Survey and collect timestamp data (end of NHTH contact and IVR system engagement). Timestamp data and ID number (associated with the unique Web survey link in the case of a or chatter) will be used to link respondent data from the Immediate Survey and Two-Week Follow Up Survey and to link data from both surveys to a limited set of variables from the initial NHTH contact routinely collected by NHTH advocates. Once the linkage is done, it will be possible to examine whether survey respondents were systematically different from nonrespondents across available measures. We will also assess the feasibility of weighting data from the Immediate Survey to the total population of contacts during the data collection period to address any observed non-response bias.

B6. Production of Estimates and Projections

Substantive and methodological findings generated through this effort are largely experimental and will be used internally to increase understanding of contactors’ perceptions of the services provided by the NHTH, improve NHTH Program services, and assess the feasibility of routinely collecting data from NHTH contactors for the purpose of performance measurement. Additionally, other victim service hotlines may use study findings to develop performance measurement approaches and improve hotline services.

Data from the surveys will be linked to administrative data from the original NHTH contact using time stamps from both collections. The NHTH data will be used to provide context to the key survey estimates, and as noted in section B5, will be used to assess the feasibility of weighing the survey estimates. Written materials and table notes will include appropriate discussion about relevant data quality considerations. The key estimates to be generated from the two surveys are described in more detail in the section B7.

B7. Data Handling and Analysis

Data Handling

RTI staff will thoroughly test the instruments to ensure that the surveys are programmed into IVR, CATI, and Voxco Web accurately and in a manner that minimizes error. This includes automated and manual checks on the programming of each system to ensure that skip patterns are properly programmed, response options are within allowable ranges.

The surveys will be hosted in RTI's National Institute of Standards and Technology (NIST) Moderate network and as data are collected, they will be stored in SQL server databases within RTI’s NIST Moderate network.

Data Analysis

Data from the Immediate Survey will be used primarily to assess contactors satisfaction with the services provided by NHTH advocates. The key estimates to be produced—which were used as the basis for target sample size calculations—are the percentages of respondents who found the service to be helpful; felt supported during the interaction; found the advocates to be knowledgeable; felt their needs were met; were satisfied with the contact; and were likely to recommend the NHTH to someone else.

Because the reason for and nature of the NHTH contact provide important context for understanding these key estimates, data from the Immediate Survey will be linked to a limited set of administrative data from the NHTH using time stamps from both data collections. The administrative data will be treated as independent variables, used for understanding basic patterns in contactors’ outcome measures. As previously described in section B5, administrative data provided by the NHTH will include date and time (start and stop) of the contact, contactor location (state), whether the contactor was anonymous; whether the contactor was a first time contactor; contactor type (e.g., potential victim, service provider); whether the contact was trafficking related; reason for the contact; whether the contactor was a minor; and whether (for callers only) the contactor was transferred to a victim service provider.

For analysis of the Immediate Survey, we will first conduct a factor analysis to assess whether the key survey questions about customer satisfaction load onto one or more dimensional factors or whether they capture separate constructs and should be independently examined. The rest of the analysis of will consist primarily of descriptive statistics and crosstabulations to assess patterns in satisfaction levels, by the characteristics of the contact. A key consideration will be ensuring that the findings for the primary respondent group of interest – those who contacted the NHTH for a trafficking case-related reason – can be broken out separately from the non-trafficking contactor. Statisticians will use SAS and SPSS for data analysis.

Data from the Two-Week Follow-Up Survey will be linked to the Immediate Survey using ID numbers assigned to each respondent behind the scenes by RTI staff and will similarly be linked to the NHTH data. Analysis of the second survey will again rely on descriptive statistics and crosstabulations for the close-ended quantitative data fields and will focus predominately on perceptions of the utility of referrals and information provided by advocates. Key estimates include the percentages of respondents who felt their needs were met; who learned more about various aspects of trafficking and services available through the contact; who felt the information provided was useful; and who received assistance from another provider based on a referral from the NHTH. While the confidence intervals around the estimates for the Two-Week Follow-Up Survey will be larger than for the Immediate Survey, it will still be essential to examine contactors who had a trafficking case-related reason for the contact separately from those who did not, to the extent possible. The follow-up survey will also ask respondents to self-report the reason for the contact, enabling assessment of the degree of overlap between how contactors and advocates describe the purpose of the contact.

Because of the broad range of potential reasons that respondents may not be able to use or may not want to use the resources provided to them by NHTH advocates, the Two Week Follow-Up Survey also contains open-ended text fields for certain questions, enabling respondents to provide their own reasons for why the resources were not used. RTI will use content analysis to code and analyze these qualitative data fields.

Data Use

A summary report that describes the data collection and presents study findings will be publicly released.

B8. Contact Person(s)

Mary Mueggenborg, ACF Office of Planning, Research, and Evaluation, [email protected]

The RTI project team includes:

Jennifer Hardison Walters, MSW, Project Director, [email protected]

Kathleen Krieger, MPH, Associate Project Director, [email protected]

Lynn Langton, PhD, Research Criminologist, [email protected]

Samantha Charm, MPH, Research Analyst, [email protected]

Megan Waggy, MA, Survey Specialist, [email protected]

Matthew Benson, MA, Lead Survey Programmer, [email protected]

David Foster, Senior Systems Engineer, [email protected]

Ron Thigpen, Web Applications Developer/Analyst, [email protected]

Attachments

Instrument 1: NHTH Immediate Survey

Instrument 2: NHTH Two-Week Follow-Up Survey

Appendix A: Recruitment Script for NHTH Advocates

Appendix B: Recruitment Script for Two-Week Follow-Up

References

Campbell, R., Adams, A. E., & Patterson, D. (2008). Methodological challenges of collecting evaluation data from traumatized clients/consumers: A comparison of three methods. American Journal of Evaluation, 29(3), 369-381.

Gould, M. S., Kalafat, J., Munfakh, J. L. H., & Kleinman, M. (2007). An evaluation of crisis hotline outcomes, part II: Suicide callers. Suicide and Life-Threatening Behavior, 37, 338-352.

1 Throughout we primarily use the term “victim” to refer to individuals who have experienced human trafficking;

however, we acknowledge that some individuals who have experienced trafficking may identify as a survivor or not identify as a victim nor survivor of human trafficking.

2 National Human Trafficking Hotline https://humantraffickinghotline.org/national-hotline-overview

3 Throughout we use the term “contactors” as the term for individuals who contact the NHTH through phone, SMS text, or live online chat.

4 A slightly better response rate of 20% would provide a sample size of 2,000 respondents. To account for the possibility that response rates may be higher than 10.5%, the burden request is for 2,000 respondents rather than the minimum target sample size.

5 NHTH advocates record data about the timing, length, nature of all contacts and actions taken by the NHTH during and following a contact. NHTH contactors are categorized by NHTH staff as either making contact for a specific trafficking case-related reason (e.g. a victim of human trafficking seeking resources; persons providing a tip about a potential human trafficking situation; persons seeking information to help a trafficking victim) or for a reason not related to a particular trafficking case, which could include seeking general information about human trafficking, as well as contactors who erroneously contact the NHTH for an issue unrelated to human trafficking entirely. Victims of trafficking who need assistance or community resources are often transferred directly to an outside victim service provider by the NHTH advocate.

6 These findings were reported in the OMB Information Collection Review for a larger 2017 evaluation of the DV Hotline (OMB number 0970-0468).

9