Description of Program Changes

Attachment J.

National Intimate Partner and Sexual Violence Survey (NISVS)

Program Changes and Adjustments

Changes to the survey

Proposed changes to the survey include:

• Addition of text to brief introductory script about CDC’s mission,

• Addition of text asking permission to send cell phone respondents a text with information about the survey (see number 7, TEXT MESSAGE FOLLOW-UP in the section below)

• Addition of a clarification question about county or resident,

• Addition of a group relationship code,

• Addition of text to the soft check to confirm an age at first that is older than the current age,

• Removal of 4 questions from the normative behaviors section (Section J) that showed limited variability in the response options,

• Shortening of 1 item in Section J to reduce redundancy, and

• Removal of questions for active duty women and men in the military and wives of active duty men, as they will not be a part of the next wave of data collection.)

These changes are described in detail in Attachment J.2. Crosswalk of Survey Changes.

Changes to 2018-2019 cycle based on advice from the NISVS Methodology Workgroup:

In compliance with the OMB’s remaining terms of clearance for 2014 and 2016, CDC has collaborated with BJS to convene a work group to obtain expert feedback and input on how to enhance the NISVS survey. Workgroup participants provided guidance on how to improve the system’s survey design (e.g., methods, sampling frame, recruitment, mode of administration, etc.) with the goals of increasing response rates, reducing non-response bias, and maximizing the collaborative opportunities across Federal surveys for covering populations of interest. Four meetings with the workgroup, which included a representative from OMB, began in February of 2017 and were completed in July of 2017. Key recommendations from the workgroup (Attachment L) have been used to inform both the 2018-2019 efforts as well as plans for the survey design and administration after 2019. CDC proposes to integrate the following activities into the 2018-2019 data collection efforts to address the recommendations provided by the workgroup. Many of the substantive recommendations provided by the Workgroup were cost prohibitive and could not be incorporated into the 2018 data collection period. However, these recommendations will be incorporated into a contract to be funded in the future.

1. COMPARISONS WITH OTHER SURVEYS

One method to evaluate nonresponse bias in NISVS is to use benchmark estimates from other national surveys that have substantially higher response rates and therefore, reduced risk of nonresponse bias in the survey estimates. Weighted estimates are then directly compared and differences interpreted as potential bias, considering differences in the designs of the surveys. By comparing how well NISVS data correspond with data from national surveys that achieved higher response rates, sometimes using other data collection methods (e.g., in person data collection) we can better understand if and how non-response bias is influencing NISVS estimates. Following the NISVS methodology panel, CDC worked with partners to identify several survey questions comparable with NISVS questions that could be used as benchmarks. Three specific health conditions measured both in NISVS and in other large-scale national surveys were identified to serve as benchmarks: hypertension, asthma, and diabetes. The National Health and Nutrition Examination Survey (NHANES), the National Health Interview Survey (NHIS), and the Medical Expenditure Survey (MEPS; a 50% sample from the NHIS but where data are collected on every member of the household) household component are in-person national surveys with higher coverage rates and response rates than RDD telephone surveys. Weighted estimates for the three health conditions were produced from NISVS and compared to those from the three in-person health surveys. Because NISVS asks behaviorally specific questions within a health context, the group found it difficult to identify a violence-related benchmark with which to compare NISVS. One such benchmark, forced vaginal intercourse among 18-44 year old women from National Survey of Family Growth, was identified for comparison. The same sort of comparison can be done as was done with the health conditions, with any differences observed interpreted as potential bias, considering differences in the designs of the surveys. This approach will be used to compare estimates in current and future NISVS data collections to external data collections.

2. Changes to data collection protocol efforts to improve response rate

   1. ADD ATLANTA LOCAL NUMBERS FOR OUTBOUND CALLS

The use of a local telephone number (i.e., a non-800 number) may increase survey participation. For example, an 800 number may imply a corporate solicitation to some. The use of a local phone number (i.e., a 404 or 770 number that is local for both CDC Atlanta offices) will be used to avoid participants that have been dissuaded to answer an 800-phone number. Additionally, an online survey of over 2,300 randomly selected Internet users within the United States found that only 7% of people are likely to answer an unknown caller from a toll-free number but more than twice as many (about 15%) would answer a call from a local number, even if out-of-state (Borowski, 2017).

2. INCREASE THE NUMBER OF TELEPHONE NUMBERS OF EACH TYPE

(800 AND 404/770) FOR OUTBOUND CALLS

To date, an 800 number has been used for outbound calls to NISVS sample members (which potentially appears on sample member’s Caller ID), during both Phase 1 and Phase 2.

Recent years have seen the development and increased use of phone applications that block repeated calls from 800 numbers. Thus, NISVS will no longer be using an outbound 800 number. However, because of the potential use of call blocking software, additional local (770/404) numbers for outbound calls are needed so that outbound phone numbers can be changed more frequently. This may reduce the problem of erroneous flagging and blocking of the study phone number as spam by cell phone carrier applications and potentially increase the number of people who answer the phone. Note that since landline users can also block numbers, additional numbers for that frame are also needed. We recognize that it will be difficult or impossible to evaluate the effectiveness of this specific method. However, a notice released by AAPOR in mid-2017 highlighting the impact of new software designed to block numbers flagged as spam on RDD survey response rates underlines the need to proactively address the problem to avoid potential negative implications in the 2018 data collection period.

Care will need to be taken to determine when a number should/should not be changed. This is of particular concern as it relates to item 6 below (TEXT MESSAGE FOLLOW-UP) as once the answerer gives permission to text, they may recognize the number initially dialed as legitimate and be more likely to pick up the next time it is dialed. This may also be important to consider for landline calls where a message is left on an answering machine or where someone in the household answers and does not have time to talk. It is important to avoid a situation where a potentially interested participant sees a different phone number and does not realize that it is about the study.

c. CUSTOMIZED TEXT TO CALLER ID

Potential respondents might be reluctant to answer the phone if they think that a solicitor is calling. A brief description (instead of a phone number) on the caller ID might increase survey participation. The customized text, “Health and Injuries Study”, will be displayed on caller IDs for landlines. This text display does not apply to the cell phone frame since cell phone caller IDs only receive the phone number.

d. INCREASE LENGTH OF TIME FOR PHASE 2 DATA COLLECTION

NISVS uses a two-phase design. In the first phase, sampled individuals are offered a $10 incentive. In Phase 2, a subset of non-respondents fully worked during Phase 1 are offered a $40 incentive and additional attempts are made to contact these individuals. Phase 2 may help reduce non-response bias by recruiting individuals who may have characteristics similar to non-respondents. To this end, we will start Phase 2 earlier in the data collection period and increase the sample size in Phase 2. Approximately 50% of the Phase 1 sample will be sampled for Phase 2. Phase 2 will start approximately 2-3 weeks earlier than in prior survey administrations. CDC will refine the call protocol and algorithm to determine when Phase 1 should end and Phase 2 should begin.

e. REMINDER POSTCARD

An advance letter is currently sent to potential landline respondents introducing the study, the incentive, and contact information should they have questions (Appendix I). Additionally, the envelope and letterhead used for the advance letter were modified slightly so they now feature the CDC logo more prominently. This could help to increase potential respondents’ recognition of NISVS as a federal survey sponsored by CDC and may help improve response rate. A reminder will be sent to non-respondents approximately 2 weeks after the advance letter. This reminder will re-introduce the study and provide information about the incentive, but this time using an altered format (a postcard, which does not have to be opened, rather than the letter). 

A second postcard reminder highlighting the increased incentive will be sent before the start of Phase 2 to the subset of non-respondents selected for Phase 2. This information may increase response from selected sample households that have received the initial advanced letter but have not yet been reached or who initially declined participation. Proposed language is included below:

Text for Phase 1 postcard: We recently attempted to call you for an important research study conducted by the Centers for Disease Control and Prevention (CDC). Your household has been chosen by chance from among all U.S. households to represent thousands of others like yourselves. If someone in your house is eligible for the study and completes the interview, we will send him/her a $10 check in appreciation. Please call us at 1-800-XXX-XXXX at your earliest convenience.

Text for Phase 2 postcard: We recently attempted to call you for an important research study conducted by the Centers for Disease Control and Prevention (CDC). Your household has been chosen by chance from among all U.S. households to represent thousands of others like yourselves. If someone in your house is eligible for the study and completes the interview, we will send him/her a $40 check in appreciation. Please call us at 1-800-XXX-XXXX at your earliest convenience.

. f. MESSAGE FOLLOW-UP

For those in the cell phone frame that answer their phone but are not willing or able to participate at the time, interviewers will ask permission to send them some information by text message. If granted, a text will be sent to the individual’s cell phone (similar to the advance mailing for those in the landline frame) indicating that they have been selected for a health and injury survey along with details about the incentive amount and a number they can call to participate.

Sampled individuals who indicate that they are younger than 18 years old will be classified as age-ineligible (screened out), and no further call attempts will be made to that number. This could reduce costs and improve the response rate.