Coverage and Nonresponse Bias Follow Up Study

The Coverage and Nonresponse Bias Follow Up Study for Veteran Experience Trust Measure Survey

Introduction

The Coverage and Nonresponse Bias Follow Up for VE Trust Measure Survey is designed to investigate the reasons that prevent the sampled veterans or beneficiaries from participating in the quarterly Veteran Experience Trust Measure Survey. To increase response rate and enhance the rate of successful contact, the Coverage and Nonresponse Bias Follow Up Survey will rely on Computer Assisted Telephone Interview (CATI) mode to reach the sampled participants that did not response to the online VE Trust Measure Survey. The purpose of this document is to detail the methodology of the Coverage and Nonresponse Bias Follow Up for VE Trust Measure Survey.

Methodology

Survey mode and instrument design.

The Coverage and Nonresponse Bias Follow Up Study is designed as a telephone survey using a Computer Assisted Telephone Interview (CATI) system. The survey instrument consists of 4 sections: (1) access to Internet, smartphone, and email, (2) VE Trust measures, (3) Reasons of not responding to the online VE Trust Measure Survey, and (4) demographic information. These questions are developed to understand and identify the potential obstacles that prevents VA customers from participating the online Trust Measure Survey, while collecting participants’ trust of the most recent VA services they used in the past quarter and demographic information to evaluate potential coverage and nonresponse biases to the Trust Measure Survey.

The interview, including the consent process, will take approximately 10 minutes to administer. RTI’s data collection staff will program the instrument in the CATI system. A maximum of three callback attempts will be made to reach the same telephone number when the initial call cannot make through and establish contact. The sampled case will be marked as non-respondent when RTI’s phone interviewers are not able to reach the participant.

Sampling

The sampling frame of the Coverage and Nonresponse Bias Follow Up is consistent of the non-respondents to the 2017 Q2 VE Trust Measure Survey. A subset of random sample will be drawn from the non-respondents following the same sampling strata and rules that constructed the sample of the main survey.

The minimum requirement of the sample data consists of the unique identifier, the telephone number, the full name of the sampled veterans or beneficiaries, and the date and the name of VA service used by the sampled participants.

Data management

Data transfer procedure for the Sample file

Given that the sample file contains personal identifiable information (PII), RTI employs the CATI hosted within its FIPS-moderate enhanced security network to conduct the Coverage and Nonresponse Bias Follow Up phone interviews. The data transfer protocol is developed to ensure the data security during the transfer.

First, RTI creates an internal project storage space within the FIPS-moderate enhanced security network and grants access to the space to the project staff. Next, the project director and survey operation lead will be granted access to VA’s FISMA moderate storage space. RTI’s project team will facilitate the data transfer between two isolated secured networks by using a File Transfer Protocol (FTP) software while connecting to both networks via virtual private network (VPN) connectivity. Once the sample file is transferred to RTI’s secured storage space, the data collection team can access the sample file within the FIPS-moderate network and load the sample into the CATI system for the telephone interview.

Data management, analysis and transfer for survey data

During the interviewing period, the supervisors of the telephone interviewers will have access to the CATI for monitoring interviewer performance and managing survey data. Upon the completion of the telephone interview, the survey data will be transferred to the project storage space within RTI’s FIPS-moderate network. The de-identified survey data will be made available to transfer to RTI’s FIPS-low network for data analysis. The project staff will transfer the de-identified survey data back to VA by encrypted email.

Interviewer Training and survey pretesting

All staff members of the telephone interviewing team will undergo a VA onboarding and security check process. The onboarding process includes the training of VA ethics, data confidentiality and HIPAA compliance. The team will also undergo interviewer training that covers interviewer responsibility, respondent’s rights and confidentiality, strategies of gaining cooperation, techniques to avoid refusals and distressed respondents, and specifically the content of the Coverage and Nonresponse Bias Follow Up questionnaire.

The instrument will undergo an internal pretesting to confirm the accuracy of the instrument and the CATI is operating properly. To identify any potential errors in the sample, the questionnaire, and the CATI and to prepare the interviewers further, RTI will conduct a brief live-pretesting to complete 10-20 interviews with the sampled respondents.

1