10 day Letter

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

December 19, 2017

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, exp. 07/31/2018) has entered into a receivable reimbursable Interagency Agreement (IAA) with the Assistant Secretary for Planning and

Evaluation (ASPE) and plans to conduct an independent cognitive interviewing study to examine Criminal Justice System Questions for ASPE.

We propose to start recruiting for volunteer participants as soon as we receive clearance and to start testing as soon as possible after that.

Background Information about Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. The purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Proposed project: ASPE Criminal Justice System Involvement survey questions

Purpose of testing and importance of the topic

ASPE is asking NCHS (CCQDER) to test criminal justice system questions in order to understand the constructs of arrest, conviction, and incarceration. This will provide a two-fold benefit of improving the utility of information that is already collected and providing empirical evidence of question performance for data collection programs that may be considering adding such measures.

Understanding arrest, conviction, and incarceration is important to the HHS mission. Collecting data on this population supports HHS in both its ability to monitor the health status and disparities experienced by this population, and in developing policies and programs to improve health. The United States incarcerates more people than any other country. While the United States represents about 4.4 percent of the world’s population, it houses around 22 percent of the world’s prisoners (Walmsley, 2013). Roughly eight percent of children in the United States will have their father imprisoned by the time they reach 14 years of age. There are also clear disparities in imprisonment with 25 percent of African-American/Black children and three percent of White children experiencing this event at some point in their childhood (Wildeman and Anderson, 2005; Wildeman, 2009). It is important to capture information on this non-trivial proportion of the population.

Additionally, there is growing evidence that contact with the justice system is connected to health disparities. Previously incarcerated individuals display higher rates of substance abuse, engagement in risky sexual behaviors, intermittent homelessness, and on average have poorer health outcomes than the general population (National Commission on Correctional Care, 2006; James and Glaze, 2006; CDC, 2006). There is also evidence of higher mortality rates for former inmates, especially the first months after release (Spaulding et al., 2010; Binswanger et al., 2007). Therefore, not accurately measuring or capturing this information makes HHS population-based health surveys less comprehensive. These negative outcomes are exacerbated by disproportionate representation of racial and ethnic minorities in the justice system. These same minority groups evidence health disparities among the general population (Heckler, 1985).

In ASPE-funded research on the availability of data sources to address the opioid epidemic, state all-payer claims databases and criminal justice data were consistently noted by stakeholders as valuable to the HHS response (Smart et al., 2017). The report recommends that person-level linkages of public health datasets with criminal justice data on arrests and incarcerations could be of value to HHS programs addressing the opioid crisis. There has been prior research that took de-identified data that links state administrative data on clients receiving publically-funded substance abuse treatment to data from state criminal justice agencies and small-area analyses of drug seizure data and drug overdose data. However, a partnership between the Department of Justice and HHS to develop data systems focused on criminal justice data and opioid use could potentially provide researchers with de-identified files that would support analyses at a more granular level of geographic detail. Refining measurement of justice system involvement among the general public would also improve data and research in this overlapping area.

Finally, the bulk of BJS data collections and survey items are targeting individuals that are already incarcerated. They do not ask questions about justice involvement. The two household surveys that BJS conducts, the National Criminal Victimization Survey (NCVS) and the National Intimate Partner and Sexual Violence Survey (NISVIS) do not contain questions about justice involvement. Thus data collected by BJS does not provide a complete picture of individuals with a history of justice involvement. BJS staff have acknowledged the importance of data on the individuals with a history of justice involvement in HHS data collections.

Collaboration in question development

This project is congruent with other ASPE collaborative efforts. In general, the Division of Data Policy works to improve the data capacity, quality, and utility of HHS data resources to help support policy decisions. For example, Data Policy has worked with CCQDER in the past to cognitively test and develop questions on sexual orientation and gender identity (SOGI). The division has also supported other data alignment efforts throughout HHS.

Collaborative efforts on this specific topic have also occurred. For example, ASPE previously sponsored Measuring Incarceration in Household Survey Invitational Forum and the resulting Survey on Criminal Justice Experience (https://www.bgsu.edu/content/dam/BGSU/college-of-arts-and-sciences/NCFMR/documents/Research%20Conferences/Measuring%20Incarceration/SCJE.pdf) that studied the prevalence of currently and formerly incarcerated individuals in the United States.

In addition, BJS participated in the CNSTAT workshop, and contributed to the discussion and workshop proceedings that led to the development of the questions ASPE and CCQDER wish to test. The workshop (entitled: Improving Collection of Indicators of Criminal Justice System Involvement in Population Health Data Programs. See http://sites.nationalacademies.org/DBASSE/CNSTAT/DBASSE_170301) gathered experts on criminal justice, health, and survey methodology for two days and produced a list of questions. All of the questions from the workshop are the items we wish to cognitively test. We want to see which of these measures and questions make sense to respondents and accurately capture the underlying constructs of interest (i.e., arrest, conviction, and incarceration). This workshop and the aforementioned forum demonstrate ASPE’s efforts to partner with subject matter experts in order to develop accurate measures of justice involvement in the general public.

Finally, ASPE also reached out to colleagues at BJS and asked for their additional input. BJS reviewed the list of questions we wish to test and provided feedback. They also stated that they fully support this work and are looking forward to the final report. They have had discussions with both SAMHSA and CDC about the importance of including justice involvement questions to HHS household surveys based on the recommendations of the CNSTAT workshop. BJS confirmed that they do not ask justice involvement questions in their household surveys and do not include them on their institution surveys because the population is already incarcerated. As part of our outreach to prepare for this project, BJS staff have shared that HHS survey data provide the only source for household estimates of CJ [criminal justice] involvement along with health and substance use data.

Eventual use of the questions

Once we have completed the cognitive testing and have the final report and recommendations, we will coordinate across HHS to determine how we can improve existing measurement and areas where there may be opportunities to add measures. In the past the Division of Data Policy has supported the cognitive testing of sexual orientation and gender identity (SOGI) questions to improve measurement and data quality in this area. The goal of this project is also to improve data quality. As stated earlier, this is part of the work that the Division of Data Policy conducts.

Plan for the CCQDER study

CCQDER will conduct and analyze 45 cognitive interviews in iterative rounds. The goal is to determine whether the tested items accurately measure arrest, conviction, and incarceration. This is accomplished by examining the types of experiences respondents include in their answers and any difficulties they experience when answering. Difficulty answering questions may include trouble with comprehension, recall, phrases, and terminology, and/or trouble with proxy reporting (i.e., answering for a spouse). Systematic analysis will identify constructs captured by the questions as well as the potential for (and types of) response error.

The Criminal Justice System questions we are evaluating, as well as the survey question reference/justification, are included as Attachments1a and 1b, respectively. While none of the proposed questions have been used on any other surveys, we acknowledge within attachment 1b items from the National Survey on Drug Use and Health (NSDUH) OMB No. 0930-0110, Exp. 07/31/19) and the National Survey of Family Growth (NSFG) OMB #0920-0314, Exp. 05/31/18) that have some similarities. The testing procedure conforms to the cognitive interviewing techniques that have been described in CCQDER’s generic OMB clearance package (No. 0920-0222, exp. 07/31/2018).

We propose to recruit 45 English speaking adults (aged 18 and over) who have been arrested, convicted, and/or served jail time. We will also recruit some individuals who have a spouse or family member who has been arrested, convicted, and/or served jail time. Recruitment will be carried out through a combination of a newspaper advertisement, flyers, and word-of-mouth. The newspaper advertisement/flyer used to recruit respondents is shown in Attachment 2. The five-minute screener used to determine eligibility of individuals responding to the newspaper advertisement/flyer is shown in Attachment 3. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. It is anticipated that as many as 72 individuals may need to be screened in order to recruit 45 participants.

Interviews averaging 60 minutes (including the completion of a Respondent Data Collection Sheet) will be conducted by CCQDER staff members with English speaking respondents. Interviews will be conducted in the Questionnaire Design and Evaluation Research Laboratory as well as at off-site locations. All interviews conducted in the Questionnaire Design and Evaluation Research Laboratory will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. Interviews conducted off-site will only be audio recorded. These recordings will allow researchers to ensure the quality of their interview notes. In the rare case that a study participant initially agrees to audio recording during the telephone screening, but changes their mind and checks “no” to allowing the interview to be recorded on the informed consent document the interview will proceed without audio recording. In this case the interviewer will depend on their handwritten notes when conducting analysis. In addition, individuals who select “yes” for allowing the audio recording on the informed consent form, but “no” for retaining the recording for future research (final text before signatures on informed consent form), will be allowed to participate in the study.

After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5). This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about your, or your spouse, or your family member’s, experiences being arrested, convicted, or served jail time. We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? If yes, answer questions. If not, let’s get started.

After the interview, respondents will be given the thank-you letter (document contained in umbrella package) signed by the Charles J. Rothwell, Director of NCHS (Attachment 6), a copy of the informed consent document, and $40.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

We propose giving participants $40 incentives, which is our standard incentive. In total, for this project, the maximum respondent burden will be 51 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener	72	1	5/60	6
Questionnaire	45	1	1	45

Attachments (7)

cc:

V. Buie

T. Richardson

DHHS RCO

REFERENCES

1) Roy Walmsley (November 21, 2013). World Prison Population List (tenth edition). International Centre for Prison Studies. Retrieved July 11, 2014.

2) Wildeman C, Andersen LH. (2015) “Cumulative risks of paternal and maternal incarceration in Denmark and the United States”, Demographic Research, 32:1567-1580.

3) Wildeman C. (2009) “Parental Imprisonment, the Prison Boom, and the Concentration of Childhood Disadvantage”, Demography, 46(2):265-280. doi:10.1353/dem.0.0052.

4) National Commission on Correctional Health Care. The Health Status of Soon-to-be-Released Inmates: A Report to Congress. Chicago, IL: National Commission on Correctional Health Care; 2002. (www.ncchc.org/pubs/pubs_stbr.html). (Accessed April 10, 2006).

5) James DJ, Glaze LE. (2006) Mental Health Problems of Prison and Jail Inmates.Washington, DC: Bureau of Justice Statistics, US Department of Justice, DOJ publication no. NCJ 213600.

6) HIV prevalence estimates—United States, (2006) MMWR Morb Mortal Wkly Rep. (2008) 57(39);1073–1076.

7) Heckler, Margaret M. (1985) Report of the Secretary’s Task Force Report on Black and Minority Health Volume I: Executive Summary, US Department of Health and Human Services, Government Printing Office, Washington, DC.

8) Ingrid A. Binswanger, M.D., Marc F. Stern, M.D., Richard A. Deyo, M.D., Patrick J. Heagerty, Ph.D., Allen Cheadle, Ph.D., Joann G. Elmore, M.D., and Thomas D. Koepsell, M.D. (2007) “Release from Prison —A High Risk of Death for Former Inmates”, The New England Journal of Medicine, 356: 157-65.

9) Ryan M. Seals, Victoria A. McCallum, Sebastian D. Perez, Amanda K. Brzozowski, and N. Kyle Steenland. (2011) “Prisoner Survival Inside and Outside of the Institution: Implications for Health- Care Planning”, American Journal of Epidemiology, 173: 479-487.

10) Willson, S., Miller, K., Ryan, M. (2013). Cognitive Interviewing Study Findings of the Uniform Blood Donor History Questionnaire, National Center for Health Statistics (NCHS), Hyattsville, MD.

11) Willson, S. (2013). Cognitive Evaluation of the Donor Risk Assessment Interview(DRAI): Results of Interviews Conducted April – December, 2013, National Center for Health Statistics, Hyattsville, MD.

6 | Page