A-NCVSResearchSupporting_Statement_3

SUPPORTING STATEMENT

A. JUSTIFICATION

1. Necessity of Information

The Bureau of Justice Statistics (BJS), sponsor of the National Crime Victimization Survey (NCVS; OMB No. 1121-0111), requests a generic clearance to conduct research studies as part of the “Methodological Research to Support the Redesign of the National Crime Victimization Survey (NCVS).” The group of developmental activities described below will be collectively referred to as the NCVS Redesign Research (NCVS-RR) program.

The NCVS was instituted in 1972 to produce national estimates of the levels and rates of violence and theft, including crime not reported to police departments. In addition to measuring year-to-year change, the NCVS also provides information on the characteristics of victims and offenders and the consequences of crime. While the survey’s core methodology has been validated over the past 3 decades, the viability of the survey and its ability to meet the goals it was created to meet are threatened by the increasing costs of conducting the survey in a period of fiscal austerity. To develop an agenda for reconfiguring the survey and restoring its ability to effectively measure the extent and consequences of crime in the U.S., the Bureau of Justice Statistics (BJS) sponsored an expert panel study. The Committee on National Statistics (CNSTAT) of the National Research Council reviewed the survey’s methodology and provided recommendations for redesigning the NCVS. The panel’s initial recommendations are contained in: Surveying Victims: Options for Conducting the National Crime Victimization Survey (National Research Council 2008) which can be downloaded online at http://www.nap.edu/-catlog.php?record_id=12090.

To address these recommendations, BJS has awarded six initial cooperative agreements to conduct research in five areas: subnational estimates, sampling, mixed-mode data collection, reference period, and nonresponse bias analysis. The studies pertaining to subnational estimates, sampling, and nonresponse bias will be conducted using existing data. The projects to examine mixed mode data collection and means for improving recall for a 12-month reference period will require additional data collection. In addition, other research projects to support the NCVS-RR, such as exploring a modified screener, will be identified within the 3 years covered by this clearance request. Specifications of these yet-to-be-defined projects will be provided as they become available.

BJS is seeking a generic clearance to permit pretesting, developmental work, and data collection to examine the impact of using a longer reference period, using mixed mode data collection, and reducing the screener length and for future projects to be conducted as part of this effort. The planned projects respond to the CNSTAT recommendations and are briefly described below.

CNSTAT Panel Recommendation 4.2: Changing from a 6-month reference period to a 12-month reference period has the potential for improving the precision per-unit cost in the NCVS framework, but the extent of loss of measurement quality is not clear from existing research based on the post-1992-redesign NCVS instrument. BJS should sponsor additional research—involving both experimentation and analysis of the timing of events in extant data—to inform this trade-off.

• Study area: reference period (awarded to NORC)

The objective of this project is to complete a feasibility study examining the use of a 12-month reference period in the NCVS. NORC will perform the following tasks:

1. Develop methods for improving event recall and anchoring, using event calendaring and cues to stimulate recall of violent and property offenses measured in the NCVS (for use in both telephone and personal interviewing formats);

2. Design and carry out a study to test the recall improvement methods developed;

3. Conduct testing of the protocols developed for improving recall over a 12-month reference period; and

4. Assess the burden on respondents, ease of use, and effectiveness of the use of the developed recall improvement techniques for measuring violent and property crimes over a 12-month reference period.

CNSTAT Panel Recommendation 4.8: BJS should investigate the introduction of mixed mode data collection (including self-administered modes) into the NCVS.

• Study area: interview mode (awarded to RTI)

Under this award, RTI will conduct the following parallel studies:

1. An experiment using different data collection modes (self-administered Web interview, computer-assisted telephone interview (CATI), and computer-assisted personal interview (CAPI));

2. A study of mode bias;

3. An examination of the optimal number of persons to interview in each household;

4. An examination of the feasibility of using address-based samples to collect data in different modes; and

5. A pilot test of informed consent/assent procedures.

• Study area: interview mode (awarded to Westat)

Westat will perform the initial following tasks:

1. Initial contact by mail with self-administered data collection comparing the Web and interactive voice response (IVR) technology

2. Initial contact by telephone and comparing computer-assisted telephone interviews (CATI) with self-administered data collection using the Web or interactive voice response (IVR) technology

3. Initial contact in-person and data collected using computer-assisted personal interviews (CATI)

Additional study areas identified by BJS:

• Study areas: as yet unspecified

BJS expects to conduct other methodological studies over the 3 years covered by this request. The nature and specifications of these studies have not yet been determined. All activities undertaken will, however, be designed to enhance the coverage, efficiency, or efficacy of the NCVS.

BJS requests approval for a generic clearance to conduct methodological research to support the NCVS. Supporting materials will be submitted to OMB for approval for each individual study as they are developed for pretesting and developmental activities related to the initial research (NORC-reference period, RTI-mode, Westat-mode and screener modification) and future research activities under the NCVS-RR. It is anticipated that the initial activities will be conducted over an 18-month period from July 2009 through December 2010. Additional studies may be undertaken in FY10 and FY11. Data from these studies will be available as methodological reports at the conclusion of research.

Approvals from the appropriate Institutional Review Boards (IRB) will be obtained prior to conducting any testing under this clearance. In turn, BJS will submit to OMB each pretesting request describing the sample size, the methodology and study design, and all data collection materials. BJS will also submit an annual summary to OMB that describes all testing activities conducted under this clearance. The first summary should be submitted by September 30, 20010, for the period covering the effective date of this approval through June 30, 2010. The second and third summaries should be submitted by September 30 of 2011 and 2012 covering the July 1 – June 30 time frame.

The Omnibus Crime Control and Safe Street Act of 1968 (see attachment) as amended (42 U.S.C. 3732) authorizes BJS to collect and disseminate statistical data on all aspects of criminal justice occurring in the United States, including criminal victimization. The methods proposed for pretesting and developmental work are as follows:

a) Focus groups

This is a qualitative method that uses a small group of individuals to discuss specific topics of interest to the research identified prior to the meeting. A protocol for the focus group is designed and used by a trained facilitator to help guide discussions. Focus groups are useful to explore new areas for surveys.

b) Cognitive interviews

This method involves intensive, one-on-one interviews with respondents. Respondents are instructed to “think aloud” and describe what mental processes are functioning as he or she answers survey questions. Several different variations of cognitive interviewing can occur including asking respondents to paraphrase questions, asking follow-up probing questions to determine how the respondent decided on a particular answer, and asking the respondent’s opinion of what the researcher’s intent for a given question or response option. The objective is to identify ambiguous terminology or other confusing wording construction that present respondents with comprehension problems. This is frequently the first stage of questionnaire development.

c) Pretests

This stage of research involves testing a survey instrument before it is fielded on a large scale. The purpose of pretesting is to identify any errors in the instrument such as skip patterns or difficulty administered under simulated conditions. Pretesting may often be an iterative process until the researcher concludes that the instrument is in its best possible form and ready to be fielded.

d) Field tests

These tests will use respondents to test procedures or questionnaires that may be used in future victimization surveys. For purposes of this clearance, a field test is defined as a small data collection with a sample large enough to detect differences but smaller than a national sample used to produce substantive estimates. These sample sizes will vary by research project and how many number of treatment comparisons, but will generally be limited to about 500 cases or less.

Field tests conducted under this clearance will involved purposive or statistically representative samples. Under this clearance a variety of surveys will be tested, and the exact nature of the surveys and the samples mostly undetermined at present.

The purpose of the field test is to evaluate the questionnaire and the overall survey procedures supporting the data collection. BJS intends to publish methodological papers that discuss field tests conducted under the NCVS redesign program and will ensure that published reports do not identify individual respondents. These reports will also state that the results do not represent official statistics.

e) Split ballot tests

Split ballot tests refer to controlled experimental designs in which different modes or procedures questionnaire versions are administered to different portions of the sample for the purpose of determining which methodologies or question wordings are superior. BJS intends to publish methodological papers that discuss field tests conducted under the NCVS redesign program but will ensure that published reports do not identify individual respondents.

2. Needs and Uses

The purpose of this generic clearance request is to obtain approval to conduct developmental work necessary to begin data collection under the “Methodological Research to Support the Redesign of the National Crime Victimization Survey (NCVS)” grant solicitation in addition to other activities that may be defined over the 3 years covered by this clearance. As such, there are no uses of any data collected under this clearance and program other than to refine and revise the survey questionnaires and procedural designs. The results from the pretests are anticipated to have a minimal impact on the privacy of the respondents, as their participation is confidential and voluntary. Testing and development is needed in order to adequately assess methodological changes being considered for the redesign of the NCVS.

Once the research data collections have been developed, the data that are collected under the initial projects will be used to make decisions about which combination of modes to use when the NCVS is redesigned and which techniques can improve recall by increasing the reference period from 6 to 12 months. Future projects will develop improvements to the questionnaire and other survey protocols. Data from these projects will be included only in Methodological Reports published by BJS. Similar publications were produced during the development of the early National Crime Survey and subsequent to the NCVS redesign in 1992. Data collected under generic clearances will not be used to calculate substantive results/estimates that will be released outside the agency.

Users of these data include the following:

• U.S. Department of Justice – The Bureau of Justice Statistics is the sponsor of the NCVS data collection. Staff at the BJS will work with this data closely to make decisions about the future design of the NCVS.

• Office of Management and Budget (OMB) – OMB shall be given access to results of these studies to help BJS make a decision regarding funding of the NCVS.

• Expert panels – BJS will work in concert with expert panels to be convened at critical decision points during the redesign phase of the NCVS.

3. Use of Technology

One of goals of the research under this generic clearance is to reduce the cost incurred to screen for crime victimization, possibly adopting self-administered interview modes. The research activities to be conducted under the initial phase of the research will be carried out using existing screener questions in the NCVS. In cooperation with BJS, a modified incident report will be used by all contractors to minimize respondent burden. One of the research projects under the NCVS-RR will test the impact of different data collection modes. As a result, interactive voice response (IVR), computer assisted personal interviewing (CAPI), and Web-based technology will be used.

4. Efforts to Identify Duplication

This research does not duplicate any other questionnaire design work being done by BJS or any other federal agencies. In each study area, as an element of the statement of work contractors will review the existing survey literature to incorporate current knowledge of survey methodology and not duplicate similar efforts.

The purpose of the research under this clearance is to study the impact of methodological changes being considered for the NCVS. The NCVS is the only ongoing national program that collects data on the incidence, characteristics and consequences of reported and unreported nonfatal property and violent crime.

5. Impact on Small Businesses

This research does not involve small businesses or other small entities. The respondents are noninstitutionalized persons living in households in the United States.

6. Consequences of Less Frequent Collection

This clearance involves developmental work for projects that measure the impact of methodological changes to the NCVS. The developmental work for each project will only be conducted once. Moreover, if these studies are not conducted, it will not be possible to make informed decisions about the redesign of the NCVS.

7. Special Circumstances Influencing Collection

As covered in the CNSTAT report, the NCVS currently faces escalating costs, declining precision, and circumstances that demand research into how to redesign the survey. BJS recognizes the need to collect information in a manner that places minimal burden on each respondent. This data will be collected in a manner consistent with the guidelines in 5 CFR 1320.6.

8. Federal Register Publication and Outside Consultation

The NCVS-RR program has been informed by the CNSTAT panel recommendations. The initial research solicitations under the NCVS-RR, address recommendations identified in the CNSTAT report.

The research under this clearance is consistent with the guidelines in 5 CFR 1320.6. Comments on this data collection effort were solicited in the Federal Register, Vol. 73, No. 231, on Monday, December 1, 2008. No comments were received in response to the information provided.

9. Payment or Gift to Respondents

No payment or gifts will be offered to NCVS-RR respondents. Compensation may be offered to cover a person’s travel and time to participate in focus groups or cognitive testing of questions. The stipends for this compensation will not exceed $75 for focus groups and $50 for cognitive interviews. The clearance packages for specific projects will discuss the amount and rationale for such compensation.

10. Assurance of Confidentiality

BJS and all contractors will hold in confidence any information that could identify an individual according to Title 42, United States Code, sections 3735 and 3789g. All respondents who participate in pretest activities will be given written or oral assurance that the identity of all participants and respondents will be protected as required under Title 42.

Results from data collected as part of the cognitive and pretest interviews will not be linked to individual respondents, either directly or by inference. As required under Title 42 USC, section 3879g, BJS and its data collection agents will take all necessary steps to mask the identity of survey respondents, including suppression of demographic characteristics and other potentially identifying information.

Prior to beginning any pretesting activity under this clearance, BJS and its collection agents will obtain the requisite approvals from the appropriate Institutional Review Boards (IRB) to ensure that the data collection procedures are in compliance with human subjects protection protocols and confidentiality regulations.

11. Justification for Sensitive Questions

The NCVS asks about experiences that may be sensitive for some respondents (e.g. rape and sexual assault). Given the objective of the NCVS — to estimate the amount of victimization in the nation — this is inevitable. The importance of estimating crime levels, as well as the potential value of detailed information about victimization for designing crime prevention strategies, will be explained to any respondent who seems hesitant to answer. We have structured the questions and interviewer application of them to lessen their sensitivity. All respondents have the option of refusing to answer any question.

12. Estimate of Hour Burden

The three-year total estimated respondent burden for NCVS-RR current and future activities is 16,340 person-hours. The largest annual burden falls in the first year with an estimated 6,280 annual burden hours. We estimate that cognitive interviews and pretesting instruments will take an average of 1 hour, except for activities that use event history calendaring. Developing techniques to improve a respondent’s recall when moving from a 6-month to a 12-month reference period may take as long as 2 hours for each cognitive interview. Once the techniques are developed, we estimated using the assistance of event history calendars may take twice as long as other interviews, or 30 minutes. Cognitive interviews for the interview mode projects are estimated to take 1 hour per respondent. Once these procedures have been finalized, we estimate that a screener and abbreviated incident report interview will take an average of 15 minutes. Interviews using event history calendaring techniques are estimated to take 30 minutes.

Burden hours reflect the relative rarity of crime and the need to screen a large number of people to obtain sufficient numbers of victims to enable comparisons across all conditions. Abbreviated screeners will be used to reduce burden. The calculation of burden hours for the NCVS-RR current and future activities are provided below in Figure 1.

Figure 1. Burden Hour Calculation
			Time	Annualized burden				Total
Contractor	Type of interview	# of respondents	(hrs)	Year1	Year2	Year3		burden
Reference period study area
NORC	cognitive interviews for event history calendar	80	2	160				160
NORC	cognitive interviews for abbreviated screener	20	1	20				20
NORC	interview w/event history calendar	3,000	0.5	1,500				1,500
Interview mode study area
Westat	cognitive interviews for abbreviated screener	20	1	20				20
Westat	pretesting IVR	20	1	20				20
Westat	pretesting Web instrument	20	1	20				20
Westat	screen+incident	15,000	0.25	3,750				3,750
RTI	pretesting CAPI (also used for CATI)	20	1	20				20
RTI	pretesting Web instrument	20	1	20				20
RTI	screen+incident	3,000	0.25	750				750
Additional study areas
TBD*	Methodological research TBD	12,000	0.25		3,000			3,000
TBD	Methodological research cognitive interviews (3 rounds of 20)	60	1		60			60
TBD	Methodological research TBD	6,000	0.5		3,000			3,000
TBD	Methodological research TBD	8,000	0.25			2,000		2,000
TBD	Methodological research TBD	8,000	0.25			2,000		2,000
		TOTAL hours by year		6,280	6,060	4,000
* to be determined		TOTAL hours						16,340

13. Estimate of Cost Burden

With the exception of their time, there is no actual cost to respondents to participate in NCVS-RR projects.

14. Estimated Cost to Federal Government

The total annual cost to the federal government for the projects already identified will be approximately $4.2 million. The three currently awarded studies will cost $3.5 million. This estimate includes the costs of study design, data collection, analysis, and report/publication writing. Costs of additional studies in FY10 and FY11 will be determined by the levels of funding available. BJS will provide OMB with updated cost information as plans for additional studies are developed.

In addition to contractor costs for data collection, there will be approximately $220,000 in annual costs, representing BJS costs to manage these studies. The salary figures are calculated using 2009 GS scale rates at step 5 levels to estimate the staff costs over the 3 years of this clearance.

Figure 2. Estimated costs for NCVS-RR
Data collection
	RTI-mode					$ 1,646,348
	NORC-mode					$ 760,509
	Westat-mode					$ 1,100,021
				Contractor subtotal		$ 3,506,878
BJS Costs
	Staff salaries		Base salary (step 5)	Fringe	Salary estimates
		GS15 - Chief, Victimization Statistics Unit (1 @ 25%)	$ 136,941	$ 38,343	$ 43,821
		GS15 - Supervisory statistician for BJS Redesign working group (1 @ 20%)	$ 136,941	$ 38,343	$ 35,057
		GS14 - Senior Statisticians (2 @ 25%)	$ 116,419	$ 32,597	$ 74,508
		GS-13 Statistician (3 @ 10%)	$ 98,518	$ 27,585	$ 37,831
	Subtotal: Salary & fringe				$ 191,217
	Other administrative costs of salary & fringe (15%)				$ 28,683
	Subtotal: BJS costs @ 3 years					$ 659,699
Total estimated costs						$ 4,166,577

15. Reasons for Change in Burden

There are no changes in burden since the NCVS-RR projects are new data collections.

16. Plans for Publication

In calendar years 2009 and 2010, BJS expects to perform all data collection for the contracts to study mixed mode data collection and the 12-month reference period. All work under this clearance is expected to be completed by 2012. At the conclusion of each research project, methodological research papers will be issued by BJS staff. The types of statistics that will be available in these reports include response rates, measures of productivity such as offenses reported per respondent or household, and counts and rates for the purpose of evaluating changes in estimates due to methodology. Data collected under a generic clearance will not be used to calculate substantive results/estimates that will be released outside the agency.

17. Expiration Date Approval

The OMB control number and expiration date will be published on all forms given to respondents. In instances where information is collected electronically (e.g. Web, IVR, CAPI, and CATI), information will be displayed or read describing the nature of the survey and authority to collect the information.

18. Exceptions to the Certification Statement

There are no exceptions to the Certification Statement.

11